[Rivet] [Yoda] YODA development

[Ben Waugh]

Hi Andy,

On 02/11/09 14:16, Andy Buckley wrote:
> The real cases I can think of are MC at NLO, where the generator itself can
> produce negative weights (although they are already unweighted by
> modulus) and of some signed observables like AEEC. I'm motivated by
> wanting to do something which doesn't just die or go crazy when negative
> weights are encountered: the AEEC observable is a case where they
> definitely will be.

What do you mean by "unweighted by modulus"? That the weights are all 
equal in magnitude but differ in sign? I don't think that makes much 
difference in principle, but in practice it makes it easier to think of 
the final distribution and sample as the difference of two independent 
subsamples, one containing the positive and the other the negative weights.

>> On that topic, I can't see why it is necessary to use a "distribution"
>> object (Dbn1D) [...]
>
> It is something which became a very useful centralising design feature
> when I started writing multiple classes that had to handle similar
> statistics. I don't have a plan to make any 2D profile classes or
> similar, for which a more extended object would be needed, so it's
> really just a convenience feature... I would certainly like to keep it
> at least for design/implementation/maintenance reasons.
>
> However, unless I'm being dense, storing various moments is required to
> get the whole-distribution moments correct under rebinning: it allows
> the shape of the distribution within the bins to be stored.

Yes, that is true. If whole-distribution moments are needed then this 
does look like a nice way of doing it.

> Yes ;) I'm inclined to throw exceptions if statistical quantities are
> requested from any distribution with sum(w) < 1

Unless the generator or underlying theory is really pathological, 
sum(w)<0 is a sign of insufficient statistics combined with a bit of bad 
luck. All the usual sqrt(N) or sqrt(sum(w^2)) stuff is an approximation 
that relies on good enough stats anyway, so perhaps throwing an 
exception is a reasonable response in this case, as it would be for 
division by 0.

But I think 0<sum(w)<1 should be handled in the same way as any other 
positive but small N. A small sum(w) could result e.g. from a large 
number of low-weighted events in a region that has a small cross section 
but has been densely sampled because it is of particular interest.

> Yes, that's right. I also realised that hasty mistake, but it also
> raised the question of what to use for N.

I think we agree, don't we, that wherever N is a meaningful quantity 
sum(w) is the appropriate way to calculate it?

>> If we do get events, then we should still calculate the error on the
>> bin height to be the usual sum(w^2), regardless of the sign of w. In
>> our rather pathological toy model where weights are either +1 or -1,
>> suppose we generate a sample of size such that we expect 2N events in
>> a particular bin. We would expect in this bin to see N +- sqrt(N) with
>> weight +1 and N +- sqrt(N) with weight -1, giving a bin height of 0 +-
>> sqrt(2N).
>>
>> Here I am using Poisson rather than binomial statistics. I haven't yet
>> considered the difference in error calculation depending on whether we
>> are dealing with known luminosity (but variable number of events) or
>> known number of events (but variable fraction in each bin).
>
> Okay, so I'm now a little uncertain about what stats should be used for
> bin height errors... this way the error grows with number of events,

As it should! This is the error on the expected number of events, and 
grows as the square root of the sample size. The error on the 
probability density falls as 1/sqrt(N).

> while the mean remains at zero in the pathological case. I was coming to
> a conclusion that the binomial error makes sense if calculated as
> sqrt(sum(w)), with a thrown exception if sum(w) < 1.

I still disagree with this. This way you could reduce your errors by 
doubling the weight of all events, just as much as by generating twice 
the statistics.

If we can't dig out a reference for this we might just have to redo the 
derivation ourselves!

> Yes, and I don't think regions of phase space with overall negative
> weights can have a very meaningful probabilistic interpretation either.

Agreed.

> Except where the observable itself is signed... hmm.

Disagreed! I don't think the sign of the observable makes any difference.

>> However, I think this particular toy case (if I understand it
>> correctly) is actually an exception. If all events have x=1,
>> regardless of weight, then we can still say that x=1 is a good
>> estimate of the true value, if there is such a thing. Even for the
>> sample with w = -1, the mean x is 1, not -1. mean(x) = sum(wx)/sum(w)
>> = (-n)/(-n) = 1.
>
> Yep, that's why I think N=sum(w) works okay... once my <w^2 O^2> -> <w
> O^2> booboo is sorted.

Good.

Cheers,
Ben

-- 
Dr Ben Waugh                                   Tel. +44 (0)20 7679 7223
Dept of Physics and Astronomy                  Internal: 37223
University College London
London WC1E 6BT