[Rivet] [Yoda] YODA development

[Andy Buckley]

Ben Waugh wrote:
> Hi Andy,
> 
> On 02/11/09 14:16, Andy Buckley wrote:
>> The real cases I can think of are MC at NLO, where the generator itself can
>> produce negative weights (although they are already unweighted by
>> modulus) and of some signed observables like AEEC. I'm motivated by
>> wanting to do something which doesn't just die or go crazy when negative
>> weights are encountered: the AEEC observable is a case where they
>> definitely will be.
> 
> What do you mean by "unweighted by modulus"? That the weights are all 
> equal in magnitude but differ in sign?

Yes. It just means that negative weights are allowed, without spending 
lots of time on events of either kind which contribute virtually no 
weight (either positive or negative).

> I don't think that makes much 
> difference in principle, but in practice it makes it easier to think of 
> the final distribution and sample as the difference of two independent 
> subsamples, one containing the positive and the other the negative weights.

>> Yes, that's right. I also realised that hasty mistake, but it also
>> raised the question of what to use for N.
> 
> I think we agree, don't we, that wherever N is a meaningful quantity 
> sum(w) is the appropriate way to calculate it?

Good... I was still vacillating on that, and wasn't sure if you were 
disagreeing :D

>>> If we do get events, then we should still calculate the error on the
>>> bin height to be the usual sum(w^2), regardless of the sign of w. In
>>> our rather pathological toy model where weights are either +1 or -1,
>>> suppose we generate a sample of size such that we expect 2N events in
>>> a particular bin. We would expect in this bin to see N +- sqrt(N) with
>>> weight +1 and N +- sqrt(N) with weight -1, giving a bin height of 0 +-
>>> sqrt(2N).
>>>
>>> Here I am using Poisson rather than binomial statistics. I haven't yet
>>> considered the difference in error calculation depending on whether we
>>> are dealing with known luminosity (but variable number of events) or
>>> known number of events (but variable fraction in each bin).
>>
>> Okay, so I'm now a little uncertain about what stats should be used for
>> bin height errors... this way the error grows with number of events,
> 
> As it should! This is the error on the expected number of events, and 
> grows as the square root of the sample size. The error on the 
> probability density falls as 1/sqrt(N).
> 
>> while the mean remains at zero in the pathological case. I was coming to
>> a conclusion that the binomial error makes sense if calculated as
>> sqrt(sum(w)), with a thrown exception if sum(w) < 1.
> 
> I still disagree with this. This way you could reduce your errors by 
> doubling the weight of all events, just as much as by generating twice 
> the statistics.

Yes, sounds good. But isn't that in contradiction with N = sum(w). 
Should we use N = #fills rather than sum(w) for the binomial/Poisson bin 
height errors? I think this is still okay from a repartitioning 
standpoint... and since the binomial/Poisson stats relate to a number of 
trials I can see this making sense... but a definitive reference would 
be nice.

> If we can't dig out a reference for this we might just have to redo the 
> derivation ourselves!

>> Except where the observable itself is signed... hmm.
> 
> Disagreed! I don't think the sign of the observable makes any difference.

Okay, you're right. The problem here is that (A)EEC is implemented in 
Rivet by use of an extra signed term multiplying the generator event 
weight. If YODA is designed to do this properly, i.e. assuming that 
"weight" implies a statistically meaningful measure (i.e. which will be 
 >= 0 with asymptotic statistics), then the Rivet implementation(s) of 
(A)EEC will need to be updated not to abuse the weight.

Thanks for the discussion, by the way --- I think this has been a really 
useful way to work out what YODA should be doing (and what ROOT, AIDA 
etc. should have been doing, too, so that we didn't have to have this 
discussion at all!)

Cheers,
Andy

-- 
Dr Andy Buckley
SUPA Advanced Research Fellow
Particle Physics Experiment Group, University of Edinburgh

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.