[Rivet] YODA development

[Andy Buckley]

Ben Waugh wrote:
> On 30/10/09 11:47, Andy Buckley wrote:
>> I'm not sure... I looked in the stats literature quite a lot to find a 
>> definitive treatment of weighted stats combination (especially where 
>> negative weights are concerned) and found nothing, hence the bullet 
>> points are my invention. Perhaps it's sensitive to how the weights are 
>> determined as to whether high weights are statistically equivalent to 
>> many small weights, but I'm inclined to agree that my bullet points 
>> aren't the whole story and that we need a more careful treatment. 
>> Fortunately, the code in YODA that handles this for all histo types is 
>> very localised! (in Dbn1D)
>>
>> Any suggestions of alternative weighted combination recipes, which 
>> consistently handle positive and negative weights? Or, better, has 
>> anyone found any papers or books that address this issue?
> 
> I remember a useful preprint I used in my H1 days that gave some useful 
> recipes and derivations, but of course I can't find it now. No luck 
> finding anythin more authoritative either.
> 
> My recollection/understanding is that the "standard" recipe actually 
> does work for negative weights: (1+-1) - (1+-1) = (0 += sqrt(2))...

The YODA approach is to do every stat combination based on storing 
weighted moments of the variable being filled, i.e. w, w^2, wx, wx^2, etc.,

Just thinking about this now, it seems to me that the best approach is 
to use var(O) = <(wO)^2> - <wO>^2, i.e. we also need to store w^2x^2. I 
can't remember right now if this is what I'm already doing! Then if I 
fill a single histo with alternating weights of +/- 1 for N fills, then 
the variance on the bin height (O=1) will be N/N - 0/N = 1... so the 
error never decreases. Reasonable? Using this treatment is of course 
nice because it means that the combined result is invariant under 
arbitrary re-partitionings of the fills... that seems the key property 
to me.

Thoughts?

Andy

-- 
Dr Andy Buckley
SUPA Advanced Research Fellow
Particle Physics Experiment Group, University of Edinburgh

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.