[Rivet] question to yodamerge

[Hannes Jung]

Hi Frank

ahh, thanks a lot... this is what I was suspecting...
yes, it is from a process which ahs a very small xsection...
so I perhaso just need to increase the statistics 

Thanks a lot for pointing this out
CHeers
hannes

> On 14.04.2015, at 10:30, Frank Siegert <Frank.Siegert at cern.ch> wrote:
> 
> Hi Hannes,
> 
> I see what happens. Your Job_15.yoda has basically no entries, only in
> some of the histograms you seem to have a single event, and in the
> normalized ones from CMS_2013_I1256943 there is nothing. Obviously, to
> normalize an empty histogram does not make sense, so YODA simply
> ignores the normalize step and thus doesn't add the "ScaledBy"
> information to the histo in Job_15. The yodamerge script relies on
> that to merge them by undoing the normalisation, then adding, and then
> redoing a combined normalisation.
> 
> If there is no way for you to make sure that you have at least one
> event in each histogram that is to be normalized, then a workaround
> would be to simply copy the ScaledBy line to the empty histograms --
> it won't matter what number there is since it is going to be
> multiplied with 0 anyway as far as I can tell.
> 
> Cheers,
> Frank
> 
> 
> On 14 April 2015 at 10:10, Hannes Jung <hannes.jung at desy.de> wrote:
>> Hi Frank
>> 
>> I have put 2 yoda files, where the error when using yodamerge
>> happens under
>> https://www.desy.de/~jung/rivet-checks/
>> 
>> Thanks a lot
>> Cheers
>> Hannes
>> 
>>> On 14.04.2015, at 09:49, Frank Siegert <Frank.Siegert at cern.ch> wrote:
>>> 
>>> Hi Hannes,
>>> 
>>> Ah, of course... the normalize also uses a scale in the end, so in
>>> principle the yodamerge should even work for normalized histos. I
>>> don't quite understand why there is no ScaledBy annotation in that
>>> case. Can you put the yoda files that come out of your individual runs
>>> somewhere for us to look at?
>>> 
>>> Cheers,
>>> Frank
>>> 
>>> 
>>> 
>>> On 14 April 2015 at 09:44, Hannes Jung <hannes.jung at desy.de> wrote:
>>>> Hi Frank
>>>> 
>>>> thanks a lot for your reply.
>>>> 
>>>> I fully agree with your Note 1 .... it took me quite a bit
>>>> to find out what to do to get the merging....
>>>> 
>>>> The funny thing with this particular histo is, that yodamerge
>>>> works for files generated with one process, but gives the
>>>> warning for another process.... can it be, that there is something
>>>> with the number of entries in the histo or soemthing similar ?
>>>> 
>>>> Cheers
>>>> Hannes
>>>> 
>>>> 
>>>>> On 14.04.2015, at 09:35, Frank Siegert <Frank.Siegert at cern.ch> wrote:
>>>>> 
>>>>> Hi Hannes,
>>>>> 
>>>>> this particular histogram seems to be normalised to a given integral
>>>>> (which I don't claim to understand:
>>>>> 0.7*crossSection()*_sumW/sumOfWeights()) instead of scaled
>>>>> proportional to the cross section like dsigma/dX. This means there is
>>>>> a priori no clear prescription of how they can be merged from
>>>>> different runs, and thus the simple yodamerge script fails. If there
>>>>> is some way to merge these objects even though they are normalised
>>>>> then you have to write a small Python script yourself which does it.
>>>>> 
>>>>> Note 1: The --assume-normalized switch has an unfortunate naming and
>>>>> has been a constant source of confusion for users. This has been
>>>>> discussed before, and I'll state my opinion again:
>>>>> - the switch should be the default behaviour of yodamerge (because
>>>>> most of our distributions are of the dsigma/dX type which need it)
>>>>> - there should be two options to overwrite (/replace) the broken
>>>>> heuristic of whether something should be merged or added:
>>>>> --force-merge (if you have multiple identical runs from different
>>>>> random seeds) vs. --force-add (if you have multiple samples that have
>>>>> to be added, e.g. electron and muon channel, Np0...Np4, ...).
>>>>> 
>>>>> Note 2: On the proper and automatic run combination for arbitrary
>>>>> objects (including arbitrarily normalised histograms) we have this
>>>>> ticket:
>>>>> https://rivet.hepforge.org/trac/ticket/457
>>>>> 
>>>>> Cheers,
>>>>> Frank
>>>>> 
>>>>> 
>>>>> On 14 April 2015 at 06:57, Hannes Jung <hannes.jung at desy.de> wrote:
>>>>>> Dear Rivet experts
>>>>>> 
>>>>>> i use yodamerge to add yoda files from stat independent runs
>>>>>> with
>>>>>> 
>>>>>> yodamerge --assume-normalized  .....
>>>>>> 
>>>>>> now in some cases I get the warning:
>>>>>> 
>>>>>> WARNING: Abandoning normalized merge of path /CMS_2013_I1256943/d01-x01-y01 because not all inputs have ScaledBy attributes
>>>>>> 
>>>>>> and the results are weird.
>>>>>> 
>>>>>> What is wrong there, or what could I check ?
>>>>>> Thanks a lot
>>>>>> cheers
>>>>>> Hannes
>>>>>> 
>>>>>> ***********************************************************************
>>>>>> Hannes Jung
>>>>>> Email: Hannes.Jung at desy.de
>>>>>> mobile :+49 40 8998 93741
>>>>>> http://www.desy.de/~jung
>>>>>> Tel: +49 (0) 40 8998 3741
>>>>>> Fax: +49 (0) 40 8994 3741
>>>>>> DESY, CMS 01B/02.213
>>>>>> Notkestr.85, 22603 Hamburg, FRG
>>>>>> ***********************************************************************
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Rivet mailing list
>>>>>> Rivet at projects.hepforge.org
>>>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>> 
>>>> ***********************************************************************
>>>> Hannes Jung
>>>> Email: Hannes.Jung at desy.de
>>>> mobile :+49 40 8998 93741
>>>> http://www.desy.de/~jung
>>>> Tel: +49 (0) 40 8998 3741
>>>> Fax: +49 (0) 40 8994 3741
>>>> DESY, CMS 01B/02.213
>>>> Notkestr.85, 22603 Hamburg, FRG
>>>> ***********************************************************************
>>>> 
>>>> 
>>>> 
>> 
>> ***********************************************************************
>> Hannes Jung
>> Email: Hannes.Jung at desy.de
>> mobile :+49 40 8998 93741
>> http://www.desy.de/~jung
>> Tel: +49 (0) 40 8998 3741
>> Fax: +49 (0) 40 8994 3741
>> DESY, CMS 01B/02.213
>> Notkestr.85, 22603 Hamburg, FRG
>> ***********************************************************************
>> 
>> 
>> 

***********************************************************************
Hannes Jung 
Email: Hannes.Jung at desy.de 
mobile :+49 40 8998 93741
http://www.desy.de/~jung                                  
Tel: +49 (0) 40 8998 3741         
Fax: +49 (0) 40 8994 3741
DESY, CMS 01B/02.213
Notkestr.85, 22603 Hamburg, FRG   
***********************************************************************