[Rivet] DISLepton Class fails to find outgoing lepton

[Andy Buckley]

Thanks for your patience, Marian.

My reading of the Rivet 2.5.3 code (copied below) is that it selects
the highest-|pz| final-state lepton. Looking back through my emails I
see that Andrii's concern about a final-state definition was actually
about using highest-pT, i.e. not the same thing. DIS experts, is
max-|pz| a reasonable heuristic?

Also, if the concern about contamination was from hadron-decay sources
of extra leptons, we can eliminate those safely by using a
PromptFinalState rather than a bare FinalState. If we were to do this,
the contamination would only be from rare events which have multiple
prompt/direct final-state leptons from the hard process... and then we
could refine by e.g. preferring neutral current if possible, or by a
kinematic preference like max-|pz|. Any thoughts? This would
definitely be nicer than the current graph walking, which I've now
reverse-engineered and it is fragile.

Best wishes,
Andy


CODE:

    double pzMax = -1000000000.0;
    const FinalState& fs = applyProjection<FinalState>(e, "FS");
    foreach (const Particle& p, fs.particles()) {
      double pz = _sign * p.momentum().pz();
      if (p.pid() == id && pz > pzMax) {
        _outgoing = p;
        pzMax = pz;
      }
    }

On 16 November 2017 at 10:05, Marian Heil <marian.heil at durham.ac.uk> wrote:
> Hi Andy,
>
> I just tested the tree-like HepMC option, but it does not work with Sherpa
> 2.2.4. It still produces a loop in the HepMC file (see attachment). My
> current solution is to roll back to the Rivet 2.5.3 code for DISLeptons ;-).
> For me it is unexpected that rivet relies on the full event history in its
> analysis, which is kind of artificial for MC events and would not be
> possible in experiments. But I am far from an expert in DIS, so I do not
> know what should happen in multi lepton final state (which I guess is the
> main reasoning from using the HepMC record).
>
> Thanks,
> Marian
>
>
> On 15/11/17 19:50, Andy Buckley wrote:
>>
>> Hi Marian,
>>
>> Sorry for the noise while we work out what should happen to this
>> projection in the long-term!
>>
>> Did you get a chance to run with Sherpa in the tree-like-HepMC output
>> mode? That seems likely to be the most immediate fix for you, although
>> I hope we can get a compatibility improvement into a future release.
>>
>> Andy
>>
>>
>> On 15 November 2017 at 12:15, Andrii Verbytskyi <andriish at mpp.mpg.de>
>> wrote:
>>>
>>> Hi David,
>>>
>>> in principle yes, but not in the nearest future.
>>> One can ask some students to do it.
>>>
>>> Andrii
>>>
>>> On Wed, 2017-11-15 at 11:44 +0000, David Grellscheid wrote:
>>>>
>>>> Andrii, can you please provide a test plot some time, showing the old
>>>> and new behaviour of the projection before/after your fix in some
>>>> exemplary analysis? I'm trying to assemble a set of interface stability
>>>> tests (also for the XYZFinders), and this would be a good addition.
>>>>
>>>> Thanks,
>>>>
>>>>    David
>>>>
>>>>
>>>> On 15/11/2017 11:38, David Grellscheid wrote:
>>>>>
>>>>> Do I understand right that the answer to the "correct" lepton to pick
>>>>> becomes analysis-specific, and a global projection is not suited for
>>>>> everyone.
>>>>>
>>>>> If the experimental analysis folds in some correction for the fact that
>>>>> you may not have identified the "real" DIS lepton experimentally, then
>>>>> going through the event tree in Rivet is (grudgingly ;-) ) acceptable.
>>>>>
>>>>> If the analysis relies on a final-state definition of which lepton to
>>>>> choose, then Rivet should use the same final-state definition, and
>>>>> would
>>>>> not need to look at event trees.
>>>>>
>>>>> See you,
>>>>>
>>>>>    David
>>>>>
>>>>>
>>>>> On 14/11/2017 18:32, Andrii Verbytskyi wrote:
>>>>>>
>>>>>> Hi Andy,
>>>>>>
>>>>>>
>>>>>> 1) the code for electron finding works for me fine.
>>>>>> Maybe because I use HEPMC_TREE_LIKE=1 for Sherpa. Maybe for other
>>>>>> reasons. That is just an observation. I suggest Marian will use it and
>>>>>> skip events with no lepton/broken lepton. Another option is to ignore
>>>>>> state=3 particles in the finder.
>>>>>>
>>>>>> 2) I'm sure there are cases where your arguments are valid, but
>>>>>> as of 2017, (simple NC) DIS is well defined in PDG.
>>>>>> http://pdg.lbl.gov/2017/reviews/rpp2016-rev-qcd.pdf
>>>>>> Not the case you describe. Of course that might be different next
>>>>>> year,
>>>>>> but I see no reasons for that and do not know anything about
>>>>>> theoretical
>>>>>> advances that can change it.
>>>>>>
>>>>>> 2a) Yes, final state definition is a good thing. That is why I find
>>>>>> the
>>>>>> fact that Sherpa puts final state particles into hadronisation blob
>>>>>> problematic. But that is an implementation, not definition.
>>>>>>
>>>>>> Best regards,
>>>>>> Andrii
>>>>>>
>>>>>>
>>>>>> On Tue, 2017-11-14 at 18:01 +0000, Andy Buckley wrote:
>>>>>>>
>>>>>>> Hi Andrii,
>>>>>>>
>>>>>>> I'm not sure what you mean by your point 1)...?
>>>>>>>
>>>>>>> On (2), I'm sure you're aware that the evoution of experimental
>>>>>>> well-definedness has been gently evolving. There are also areas like
>>>>>>> precision EW physics where definitions long held to be solid are
>>>>>>> being
>>>>>>> challenged by theoretical advances. Also, HepMC and associated
>>>>>>> standards are the operative definition of well-definedness: if those
>>>>>>> standards are currently incompatible with a theoretically
>>>>>>> well-defined
>>>>>>> process, then we should revisit the standards. "The lepton momentum
>>>>>>> that interacts with a W/Z" is the sort of definition that has caused
>>>>>>> problems, however, hence the ongoing shift towards more final-state
>>>>>>> oriented fiducial definitions.
>>>>>>>
>>>>>>> I had no idea that the distinction between this definition and the
>>>>>>> former one was so strong... I thought we were talking about more like
>>>>>>> a few percent. Obviously we need to find a balance.
>>>>>>>
>>>>>>> Andy
>>>>>>>
>>>>>>>
>>>>>>> On 14 November 2017 at 17:53, Andrii Verbytskyi
>>>>>>> <andrii.verbytskyi at desy.de> wrote:
>>>>>>>>
>>>>>>>> Hi Andy,
>>>>>>>>
>>>>>>>> I had a long day, so in short:
>>>>>>>> 1) works for me. With Sherpa as well.
>>>>>>>> 2) DIS is well defined  since many years, in many experiments and MC
>>>>>>>> programs. Regardless of Rivet, HepMC, you and me. Sad but true.
>>>>>>>> 3) Last time "some accuracy" was 50% or so (I honestly do not
>>>>>>>> remember).
>>>>>>>> Not a good idea. For sure there will be a bunch of students that
>>>>>>>> will
>>>>>>>> not be aware (or will forget) that "some accuracy" is missing.
>>>>>>>>
>>>>>>>> Andrii
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, 2017-11-14 at 17:32 +0000, Andy Buckley wrote:
>>>>>>>>>
>>>>>>>>> Unfortunately that DIS definition doesn't correspond to something
>>>>>>>>> well-defined in HepMC. There is no standard for hard-process
>>>>>>>>> interaction representation.
>>>>>>>>>
>>>>>>>>> It would be good to know if this works with the Sherpa tree-like
>>>>>>>>> mode,
>>>>>>>>> but generally we want to avoid such sensitivities. I am more
>>>>>>>>> inclined
>>>>>>>>> to sacrifice some accuracy
>>>>>>>>
>>>>>>>>
>>>>>>>>> for robustness... and an observable
>>>>>>>>> definition that cannot be reproduced is a fundamentally problematic
>>>>>>>>> thing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Andy
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 14 November 2017 at 17:05, Andrii Verbytskyi
>>>>>>>>> <andrii.verbytskyi at desy.de> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Marian, Andy,
>>>>>>>>>>
>>>>>>>>>> 0) The event contains loops. It is better to generate events with
>>>>>>>>>> Sherpa
>>>>>>>>>> with HEPMC_TREE_LIKE=1 or so. See docs.  Have you used it?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1)
>>>>>>>>>> On Tue, 2017-11-14 at 16:43 +0000, Andy Buckley wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Andrii,
>>>>>>>>>>>
>>>>>>>>>>> If the particle status codes are not 1 or 2, there is no
>>>>>>>>>>> guarantee
>>>>>>>>>>> that HepMC vertices correspond to physical processes. They may
>>>>>>>>>>> just be
>>>>>>>>>>> bookkeeping devices, e.g. to absorb parton shower recoils (or in
>>>>>>>>>>> this
>>>>>>>>>>> case perhaps the QED radiation treatment) -- this wouldn't be a
>>>>>>>>>>> Sherpa
>>>>>>>>>>> bug, but a valid use of the freedoms in the HepMC standard. So we
>>>>>>>>>>> can't have projection code that assumes particular vertex
>>>>>>>>>>> structures,
>>>>>>>>>>> because there will always be such edge-cases.
>>>>>>>>>>
>>>>>>>>>> In general you are right, but not for electron in DIS that is
>>>>>>>>>> coming
>>>>>>>>>> from hard process. The only thing one expects from it is
>>>>>>>>>> e->e+gamma  or
>>>>>>>>>> e-> W nu ( not in Rivet anyway). But no hadronisation vertices.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sherpa mixes everything together, in one vertex.
>>>>>>>>>> I haven't said that is a bug, but a *problem*. A problem for
>>>>>>>>>> kinematics
>>>>>>>>>> reconstruction.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Can you explain what the logic of your DIS lepton finder is? (The
>>>>>>>>>>> code
>>>>>>>>>>> is not super-easy to follow.) Hopefully an understanding of the
>>>>>>>>>>> intention will help us to find a safer definition.
>>>>>>>>>>
>>>>>>>>>> The logic comes from  DIS definition: scattered lepton is the beam
>>>>>>>>>> electron that went through e->e gamma/Z0 or e-> W nu vertices.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Andrii
>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Andy
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 14 November 2017 at 16:32, Andrii Verbytskyi
>>>>>>>>>>> <andrii.verbytskyi at desy.de> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Marian, Andy,
>>>>>>>>>>>>
>>>>>>>>>>>> 0) I see no attachment. It is hard to guess which kind of
>>>>>>>>>>>> process will
>>>>>>>>>>>> produce electron+ some leptons from a single electron.
>>>>>>>>>>>> If that is not SM process, the code was not designed to handle
>>>>>>>>>>>> BSM.
>>>>>>>>>>>> 1) Yes, it is complicated with Sherpa. It merges too many things
>>>>>>>>>>>> together in one vertex. Just skip events with no electron/bad
>>>>>>>>>>>> electron.
>>>>>>>>>>>> 2) Everything "works" in "some way" with some conditions.
>>>>>>>>>>>> 3) I cannot say without looking at the event that this is Sherpa
>>>>>>>>>>>> problem, but I suspect it is, so it is not clear if any fix is
>>>>>>>>>>>> needed.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Andrii
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, 2017-11-14 at 15:06 +0000, Andy Buckley wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Marian,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the report. I've CC'd Andrii, who provided that
>>>>>>>>>>>>> updated
>>>>>>>>>>>>> DISLepton logic. Andrii, can you comment on how we can fix this
>>>>>>>>>>>>> behaviour in a generator-independent way?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Andy
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 10 November 2017 at 16:32, Marian Heil
>>>>>>>>>>>>> <marian.heil at durham.ac.uk> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dear rivet authors,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think I found a bug in the DISLepton class in rivet version
>>>>>>>>>>>>>> 2.5.4: It does
>>>>>>>>>>>>>> not find the outgoing lepton for a DIS scattering generated
>>>>>>>>>>>>>> with Sherpa (the
>>>>>>>>>>>>>> corresponding HepMC file is attached).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As far as I understand it in DISLepton::project the iteration
>>>>>>>>>>>>>> over all
>>>>>>>>>>>>>> vertices and fails on the vertex "-6", because there are two
>>>>>>>>>>>>>> outgoing
>>>>>>>>>>>>>> leptons ("10009" and "10015"). The first electron loops over
>>>>>>>>>>>>>> vertex "-5"
>>>>>>>>>>>>>> back into vertex "-6" (for what ever reason), so it is not an
>>>>>>>>>>>>>> actual final
>>>>>>>>>>>>>> state particle. The old code from version 2.5.3 does actually
>>>>>>>>>>>>>> work for the
>>>>>>>>>>>>>> event.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>> Marian
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Rivet mailing list
>>>>>>>>>>>>>> Rivet at projects.hepforge.org
>>>>>>>>>>>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Rivet mailing list
>>>>>> Rivet at projects.hepforge.org
>>>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>>>
>>>>> _______________________________________________
>>>>> Rivet mailing list
>>>>> Rivet at projects.hepforge.org
>>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>>
>>>
>>
>>
>



-- 
Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Expt Group, University of Glasgow