[Rivet] HepData consistency

[Andy Buckley]

On 05/10/16 08:48, David Grellscheid wrote:
>> My understanding is that the new HepData will have some sort of
>> versioning. Maybe you could check with Graham Watt and make sure that
>> there will be stable URLs for accessing the different versions?
>
> I'll try and find him today and see what we can do. I'll also ask about
> the shifting d-x-y labelling. If their beta page is stable enough, I'll
> suggest that we can sync against that in future, and make use of new
> features already.

Sounds good.

> With the stable URLs, do you suggest that Rivet points at a fixed one,
> and at a controlled point switches over to a newer version, but freeze
> that again? That sounds much sner than a constantly shifting reference
> to whatever the newest version is.

Yes, that sounds reasonable. Still need to figure out how we become 
aware of the updates -- if it were an unstable URL we'd just re-pull 
from HepData with each release and look for diffs -- but it's a better 
position than the current one.

Andy


> On 05/10/2016 08:39, Andy Buckley wrote:
>> On 05/10/16 08:02, David Grellscheid wrote:
>>> Hi all,
>>>
>>> What a lovely mess. I was starting to miss them!
>>
>> :-D
>>
>>> This needs to be fixed at so many levels. In Rivet, how about we do the
>>> following, if it's technically feasible:
>>>
>>> * YODA files must match Hepdata, to allow for replayability
>>
>> Yes. By hook or by crook: this is the only way to sanity. How are we
>> going to get there?
>>
>>> * In the plot config file, apply the scaling through a factor, suitably
>>> commented
>>
>> I don't think this is sufficient: we want to be able to uniformly use
>> datasets for numerical processing (e.g. chi2 or likelihood computation)
>> as well as plotting. Histograms that plot with good data agreement but
>> mysteriously produce bad chi2s are a bit too confusing and magical for
>> my liking...
>>
>>> Independently, Hepdata needs to find a way to issue v2 datasets that are
>>> corrected. I see the point of keeping the original ones around, to match
>>> the publication, but having the newest correction available as full data
>>> would be useful.
>>
>> My understanding is that the new HepData will have some sort of
>> versioning. Maybe you could check with Graham Watt and make sure that
>> there will be stable URLs for accessing the different versions?
>>
>> Thanks,
>> Andy
>>
>>
>>
>>> On 04/10/2016 22:56, Christian Gutschow wrote:
>>>> As far as I’m aware — if it’s an actual erratum, then the HEPData
>>>> tables will be fixed. If however — for example — a more precise
>>>> luminosity calibration becomes available subsequently (which would
>>>> only shifts the central values by the same amount), then usually a
>>>> note is added to the description of the HEPData entry, but this change
>>>> isn’t propagated through to the tables. The logic (coming from
>>>> upstairs, mind you) being that the old values aren’t wrong per se, but
>>>> rather reflect the most up-to-date (but still preliminary) calibration
>>>> available at the time of publication.
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>>
>>>> On 4 Oct 2016, at 22:37, Andy Buckley
>>>> <andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>> wrote:
>>>>
>>>> Eek, ATLAS isn't propagating errata to HepData?!
>>>>
>>>> Hope I misunderstood...
>>>>
>>>> Andy
>>>>
>>>>
>>>> On 04/10/16 13:39, Chris G wrote:
>>>> Hi both,
>>>>
>>>> actually it’s the other way around — the HEPData tables weren't
>>>> changed and just a note has been added to the entry. The modification
>>>> would then have to be applied manually.
>>>>
>>>> Now, there were a couple of cases where I wrote a routine for an old
>>>> analysis where I noticed this and put that manual change in, such that
>>>> the Rivet reference data is fully up to date.
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> On 4 Oct 2016, at 13:31, Andy Buckley
>>>> <andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>> wrote:
>>>>
>>>> Absolutely. I'm not sure how good the groups are at propagating errata
>>>> to us. There's maybe been an assumption that we'll pick up the updates
>>>> from HepData -- which was our intention all along -- but the
>>>> divergence of HD from what we're given has broken that. It's a big
>>>> problem... and not one we can ignore. We need to involve the HepData
>>>> people.
>>>>
>>>> Andy
>>>>
>>>>
>>>>
>>>> On 04/10/16 13:07, David Grellscheid wrote:
>>>> Hi Chris,
>>>>
>>>> In that case, shouldn't the change propagate to Rivet, too?
>>>>
>>>> See you,
>>>>
>>>> David
>>>>
>>>>
>>>> On 04/10/16 12:46, Christian Gutschow wrote:
>>>> Hi Holger,
>>>>
>>>> there are cases where the Rivet reference data values and the HEPData
>>>> reference values will be different because e.g. there’s a note on
>>>> HEPData saying the central values ought to be shifted by some factor in
>>>> order to account for the the final luminosity calibration for the
>>>> respective year (which hadn’t been available when the HEPData entry was
>>>> made public).
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>>
>>>>
>>>> On 4 Oct 2016, at 11:39, Holger Schulz
>>>> <holger.schulz at durham.ac.uk<mailto:holger.schulz at durham.ac.uk>
>>>> <mailto:holger.schulz at durham.ac.uk>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I wrote a script that does some trivial checks for the reference
>>>> data files shipped with rivet and their counterpart obtainable from
>>>> HepData.
>>>>
>>>> It generates reports which can be found here:
>>>>
>>>>   https://users.hepforge.org/~holsch/HepDataRivetReports/
>>>>
>>>> There is quite a number of inconsistencies.
>>>>
>>>> Quite often in LHC analyses the dxy tags differ, a number of
>>>> ref data sets differ in the number of data points between rivet and
>>>> hepdata
>>>> and of course there are several analyses where the data is not in
>>>> hepdata
>>>> at all.
>>>>
>>>> And then there are some "goodness of fit" issues where I compare
>>>> two scatters as such (can clearly be improved):
>>>>
>>>>
>>>>   def gof(P1, P2):
>>>>       chi=0.0
>>>>       for num, p in enumerate(P1):
>>>>           chi += (p.y - P2[num].y)
>>>>           chi += (p.yErrAvg - P2[num].yErrAvg)
>>>>           chi += (p.x - P2[num].x)
>>>>           chi += (p.xErrAvg - P2[num].xErrAvg)
>>>>       return chi
>>>>
>>>>
>>>> So yeah I don't really know what course of action to take but it seem
>>>> quite clear
>>>> that HepData and rivet have diverged quite substantially.
>>>>
>>>> Holger
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Rivet mailing list
>>>> Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
>>>> <mailto:Rivet at projects.hepforge.org>
>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> —
>>>>
>>>> Dr. Christian Gütschow
>>>>
>>>> Department of Physics and Astronomy
>>>> University College London
>>>> Gower Street
>>>> London WC1E 6BT
>>>>
>>>> D10 Physics Building
>>>> +44 (0)20 7679 3775
>>>> chris.g at cern.ch<mailto:chris.g at cern.ch> <mailto:chris.g at cern.ch>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Rivet mailing list
>>>> Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>
>>>> _______________________________________________
>>>> Rivet mailing list
>>>> Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>
>>>>
>>>>
>>>> --
>>>> Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
>>>> Particle Physics Expt Group, University of Glasgow
>>>> _______________________________________________
>>>> Rivet mailing list
>>>> Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>
>>>>
>>>>
>>>> --
>>>> Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
>>>> Particle Physics Expt Group, University of Glasgow
>>>>
>>>>
>>>>
>>>>
>>>>  —
>>>>
>>>>  Dr. Christian Gütschow
>>>>
>>>>  Department of Physics and Astronomy
>>>>  University College London
>>>>  Gower Street
>>>>  London WC1E 6BT
>>>>
>>>>  > D10 Physics Building
>>>>  > +44 (0)20 7679 3775
>>>>  > chris.g at cern.ch<mailto:chris.g at cern.ch>
>>>>
>>>>
>>>>
>>>>
>>
>>


-- 
Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Expt Group, University of Glasgow