[Rivet] HepData consistency

[Andy Buckley]

On 05/10/16 08:02, David Grellscheid wrote:
> Hi all,
>
> What a lovely mess. I was starting to miss them!

:-D

> This needs to be fixed at so many levels. In Rivet, how about we do the
> following, if it's technically feasible:
>
> * YODA files must match Hepdata, to allow for replayability

Yes. By hook or by crook: this is the only way to sanity. How are we 
going to get there?

> * In the plot config file, apply the scaling through a factor, suitably
> commented

I don't think this is sufficient: we want to be able to uniformly use 
datasets for numerical processing (e.g. chi2 or likelihood computation) 
as well as plotting. Histograms that plot with good data agreement but 
mysteriously produce bad chi2s are a bit too confusing and magical for 
my liking...

> Independently, Hepdata needs to find a way to issue v2 datasets that are
> corrected. I see the point of keeping the original ones around, to match
> the publication, but having the newest correction available as full data
> would be useful.

My understanding is that the new HepData will have some sort of 
versioning. Maybe you could check with Graham Watt and make sure that 
there will be stable URLs for accessing the different versions?

Thanks,
Andy



> On 04/10/2016 22:56, Christian Gutschow wrote:
>> As far as I’m aware — if it’s an actual erratum, then the HEPData
>> tables will be fixed. If however — for example — a more precise
>> luminosity calibration becomes available subsequently (which would
>> only shifts the central values by the same amount), then usually a
>> note is added to the description of the HEPData entry, but this change
>> isn’t propagated through to the tables. The logic (coming from
>> upstairs, mind you) being that the old values aren’t wrong per se, but
>> rather reflect the most up-to-date (but still preliminary) calibration
>> available at the time of publication.
>>
>> Cheers,
>> Chris
>>
>>
>> On 4 Oct 2016, at 22:37, Andy Buckley
>> <andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>> wrote:
>>
>> Eek, ATLAS isn't propagating errata to HepData?!
>>
>> Hope I misunderstood...
>>
>> Andy
>>
>>
>> On 04/10/16 13:39, Chris G wrote:
>> Hi both,
>>
>> actually it’s the other way around — the HEPData tables weren't
>> changed and just a note has been added to the entry. The modification
>> would then have to be applied manually.
>>
>> Now, there were a couple of cases where I wrote a routine for an old
>> analysis where I noticed this and put that manual change in, such that
>> the Rivet reference data is fully up to date.
>>
>> Cheers,
>> Chris
>>
>> On 4 Oct 2016, at 13:31, Andy Buckley
>> <andy.buckley at cern.ch<mailto:andy.buckley at cern.ch>> wrote:
>>
>> Absolutely. I'm not sure how good the groups are at propagating errata
>> to us. There's maybe been an assumption that we'll pick up the updates
>> from HepData -- which was our intention all along -- but the
>> divergence of HD from what we're given has broken that. It's a big
>> problem... and not one we can ignore. We need to involve the HepData
>> people.
>>
>> Andy
>>
>>
>>
>> On 04/10/16 13:07, David Grellscheid wrote:
>> Hi Chris,
>>
>> In that case, shouldn't the change propagate to Rivet, too?
>>
>> See you,
>>
>> David
>>
>>
>> On 04/10/16 12:46, Christian Gutschow wrote:
>> Hi Holger,
>>
>> there are cases where the Rivet reference data values and the HEPData
>> reference values will be different because e.g. there’s a note on
>> HEPData saying the central values ought to be shifted by some factor in
>> order to account for the the final luminosity calibration for the
>> respective year (which hadn’t been available when the HEPData entry was
>> made public).
>>
>> Cheers,
>> Chris
>>
>>
>>
>> On 4 Oct 2016, at 11:39, Holger Schulz
>> <holger.schulz at durham.ac.uk<mailto:holger.schulz at durham.ac.uk>
>> <mailto:holger.schulz at durham.ac.uk>> wrote:
>>
>> Hi,
>>
>> I wrote a script that does some trivial checks for the reference
>> data files shipped with rivet and their counterpart obtainable from
>> HepData.
>>
>> It generates reports which can be found here:
>>
>>   https://users.hepforge.org/~holsch/HepDataRivetReports/
>>
>> There is quite a number of inconsistencies.
>>
>> Quite often in LHC analyses the dxy tags differ, a number of
>> ref data sets differ in the number of data points between rivet and
>> hepdata
>> and of course there are several analyses where the data is not in hepdata
>> at all.
>>
>> And then there are some "goodness of fit" issues where I compare
>> two scatters as such (can clearly be improved):
>>
>>
>>   def gof(P1, P2):
>>       chi=0.0
>>       for num, p in enumerate(P1):
>>           chi += (p.y - P2[num].y)
>>           chi += (p.yErrAvg - P2[num].yErrAvg)
>>           chi += (p.x - P2[num].x)
>>           chi += (p.xErrAvg - P2[num].xErrAvg)
>>       return chi
>>
>>
>> So yeah I don't really know what course of action to take but it seem
>> quite clear
>> that HepData and rivet have diverged quite substantially.
>>
>> Holger
>>
>>
>>
>> _______________________________________________
>> Rivet mailing list
>> Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
>> <mailto:Rivet at projects.hepforge.org>
>> https://www.hepforge.org/lists/listinfo/rivet
>>
>>
>>
>>
>>
>>
>> —
>>
>> Dr. Christian Gütschow
>>
>> Department of Physics and Astronomy
>> University College London
>> Gower Street
>> London WC1E 6BT
>>
>> D10 Physics Building
>> +44 (0)20 7679 3775
>> chris.g at cern.ch<mailto:chris.g at cern.ch> <mailto:chris.g at cern.ch>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Rivet mailing list
>> Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
>> https://www.hepforge.org/lists/listinfo/rivet
>>
>> _______________________________________________
>> Rivet mailing list
>> Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
>> https://www.hepforge.org/lists/listinfo/rivet
>>
>>
>>
>> --
>> Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
>> Particle Physics Expt Group, University of Glasgow
>> _______________________________________________
>> Rivet mailing list
>> Rivet at projects.hepforge.org<mailto:Rivet at projects.hepforge.org>
>> https://www.hepforge.org/lists/listinfo/rivet
>>
>>
>>
>> --
>> Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
>> Particle Physics Expt Group, University of Glasgow
>>
>>
>>
>>
>>  —
>>
>>  Dr. Christian Gütschow
>>
>>  Department of Physics and Astronomy
>>  University College London
>>  Gower Street
>>  London WC1E 6BT
>>
>>  > D10 Physics Building
>>  > +44 (0)20 7679 3775
>>  > chris.g at cern.ch<mailto:chris.g at cern.ch>
>>
>>
>>
>>


-- 
Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Expt Group, University of Glasgow