[Rivet] Compatibility of YODA reference data from Rivet with HEPData

[Peter Skands]

Dear Graeme,

Thanks for this! Not a Rivet developer myself of course, but this seems to
be a good first step towards getting back to the original vision of having
HepData records and Rivet ones evolving in lock step with each other. Keen
to hear the feedback from the Rivet authors.

Keith mentioned that there is in principle another person in Durham
associated with HepData, who has been typing in data records etc. Do you
think it would be realistic for that person to have a look at the
consistency of the older records between Rivet and HepData and perhaps
address some of them?

All the best,
Peter

—

*PETER SKANDS*Associate Professor


*School of Physics and Astronomy*Monash University
10 College Walk, Clayton Campus
Melbourne, VIC 3800
Australia

T: +61 3 990 53692 <//+61%203%20990%2053692>
E: peter.skands at monash.edu
W: skands.physics.monash.edu

On 22 May 2018 at 5:16:56 am, Graeme Watt (graeme.watt at durham.ac.uk) wrote:

Dear Rivet developers,

I wrote a Python script to compare a YODA reference data file, intended for
inclusion in Rivet, with the corresponding YODA file downloaded from
HEPData:


https://github.com/HEPData/miscellaneous/blob/master/scripts/yoda_compare_hepdata.py
  Example usage:  ./yoda_compare_hepdata.py ATLAS_2017_I1614149.yoda -i
1614149 -a

This means: compare a local YODA file "ATLAS_2017_I1614149.yoda" with a
YODA file downloaded from the HEPData record with INSPIRE ID "1614149" and
also compare YODA annotations "-a".  Since the HEPData YODA file might
contain additional analysis objects compared to the Rivet YODA file, and
since there might be inconsequential rounding errors or differences in
number formats, comparison using a simple "diff" of .yoda files is not
always adequate.

I had a few problems with the YODA 1.7.0 software when writing the Python
script, which could perhaps be improved in future:

* Calling dump() on Scatter objects does not output the same format as in
the input .yoda files, e.g. dumping a Scatter2D gives "HISTO1D" in the
output and the central value of the x bin is not output.  This might be due
to deficiencies in https://yoda.hepforge.org/trac/browser/src/WriterFLAT.cc
.
* I expected to be able to check (fuzzy) equality of two Scatter objects
using "s == s1", which seems to be implemented in C++ but not in Python.
* Checking (fuzzy) equality of two Point2D objects using "p == p1" is
implemented in Python, but it only compares the x axes and not the y axes.
Similarly, for Point3D objects, the (fuzzy) equality operator "==" only
compares the x and y axes, but not the z axes.
* In the end, I just copied the definition of "fuzzyEquals" from the C++
code into my script and did my own comparisons, without relying on the "=="
operator for Point or Scatter objects.

Recall that Holger Schulz made some similar comparisons in 2016 between
YODA reference data files from Rivet and from the old HepData, where he
found significant inconsistencies:

  https://www.hepforge.org/lists-archive/rivet/2016-October/007318.html

https://rivet.hepforge.org/trac/browser/contrib/devscripts/HepDataConsistency

While fixing all Rivet/HEPData inconsistencies is probably unrealistic for
now, we could at least start by ensuring that analyses added to new Rivet
releases include a YODA file that's compatible with the YODA download given
by HEPData.  My new script should be useful for these purposes.  (This work
was prompted by a conversation with Peter Skands and Jon Butterworth last
month in Durham.)  You're welcome to modify my script as you need and
include it in a future Rivet release.  The script could be run by the
experiment contact persons before they upload new analyses to the Rivet
"contrib" area.  It could also be run (perhaps in a more automated way) by
the Rivet developers when moving analyses from the "contrib" area to a new
Rivet release.  I just ran the script for all the new analyses in the
current Rivet "contrib" area (
https://www.hepforge.org/archive/rivet/contrib/ ) and it already turned up
some useful information:

* ATLAS_2014_I1310835, ATLAS_2017_I1614149, and ATLAS_2017_I1624693, are
all compatible with HEPData.
* ATLAS_2016_I1502620 has multiple .yoda files which can't be handled by my
script.
* ATLAS_2017_I1625109 showed some apparent inconsistencies but the problem
looks to be on the HEPData side.  The dataset index starts at 0 (instead of
1) due to a bug in the hepdata-converter package (which I'll fix) and the
year in the analysis name is 2018 (instead of 2017) taken from the journal
publication.
* ALICE_2017_I1620477 is incompatible with HEPData.
* CMS_2016_I1487277 is compatible with HEPData.
* CMS_2012_I1111014 and CMS_2016_I1491950 are incompatible with HEPData.
* CMS_2017_I1499471, CMS_2017_I1635889, CMS_2018_I1662081 and
CMS_2018_I1663958 are all missing from HEPData.
* CMS_2017_I1605749 has a HEPData record, but the YODA conversion fails due
to a problem with the original HEPData submission.
* LHCF_2015_I1351909 is compatible with HEPData, but the annotations differ.
* LHCF_2016_I1385877 is incompatible with HEPData.

I hope this gives you something to discuss at this week's Rivet workshop!
:-)

Best regards,
Graeme Watt (HEPData)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.hepforge.org/lists-archive/rivet/attachments/20180521/d827d035/attachment.html>