[Rivet] compatibility issues between 2.2.0 and 2.X.X

[Andy Buckley]

On 29/10/15 14:37, Niccolo' Moretti wrote:
> Hi,
>
> @ Hannes
> exactly what happened to me

Our apologies for that... but I'm surprised that there has been such an 
apparent rash of problems (though this is the first that we heard of 
it). Few functions with the purely old signatures should have actually 
been removed, but most should now be deprecated. We will most likely 
remove them when version 3.0 comes out.

> @ frank
> yes, a longer period of time with a warning would have been already
> better. And yes, I'm working with raw yodas as well, so scripts which
> involve yodas must be changed as well.

There are still developments to come in the YODA format, to handle some 
new data types and additional functionality like multiple errors on 
Scatter points. Reading the files via the YODA C++ or Python interface 
(or the format converter scripts) is the best way to avoid breakage. 
Please let us know if there is functionality missing, or any suggestions 
for enhancements.

> Of course I have a 1 line script to convert yodas between the versions,
> but as I said, one would like to have the simplest and most "universal"
> environment as he can, so this modifications are a little annoying (and,
> in the case of the '#' removal, I don't see any performance/size
> advantage...)

As mentioned, this is fixed in YODA and if you use the YODA reader 
functions then it will continue to work fine. Not many more structural 
changes are expected, and we do want to get the format itself fully 
stable (in addition to the parser backward compatibility). Note that 
there are some features of the current format which are supported by the 
parser but not used in the current writers, so when we turn them on in 
the writer your raw parsing scripts may again have trouble even though 
the format has not strictly "changed".

Andy


> On 29/10/15 15:30, Hannes Jung wrote:
>> Hi Andy and Niccolo
>>
>> I observed the same problem between 2.2.0 and 2.2.1 with the Cuts system.
>> In quite a few analysis we had in CMS we were using 2.2.0 and had to
>> change
>> for our internal codes to the new definitions.... which is tricky, if
>> it is not clear, what has changed....
>> and one is not expecting such a change when you only change subversion
>> numbers...
>> Perhaps it would be good to have clearly defined, what can change
>> between the different version numbers
>>
>> Cheers
>> Hannes
>>
>>> On 29.10.2015, at 15:19, Andy Buckley
>>> <<mailto:andy.buckley at cern.ch>andy.buckley at cern.ch> wrote:
>>>
>>> On 29/10/15 14:05, Niccolo' Moretti wrote:
>>>> Hi Andy,
>>>>
>>>> Thank you for your reply.
>>>>
>>>> The first 2 things that come into my mind are the deprecation of
>>>>
>>>> jetproj.jetsByPt(pt,MAXDOUBLE,etmin,etmax,PSEUDORAPIDITY);
>>>
>>> I'm not quite sure when we removed that -- apologies if it wasn't
>>> clearly deprecated before. We brought the Cuts system in in 2.2.0 and
>>> it was intended to be a complete replacement for all these confusing
>>> numeric signatures.
>>>
>>> I'm pretty sure that none of the hundreds of standard analyses were
>>> using this full 5-double signature, hence we didn't maintain the
>>> temporary backward compatibility for as long as we have done with
>>> some other e.g. projection constructor signatures.
>>>
>>>> and the removal of the '#' from yodas (ie, from '# BEGIN HISTO' to
>>>> 'BEGIN HISTO').
>>>
>>> This was not intended to be a "visible" change -- the last several
>>> versions of YODA are able to read the format with or without leading
>>> # signs. So an upgrade to the latest YODA should fix that issue for
>>> you, and for further iterations of the YODA format we now have a way
>>> to maintain backward compatibility with old files.
>>>
>>> Andy
>>>
>>>
>>>> On 29/10/15 14:44, Andy Buckley wrote:
>>>>> Hi Niccolo,
>>>>>
>>>>> Can you be specific about the deprecations that caused problems? We
>>>>> typically try to only add functionality and explicitly mark features
>>>>> as deprecated for some time before removing them in major revisions.
>>>>> So it sounds like we have carelessly broken something between those
>>>>> minor versions, probably because the bundled set of 300+ analyses kept
>>>>> on working.
>>>>>
>>>>> As of 2.3 and 2.4 we are trying to reflect more accurately in the
>>>>> version numbering whether there have been any significant additions to
>>>>> the interface, and to flag and deprecate very explicitly when the
>>>>> preferred ways of doing something change.
>>>>>
>>>>> Apologies for the inconvenience,
>>>>> Andy
>>>>>
>>>>>
>>>>> On 29/10/15 09:51, Niccolo' Moretti wrote:
>>>>>> To rivet authors,
>>>>>>
>>>>>> I would like to complain about all the recent
>>>>>> modifications/deprecations
>>>>>> occurred in Rivet in the last few updates.
>>>>>>
>>>>>> Together with other physicists in other universities and institutions,
>>>>>> I'm in charge of a study on the effects of different MC generators on
>>>>>> particular processes.
>>>>>> One of the most important thing is therefore to set up an
>>>>>> universal,common environment to compare the results. Some months
>>>>>> ago we
>>>>>> decided to use Rivet as a validation framework because of its
>>>>>> simplicity
>>>>>> and flexibility.
>>>>>>
>>>>>> Now all of us are in a situation where the analysis source code
>>>>>> and the
>>>>>> output result depend on the used rivet version(and plugins
>>>>>> therein). All
>>>>>> the modifications and deprecations have been done without any
>>>>>> retro-compatibility function, forcing us to do different outputs and
>>>>>> codes according to the version, losing in this way the
>>>>>> universality that
>>>>>> we have been looked for.
>>>>>>
>>>>>> Moreover, it's not  possible that such modifications, in some cases,
>>>>>> have occurred between to contiguous versions, say 2.2.0 and 2.2.1.
>>>>>>
>>>>>> You should know how it's difficult to set up a common environment
>>>>>> among
>>>>>> different universities, institutions, fields (theory and experiments),
>>>>>> computer architectures and people, and of course, how it's
>>>>>> difficult to
>>>>>> reinstall or updates software in large clusters.
>>>>>>
>>>>>> I hope that in the future all the deprecations will not be done
>>>>>> instantaneously.
>>>>>>
>>>>>> Regards,
>>>>>> Niccolo' Moretti
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Rivet mailing list
>>>>>> Rivet at projects.hepforge.org <mailto:Rivet at projects.hepforge.org>
>>>>>> https://www.hepforge.org/lists/listinfo/rivet
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
>>> Particle Physics Expt Group, University of Glasgow
>>> _______________________________________________
>>> Rivet mailing list
>>> Rivet at projects.hepforge.org <mailto:Rivet at projects.hepforge.org>
>>> https://www.hepforge.org/lists/listinfo/rivet
>>
>


-- 
Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Expt Group, University of Glasgow