[Rivet] C++11 plan for Rivet

[Frank Siegert]

Hi Andy,

On 14 August 2016 at 00:24, Andy Buckley <andy.buckley at cern.ch> wrote:
> On 11/08/16 10:43, Frank Siegert wrote:
>>
>> On the same note, I realised that we should document on the website
>> which gcc versions we support (so far this is neither in the 2.5.0
>> release announcement, nor on the Getting Started page, or did I miss
>> it?). For example: are we happy with >=4.7 as still used in
>> experiments a lot, or do we need the features introduced in 4.8
>> (https://gcc.gnu.org/projects/cxx-status.html#cxx11). Since we are one
>> of the first projects in the MC landscape to switch to a requirement
>> of C++11, we should provide the users with some guidance, and also
>> mention that they have to export CXXFLAGS=-std=c++11 when compiling
>> with an older compiler.
>
> Yes, we should explicitly mention this in the docs. I think we need 4.8 --
> isn't this the version used by the experiments?

ATLAS evgen happens with gcc 4.7 (Athena release 19.2.5.x) at the moment.

> I've not tested with
> anything earlier. The compiler flags are automatically passed to analysis
> plugin building and the output of rivet-config: is it sufficient to advise
> using the latter to get the appropriate extra flags for building code units
> that use the Rivet API?

I have tested building with gcc 4.7.1, but this already fails during
configure due the "test_variadic_templates" in
m4/ax_cxx_compile_stdcxx_11.m4

To see whether we really use gcc 4.8 functions, I disabled the
configure checks, and the only place where compilation fails with 4.7
is the map::emplace in Logging.cc and the special normalize functions
in MC_MET.cc. Don't know whether it's possible/worth replacing these
two and run it through the test suite. Thus we would have a Rivet
version which still builds with 4.7 in the short term while that's
still used in the community?
This is of course "only" a social argument, which runs contrary to
David's strong opinion about not caring about backwards compatibility,
which I can understand from a programmer's point of view as well.

Cheers,
Frank



>> On 24 March 2016 at 11:55, Andy Buckley <andy.buckley at cern.ch> wrote:
>>>
>>> Yes, I agree. This should be an opportunity to clean up and make our API
>>> more expressive, not to introduce more fallbacks and complexity.
>>>
>>> Andy
>>>
>>>
>>>
>>> On 24/03/16 10:27, David Grellscheid wrote:
>>>>
>>>>
>>>> Hi Andy,
>>>>
>>>> I'm all for the transition to C++11. But if we do it, I vote for a hard
>>>> switchover. No backwards compat flags, #ifdefs falling back to Boost,
>>>> etc. Once a function is converted, it stays that way. We should assume a
>>>> fully-compliant C++11 compiler and not have to worry about use of
>>>> individual features, and when they started being supported by which
>>>> version of gcc.
>>>>
>>>> See you,
>>>>
>>>>    David
>>>>
>>>>
>>>>
>>>> On 23/03/2016 22:19, Andy Buckley wrote:
>>>>>
>>>>>
>>>>> On 17/03/16 10:27, Holger Schulz wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 17/03/16 09:19, Andy Buckley wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I'm currently hacking together a prototype physics object smearing
>>>>>>> system for Rivet 2.5 or 2.6, to be beta-tested with the BSM
>>>>>>> experimental groups and re-casters. No capitulation: we will continue
>>>>>>> to require unfolded "proper" measurements, but there is BSM demand
>>>>>>> for
>>>>>>> Rivet and they will not bite unless we can provide at least some
>>>>>>> simple machinery for efficiency curves.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I completely agree now that the ATOM guys have essentially decoupled
>>>>>> their efforts from Rivet.
>>>>>> I did something like that in my ATLAS analysis for random removal of
>>>>>> tracks based on track
>>>>>> reco efficiency and I know that there is demand to have that in rivet,
>>>>>> too.
>>>>>
>>>>>
>>>>>
>>>>> Somehow I was interested enough in this to spend a bit of time hacking
>>>>> a
>>>>> start together. Here it is:
>>>>>
>>>>>
>>>>>
>>>>> https://rivet.hepforge.org/hg/rivet/file/tip/include/Rivet/Projections/SmearedParticles.hh
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> https://rivet.hepforge.org/hg/rivet/file/tip/include/Rivet/Projections/SmearedJets.hh
>>>>>
>>>>>
>>>>>
>>>>> (Connoisseurs of borderline-evil code may like to note how the function
>>>>> hash for projection comparison is being done ;-) But no worse than what
>>>>> you need to do to use POSIX dlsym... aka the evil_cast in AGILe's
>>>>> module
>>>>> loader.)
>>>>>
>>>>> And here's how the SmearedJets looks in action in an analysis:
>>>>>
>>>>> FastJets fj(FinalState(Cuts::abseta < 5), FastJets::ANTIKT, 0.4);
>>>>> SmearedJets sj(fj, JET_EFF_ONE, JET_SMEAR_IDENTITY);
>>>>>
>>>>> Ok, those are do-nothing standard perfect efficiency and smearing
>>>>> functions, but the principle works. I'm populating a file with a few
>>>>> real efficiency and smearing functions... but they will strongly
>>>>> benefit
>>>>> from going to mandatory C++11 so we can use the new random generator
>>>>> machinery. And here's another C++11 killer feature for this use-case --
>>>>> inline lambda functions:
>>>>>
>>>>> SmearedJets sj(fj,
>>>>>                 [](const Jet& j) { return 1 - exp(-j.pT()/(10*GeV)); },
>>>>>                 [](const Jet& j) { return j; });
>>>>>
>>>>> I think this stuff makes a pretty good case for going to C++11 with the
>>>>> addition of this feature... and then gradually converting lots of our
>>>>> internal Boost and hackery to use the much nicer new language features
>>>>> instead.
>>>>>
>>>>>>> The way I'm designing it can all be done with Boost features, but
>>>>>>> once
>>>>>>> again it's stuff that is part of the core language in C++11 and using
>>>>>>> the less-standard Boost implementations feels like going in the wrong
>>>>>>> direction. Also, we are starting to see submitted analyses which use
>>>>>>> C++11 features, and it both feels unnecessarily restrictive on our
>>>>>>> "clients" and is extra work for us to have to revert that code to
>>>>>>> C++98.
>>>>>>>
>>>>>>> So I would like us to make the switch to mandatory C++11 building of
>>>>>>> Rivet in the next couple of 2.x releases. This would also help us to
>>>>>>> reduce the currently huge number of "paradigm shifts" scheduled (for
>>>>>>> lack of imagination) on v3.0.
>>>>>>>
>>>>>>> There is one major sticking point, in the form of FastJet. While
>>>>>>> C++11
>>>>>>> compatible, it makes use of auto_ptr and exposes that in its public
>>>>>>> headers, meaning that anyone compiling a Rivet analysis in C++11 mode
>>>>>>> gets a terminal output dominated by FJ auto_ptr deprecation warnings.
>>>>>>> There seems to be no way in GCC to disable these warnings -- or does
>>>>>>> someone know of one? So I think we need to put a bit of pressure on
>>>>>>> FastJet to make a non-complaining release; I already did this ~6
>>>>>>> months ago and was told that they are working on a major new
>>>>>>> development, but it has not appeared and the issue is more urgent now
>>>>>>> (I don't know what the experiments are doing re. this). So I'll prod
>>>>>>> them again and hopefully we'll be able to make this switch in Rivet
>>>>>>> 2.5 or 2.6.
>>>>>>
>>>>>>
>>>>>>
>>>>>> We can just ask again, maybe point them to this forum:
>>>>>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59325
>>>>>
>>>>>
>>>>>
>>>>> Did that: thanks for the link :-) And the timing was good: FastJet
>>>>> 3.2.0
>>>>> is out now and at the last minute they removed auto_ptr from fjcore and
>>>>> added a configure-time option to disable the auto_ptrs in the main FJ
>>>>> interface.
>>>>>
>>>>> And it looks like LCG can be persuaded to build their FJ 3.2.0
>>>>> installations using that flag, so once there is such a bundle to play
>>>>> with, we can make a mandatory C++11 release. Which I would like to use
>>>>> for beta-testing the new smearing features with interested parties.
>>>>>
>>>>> Cheers,
>>>>> Andy
>>>>>