[Rivet] Rivet boost::BOOST_FOREACH

[Andrii Verbytskyi]

Dear Andy,
1)
> >>
> >> http://www.boost.org/doc/libs/1_58_0/doc/html/foreach.html#foreach.introduction.making__literal_boost_foreach__literal__prettier
> >
> > But that is not 1.55 you are using and might not (and it doesn't!) fit
> > Rivet >1.48 requirement. Quite confusing.
> 
> I just picked the first result that came up in Google. That instruction 
> has been in place since version 1.52 in 2012, because of problems 
> introduced by clashes with the internal use of a "foreach" identifier. 
> Sounds like we should update the Rivet Boost version requirement, if 
> 1.48 is causing trouble -- we're not testing on a platform with that 
> version so it's good to know there's a problem.

The >1.47 requirement is in the Rivet *now*. I don't know why it is
there and if it causes problems.
Maybe the workaround is present there since 1.52, but it clearly fails
with 1.53.
Also, it looks SL7 will stay with 1.53 and making 1.55 a minimal
requirement will make Rivet incompatible with the default and most used
HEP OS of the next 5 years.


2)
> >> I have a feeling that this bug relates to the version of Boost you're
> >> using, in conjunction with more recent compilers.
> >
> > Well, I try to stick to the default systems, I do not want to introduce
> > my custom versions for boost, gcc, glibc and so on. Now my system is
> > vanilla CentOS7.
> 
> Maybe someone else on this list can confirm, but it sounds like the 
> default GCC + Boost versions on CentOS 7 might be a problematic combination.

It also fails with clang.  I don't have other compilers and time to try.
I cannot try other boost as recompilation of boost requires too much
external libraries and there is no binaries(rpms).





3)
>
> >> I recall that an
> >> updated boost.m4 script that we introduced in the recent version 2.3.0
> >> addresses the problem...
> >
> > No, I've tried 2.2 and 2.1 as well.
> 
> I mean that versions before 2.3 might have the problem while 2.3 is ok. 
> Or does 2.3.0 also get through your configure step and then fail at 
> compile time?

Sure.


4)
> >> or rather it detects that there will be a
> >> problem and requires a consistent GCC+Boost installation.
> > It requires boost>1.48 or so and nothing more to solve r/lvalue problem.
> >
> >
> >> The *real* solution will be to require and use the C++11 range-for
> >> construct instead of Boost's macro. This will also work a lot better for
> >> e.g. iterating over std::maps, which don't work well with the macro
> >> approach. And of course we can replace a lot (but not all, I think)
> >> Boost functionality in Rivet with C++11 standard features
> > As I see, foreach appears in quite simple constructions for which even
> > C++0x will work fine. I mean "for (std::<blabla>::iterator it=...)"
> > one will need  one or zero more lines. In total, that would require
> > ~200-300 lines.
> 
> Yes, but Rivet's whole design philosophy is aligned with the "make 
> simple things simple quoted on the Boost foreach web page". We've been 
> using this functionality for more than 5 years! We're not going to take 
> a backward step to use iterators unnecessarily when a) the current use 
> of BOOST_FOREACH works on all the systems we've tested, including LCG 
> platforms, and b) the properly language-integrated version of that macro 
> is so close to being usable.

Sure, BOOST_FOREACH looks nice and in the Rivet it also works on some
"specially prepared" systems. But "so close" implies "is not", which
means it will fail on other, "ordinary" systems.



5)
> >
> >> , so it's very
> >> appealing to many of us. But we don't feel that non-experiment HEP
> >> computing resources are yet in a state where we can make that C++11
> >> requirement without causing pain to many theory users.
> >
> > Actually the experementalists are even less flexible with the software.
> > In many cases it is _very_ hard and inconvenient  to install somewhere a
> > custom version of compiler, boost and so on in the userspace. Some
> > programs will not work with custom versions at all. And custom versions
> > for every single library is a direct way to dependency hell. For  many
> > physics software packages I would prefer to have less features but
> > easier installation and compatibility.
> 
> What I meant is that the SL *system* versions of these packages are 
> often 5+ years out of date and not sufficient, but that experiment 
> software environments usually use much newer versions of all our 
> dependencies. We could move to requiring C++11 and the experiment/LCG 
> environments would have no problem, but the system compilers would not 
> be sufficiently capable.

I would not blame compilers or old software:


https://gcc.gnu.org/gcc-4.8/
GCC 4.8.3 May 22, 2014
http://www.boost.org/users/history/
Version 1.53.0 February 4th, 2013 18:29 GMT
http://llvm.org/
3.4.2: June 2014

2013-10-25  Andy Buckley  <andy.buckley at cern.ch>	
* 2.0.0 release!!!!!!
2014-02-06  Andy Buckley  <andy.buckley at cern.ch>
* 2.1.0 release!
2014-09-30  Andy Buckley  <andy.buckley at cern.ch>
* 2.2.0 release!
2015-06-30  Andy Buckley  <andy.buckley at cern.ch>
* Bump version to 2.3.0 and require YODA > 1.4.0 (current head at




6)
> >> For what it's worth, I'm currently using GCC 4.9.2 and Boost 1.55 and it
> >> works fine. In the pre-release testing that we do on CERN lxplus we use
> >> the system compiler but install our own 1.55 version of Boost.
> >
> > As far as I understand other versions don't  work.
> > GCC 4.9.2 and Boost 1.55 are not default on the lxplus.cern.ch,
> > bastion.desy.de, gate.rzg.mpg.de, (fnal?) and many others. If it doesn't
> > work there, one should use custom version.
> 
> As I said, the defaults on HEP systems are very often not sufficient. 
> That's why the experiments override them with full up-to-date 
> environments.

For obvious reasons that will not work for any of past (i.e. non-LHC)
experiments (significant fraction of Rivet analyses). The machines and
software are either "frozen" or don't exist anymore (implies using
public computing resources, mostly with default versions of compilers
and libraries). In some cases the old software will not work with
up-to-date software.

 

7)
> > Anyway,
> > it looks it works (compiles!) for me with a trivial patch (see
> > attached).
> 
> Which is exactly what we used to have, and reversed because of the 
> problems that it can have and the updated Boost solution. I'm glad it 
> works for your package/compiler combination, but we won't apply it upstream.

Sure. The proper solution would be to replace BOOST_FOREACH or to have
different versions of that part of code for different boost versions.
(Sorry, no time to prepare that patch.)



Best regards,
Andrii


> Andy
> 
> 
> >> On 04/08/15 13:05, Andrii Verbytskyi wrote:
> >>> Dear Andy,
> >>> I just tried to compile 2.* versions of Rivet on CentOS
> >>> 7/gcc4.8.3/boost1.53.
> >>> All those attempts failed, the compiler doesn't accept
> >>> the Rivet code:
> >>>
> >>> ----------------------------------------------------------------------
> >>>     CXX      libRivetCore_la-Run.lo
> >>> In file included from ../../include/Rivet/Run.hh:5:0,
> >>>                    from Run.cc:2:
> >>> ../../include/Rivet/Tools/RivetBoost.hh:16:36: error: declaration of
> >>> namespace 'boost::BOOST_FOREACH' conflicts with
> >>>      namespace BOOST_FOREACH = foreach;
> >>>                                       ^
> >>> In file included from /usr/include/boost/foreach.hpp:89:0,
> >>>                    from ../../include/Rivet/Tools/RivetBoost.hh:12,
> >>>                    from ../../include/Rivet/Run.hh:5,
> >>>                    from Run.cc:2:
> >>> /usr/include/boost/foreach_fwd.hpp:56:1: error: previous declaration of
> >>> namespace 'boost::BOOST_FOREACH' here
> >>>    {
> >>>    ^
> >>> In file included from ../../include/Rivet/Math/MathUtils.hh:5:0,
> >>>                    from ../../include/Rivet/Math/Math.hh:5,
> >>>                    from ../../include/Rivet/Tools/Utils.hh:5,
> >>>                    from ../../include/Rivet/Config/RivetCommon.hh:26,
> >>>                    from ../../include/Rivet/AnalysisHandler.hh:5,
> >>>                    from Run.cc:3:
> >>> ../../include/Rivet/Math/MathHeader.hh:45:16: warning: 'Rivet::MAXINT'
> >>> defined but not used [-Wunused-variable]
> >>>      const double MAXINT = std::numeric_limits<int>::max();
> >>> ----------------------------------------------------------------------
> >>>
> >>>
> >>> It looks the reason is a well known (since 4 years!) BOOST bug.
> >>> 1) Which compilers do you use for tests, i.e. with which compiler Rivet
> >>> will work?
> >>> 2) Has anyone tried to compile Rivet with (new)gcc?
> >>> 3) Is it possible to drop that construction from Rivet? It looks it
> >>> doesn't appear too often in the code and when it does the replacement is
> >>> obvious.
> >>>
> >>>
> >>> Best regards,
> >>> Andrii
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Fri, 2014-12-05 at 14:10 +0100, Andrii Verbytskyi wrote:
> >>>> Hi,
> >>>> yes, environment variables are working, however passing arguments via
> >>>> environment variables is implicit way and implicit way is worse that
> >>>> explicit. Also, in many cases environment variables will make things
> >>>> less convenient or even defunct. A good example is using rivet
> >>>> from a Makefile. One can say the same about using rivet in system() and
> >>>> so on. Also, the way of setting environment variables depends on shell
> >>>> --> less convenient usage of rivet even from shell.
> >>>>
> >>>>
> >>>> Best regards,
> >>>> Andrii
> >>>>
> >>>>
> >>>>
> >>>> On Fri, 2014-12-05 at 13:00 +0000, Andy Buckley wrote:
> >>>>> Hi Andrii,
> >>>>>
> >>>>> Don't the environment variables work for these? --analysis-path is just
> >>>>> syntactic sugar for those, and to be honest I was considering removing
> >>>>> it to clean up the command-line interface!
> >>>>>
> >>>>> Andy
> >>>>>
> >>>>>
> >>>>> On 05/12/14 12:44, Andrii Verbytskyi wrote:
> >>>>>> Dear Andy,
> >>>>>> here are some (hopefully useful) improvements for Rivet.
> >>>>>>
> >>>>>> In addition to --analysis-path I think it would be useful to have
> >>>>>>
> >>>>>> --ref-path, --info-path, --plot-path.
> >>>>>>
> >>>>>>
> >>>>>> So here are the files with the changes. Please note that rivet should be
> >>>>>> generated from rivet.in with a substitution of install paths.
> >>>>>>
> >>>>>> Best regards,
> >>>>>> Andrii
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >
> 
>