[Rivet] possibility to add (hadd) yoda files, and running rivet on lxplus batch jobs

[Andy Buckley]

On 09/06/16 13:07, Raghav Kunnawalkam Elayavalli wrote:
> Hi Rivet Experts,
>
> My current analysis has exploded into quite the complicated one
> leading to a large run time ~25 seconds per 100 events. It would be
> very useful if i can submit jobs to run this in parallel say 100 job
> running over 1000 events each. In leu of this requirement, i would
> need to add the output yoda files together similar to the hadd script
> that does root files.

This is no problem: that's what the yodamerge script does.

> Im also having several difficulties with the proper python
> environment at the node where my batch jobs end up (for rivet -a
> ANALYSIS input.hepmc). this is what i used in my rivet installation
> on lxplus in the rivet-bootstrap-lcg SETUPCMD_PYTHON="source
> $LCGDIR/Python/2.7.10/x86_64-slc6-gcc48-opt/Python-env.sh" and this
> is what i set in my job source
> /cvmfs/sft.cern.ch/lcg/releases/LCG_84/Python/2.7.10/x86_64-slc6-gcc48-opt/Python-env.sh
>
>when i submit a job with this python and rivet setup, i get a
 > following error
>
> @(#)CERN job starter $Date: 2010/06/23 14:22:16 $ Working directory
> is </pool/lsf/rkunnawa/809932193> on <b64f2daace.cern.ch> Started
> running the job
>
> this error is for sourcing that python env.
>
> 'import site' failed; use -v for traceback Traceback (most recent
> call last): File
> "/cvmfs/sft.cern.ch/lcg/releases/LCG_84/lcgenv/1.3.2.1/x86_64-centos7-gcc49-dbg/lcgenv",
> line 11, in <module> import os ImportError: No module named os
>
> # output of which python: /usr/bin/python
>
> these are for the following error messages when it comes to the rivet
> -a 'import site' failed; use -v for traceback Traceback (most recent
> call last): File
> "/afs/cern.ch/work/r/rkunnawa/JEWEL/local/bin/rivet", line 20, in
> <module> import os, sys ImportError: No module named os 'import site'
> failed; use -v for traceback Traceback (most recent call last): File
> "/afs/cern.ch/work/r/rkunnawa/JEWEL/local/bin/rivet-mkhtml", line 17,
> in <module> import rivet, sys, os ImportError: No module named rivet

Hmm, I don't really know what's going on. So you set up the same Python 
environment for your job as used in the build? If you just set up that 
Python without setting up the Rivet environment, can you open a Python 
interpreter, import os, etc. without getting these errors?

This isn't really a Rivet problem, though -- the environment is always a 
pain on lxplus and other SLC6 systems, since the system compilers and 
Python are not the ones that are actually supported by LCG.

Andy

-- 
Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Expt Group, University of Glasgow