[Rivet] Running Rivet on the grid

[Ben Waugh]

Hi All,

Maybe it's time we had another look at using the CEDAR VO to make better 
use of the LCG middleware. The data management side of things seems 
rather better developed now than when we first created the VO, and there 
may be tools we can use to distribute the required software to any sites 
that agree to support CEDAR (which will be at least UCL and Durham I 
hope, and probably the other LondonGrid sites) rather than each job 
having to download its own set of tarballs.

I have not looked at this for some time because there didn't really seem 
to be the need for it, but if the stage has been reached where we want 
to use the Grid seriously, perhaps James, Gianfranco and I can get 
together some time and see what we can work out.

Cheers,
Ben

On 15/10/08 11:43, Frank Siegert wrote:
> I have a few suggestions for the grid run strategy, which mainly appeared 
> when I started building the validator:
> 
> We should adapt the downloaded grid-tarballs more to which generator is 
> being run. Is Genser available on the grid nodes? I assume no, because 
> you are including the "fake" Genser tree in your package, which then I 
> think should be split up.
> Right now it would download ~80 MB and that even only includes charybdis, 
> herwig, jimmy and pythia. For some generators we don't even need 
> AGILe/Genser, but use them through a socket.
> 
> Do you think it is feasible to link AGILe at runtime (if found in 
> LD_LIBRARY_PATH) instead of compile time? Since we have identified all 
> relevant places with an "#ifdef HAVE_AGILE" already, this might not be so 
> hard? Of course, during compilation one would need the AGILe headers 
> anyways. If nobody has a better suggestion, I could look into this. 
> Another option would be providing two Rivet tarballs, one compiled with 
> AGILe, and one without.
> 
> And as far as I know there is already a mechanism in AGILe which checks 
> and enables only the generator interfaces it can find in LD_LIBRARY_PATH, 
> right?
> 
> Then we can split up the downloaded packages into
>  - HepMC*
>  - LHAPDF*
>  - FastJet*
>  - (boost is not needed, because we only use header files AFAIK)
>  - Rivet
>    + libgsl.so link (from what i have noticed, this is missing on SL4,
>      but might only be needed during compilation?)
>  - AGILe core
>  - Each generator*, including its AGILe library for the ones where we
>    don't want to use the standalone generator through HepMC fifo.
> 
> * Everything marked with an asterisk is probably available on GENSER. So 
> if we have Genser on grid nodes, it wouldn't be necessary to bundle them 
> into our "grid tarball".
> 
> All of these should untar into the same temporary bin/ lib/ share/ 
> directories on the grid node, so with a properly set PATH and 
> LD_LIBRARY_PATH all programs can figure out what is available.
> 
> Does that sound reasonable?
> Frank
> _______________________________________________
> Rivet mailing list
> Rivet at projects.hepforge.org
> http://www.hepforge.org/lists/listinfo/rivet

-- 
Dr Ben Waugh                                   Tel. +44 (0)20 7679 7223
Dept of Physics and Astronomy                  Internal: 37223
University College London
London WC1E 6BT