[Rivet] Running Rivet on the grid

[Andy Buckley]

Frank Siegert wrote:
> Hi James, Andy,
> 
> thanks for the script.
> 
> Andy Buckley, Friday 10 October 2008:
>> James Monk wrote:
>>>     ./RivetGrid -vo <MY_VO> -g Herwig:6510 -n 1000 - p IPROC=1500
>>> etc...
>> Thanks James, this is excellent. I can't get it to run at the moment,
>> but I think that has more to do with the IPPP Grid setup than your
>> script. At least, I don't think it's getting far enough to test the
>> script properly!
> 
> The script still uses the edg-* commands to submit jobs to the GRID. As 
> far as I know, EDG resource brokers are slowly being phased out, in 
> favour of gLite workload management servers. This means the edg-* 
> commands have to be replaced by glite-wms-* in the near future. Depending 
> on which resource broker you used, Andy, you might have had one that has 
> already decommissioned its EDG resource brokers? RAL Tier1 for example 
> will drain them on 28 Nov. (see email attached at the bottom)
> 
> We should probably switch the script to using glite-wms already, I am 
> testing it now and will let you know whether it works and which changes 
> are needed. I am also still pondering whether I am going to use such a 
> script in the Rivet validation tool, or whether the grid-submission 
> capabilities should be part of the tool itself. I am in favour of the 
> latter, as the tool will have to do some simple monitoring and output 
> retrieval as well, which is maybe beyond the scope of a simple RivetGrid 
> script.

I don't know if the API is public, but the glite commands certainly are
Python scripts making use of some API:

buckley at s2:~$ less `which glite-wms-job-status`

They are using a non-standard (what a surprise!) way to access the gLite
Python modules, based on appending $GLITE_WMS_LOCATION/lib and
$GLITE_WMS_LOCATION/lib/python to the Python sys.path. (The code also
makes it clear that whoever wrote it has little experience with the
Python standard library...)

Then there are modules called UIutils, UIchecks, Job, ... and the catchy
glite_wmsui_*Wrapper which do the Griddy stuff:

buckley at s2:~$ ls $GLITE_WMS_LOCATION/lib/python/ | grep "\.py$"
glite_wmsui_AdWrapper.py
glite_wmsui_LbWrapper.py
glite_wmsui_LogWrapper.py
glite_wmsui_SdWrapper.py
glite_wmsui_UcWrapper.py
Job.py
Listener.py
rgma.py
UIchecks.py
UIutils.py
WMPClient.py
WMPConnection.py
wmproxymethods.py


I hope that helps!

Andy

-- 
Dr Andy Buckley
Institute for Particle Physics Phenomenology
Durham University
0191 3343798 | 0191 3732613 | www.insectnation.org