...
- Put all your input files into a text file, e.g.
ls /directory/containing/input/files/ > listOfInputFiles.txt
- Create a template jdl with a placeholder for the input file. This example is as simple as possible
No Format title 'template.jdl' Executable = lcsimReco_LOI_Higgs.sh # %s is going to be replaced with the input filename by the python script. When the script is submitted, the file name is then picked up as $1 by the shell script. Arguments = %s StdOutput = "std.out"; StdError = "std.err"; OutputSandbox = {"std.out", "std.err"}; InputSandbox = {"/home/hep/mbq94921/LcSimReconstruction/MainLoop.class", "/home/hep/mbq94921/LcSimReconstruction/lcsimReco_LOI_Higgs.sh"}; Requirements = RegExp(".gridpp.rl.ac.uk", other.GlueCEUniqueId);
- Use the following script to generate one jdl per line in
listOfInputFiles.txt
Code Block title createInputs.py #!/usr/bin/env python import os.path as path import os #read in the template template = open('template.jdl').read() #create a subdir for cleanliness os.mkdir('jobfiles') os.chdir('jobfiles') # read the input files for input in open('listOfInputFiles.txt'): filename = path.basename(input) name, extension = path.splitext(filename) output = open(name + '.jdl', 'w') output.write(template % input) output.close()
- Execute the script with
python createInputs.py
. It will create a directory jobfiles that contains one jdl for each input file. - download Download and compile the
MainLoop.java
- Then submit the jobs with
No Format voms-proxy-init -voms ilc glite-wms-job-submit -a -o collection.jodID --collection jobfiles