Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Put all your input files into a text file, e.g. ls /directory/containing/input/files/ > listOfInputFiles.txt
  2. Create a template jdl with a placeholder for the input file. This example is as simple as possible
    No Format
    title'template.jdl'
    Executable = lcsimReco_LOI_Higgs.sh
    # %s is going to be replaced with the input filename by the python script. When the script is submitted, the file name is then picked up as $1 by the shell script.
    Arguments = %s
    StdOutput = "std.out";
    StdError = "std.err";
    OutputSandbox = {"std.out", "std.err"};
    InputSandbox = {"/home/hep/mbq94921/LcSimReconstruction/MainLoop.class", "/home/hep/mbq94921/LcSimReconstruction/lcsimReco_LOI_Higgs.sh"};
    Requirements = RegExp(".gridpp.rl.ac.uk", other.GlueCEUniqueId);
    
    
  3. Use the following script to generate one jdl per line in listOfInputFiles.txt
    Code Block
    titlecreateInputs.py
    #!/usr/bin/env python
    import os.path as path
    import os
    #read in the template
    template = open('template.jdl').read()
    #create a subdir for cleanliness
    os.mkdir('jobfiles')
    os.chdir('jobfiles')
    # read the input files
    for input in open('listOfInputFiles.txt'):
        filename = path.basename(input)
        name, extension = path.splitext(filename)
        output = open(name + '.jdl', 'w')
        output.write(template % input)
        output.close()
    
  4. Execute the script with python createInputs.py. It will create a directory jobfiles that contains one jdl for each input file.
  5. download Download and compile the MainLoop.java
  6. Then submit the jobs with
    No Format
    voms-proxy-init -voms ilc
    glite-wms-job-submit -a -o collection.jodID --collection jobfiles