Page History

...

You will probably want to run jobs locally in a scratch directory, as they will tend to write out many files!

Input and Output Files

The job parameters may specify input files if the script uses them and optionally output file locations.

{                                                                          
    "input_files": {
        "events1.stdhep": "/path/to/events1.stdhep",
        "events2.stdhep": "/path/to/events2.stdhep",
    } 
    "output_files": {                      
        "events1.slcio": "my_events1.slcio",
        "events2.slcio": "my_events2.slcio"
    },
    "output_dir": "/path/to/outdir"                                                                                  
}     

In the above toy example, the files listed in input_files will be copied from the absolute path on the right to the file name on the left (format is "destination": "source").

The output files will be copied from the left hand path in the local scratch dir to the file name on the right (format is "source": "destination").

All output files will be copied to the directory listed under "output_dir", which can be an absolute or relative path.

To keep the names of the output files created by the job, simply list the same file name for the output file entries.

{                                                                            
    "output_files": {                      
        "events1.slcio": "events1.slcio",
        "events2.slcio": "events2.slcio"
    }
}     

This will make the job copy the file events1.slcio to a file with the same name in the output directory.

Creating Job Workflows

In order to run jobs on a batch system such as LSF or Auger, the job parameters need to be expanded into a workflow, which is a JSON file containing parameters for all the individual jobs.

...

Automatically submitting jobs to the batch system requires that you have created a workflow from your job parameters (covered in last section)last section).

The following commands use the script hps-mc-bsub to submit jobs to LSF (e.g. SLAC environment).

The command hps-mc-jsub should be used instead when submitting at JLab.

To submit all jobs in a workflow, execute a command similar to the following:

 hps-mc-batchbsub -l $PWD/logs ./tritrig.json

...

hps-mc-batchbsub -l $PWD/logs ./tritrig.json 1000 2000 [etc.]

...

hps-mc-batchbsub -l $PWD/logs -r 0:99 ./tritrig.json

...

# generate A-prime events using Madgraph 4
ap = MG4(name="ap",
        run_card="run_card_"+params.run_params+".dat",
        params={"APMASS": params.apmass},
        outputs=[filename],
        nevents=params.nevents)

Finally, the components should be added to the job and the job should be run.

job.components = [ap]
job.run()

The specific way that input and output files are used depends on the job script.

...