Page History

The Automatic Run Processor (or ARP, for short, because I want that to catch on) is a web service that allows for automatic workflows and for the easier submission of batch jobs via a web interface. A script that submits the batch job (to allow for more customization in this command) is all that is needed for this system to work.

...

The absolute path to the batch script. An example can be seen here. This  This script must contain the batch job submission command (bsub/sbatch) since it . It gives the user the ability to customize the the batch submission. Overall, it can act as a wrapper for the code that will do the analysis on the data along with submitting the job.

...

This defines where the analysis is done. While many experiments prefer to use the SLAC psana cluster (SLAC) to perform their analysis, some experiments others prefer to use HPC facilities like NERSC to perform their analysis. 

...

This script will submit /reg/d/psdm/dia/diadaq13/scratch/analyze on psdebugq and store the log files in /reg/d/psdm/dia/diadaq13/scratch/logs/<lsf_id>. Since the job definition parameters and JID parameters are defined as environment variables, /reg/d/psdm/dia/diadaq13/scratch/analyze will inherit the EXPERIMENT, RUN_NUM and JID_UPDATE_COUNTERS environment variables.

2.2)

...

analyze

The This Python script is the code that will do analysis and whatever is necessary on the run data. Since this is just an example, the Python script, submit.pyanalyze, doesn't get that involved. It is shown below.

...