The batch processor allows for the easier submission of batch jobs via a web interface. A script that submits the batch job (to allow for more customization in this command) is all that is needed for this system to work.

1) Webpage

To use this system, choose an experiment from https://pswww-dev.slac.stanford.edu/apps-dev/portal/select_experiment.php.

1.1) Batch Job Definitions

Under the Experiment tab, the Batch defs tab is the location where scripts are stored. Below is an example where the example script is used which is described in detail below. A hash name is given to the absolute path of the script along with an arbitrary number of parameters which are separated by a space. An number of hashes can be created; as a hash is entered, a new row appears.

1.2) Batch Control

Over on Run Tables tab, the Batch control tab is where this hash may be applied to experiment runs. In the drop-down menu of the Action column, any hash defined in Batch defs may be selected. There are plans to add more options such as the ability to apply a hash to every run or to automatically apply a hash to a run in real-time.

A hashtag is created which represents some script via the script's absolute path. This script contains the bsub command so that extra tags can be added and more customization of the bsub command is available. This hashtag can then be applied to events as needed.

Below is a simple example of a possible bash script. This script runs test.py and passes it any arguments that is passed to itself through "$@" and the log files are save in the logs directory with their job ID as the log file name. The python code can do analysis on the run this script was applied to

#!/bin/bash

bsub -q psdebugq -o LOG_DIRECTORY/%J.log python test.py "$@"

from time import sleep
from requests import post
from sys import argv
from os import environ

print 'This is a test function for the batch submitting.\n'


update_url = environ.get('BATCH_UPDATE_URL')
print 'The update_url is:', update_url, '\n'


params = argv
print 'The parameters passed are:'
for n, param in enumerate(params):
    print 'Param %d:' % n, param
print '\n'


for i in range(10):
    sleep(2)
    print 'Step:', i
    post(update_url, json={'counters' : {'Example Counter' : [i + 1, 'red']}})