The small data generation takes advantage of the local development of the PSANA framework. It can be customized and run both again the xtc files while they are being written on the ffb system (ongoing experiment only) as well as against the .xtc files on the offline system,. Parallel computing has been built-in. For a typical experiment Silke will set up the code in the directory /reg/d/psdm/<hutch>/<expname>/results/smalldata_tools for the offline running of /cds/data/drpsrcf/<hutch>/<experiment>/scratch/smalldata_tools when running in the new ffb.
A "driver" python file (typically
called SmallDataProducercalledsmalldata_producer.py in the examples subdirectory) can then be edited to, e.g., choose and optimize a different ROI on area detectors, define beam center for radial integration, define delay time range and bin sizes, etc. How to set up this "userData" or reduced data / feature extracted data is described in more detail in
SmallData Production - DetObject.
...
/reg/d/psdm/<hutch>/<expname>/hdf5/smalldata or /cds/data/drpsrcf/<hutch>/<experiment>/scratch/hdf5/smalldata
The filenames will be of the form: <expname>_Run<runnr>.h5
To run a single DAQ run, you can use (in the res directory or in your own private release):the ARP. If you would like to run an interactive test job with only a few events, you can use:
./acp_scripts/submitt_smd.sh smallDataRun -r <#> -e <experiment_name> During data taking, you can omit the "-e experiment_name" parameter and the jobs will be sent to a special queue with priority access.--nevents <#>
The full list of options is here:
(129) snelson@psanaphi10717) snelson@drp-srcf-eb003:/ |
|
regdpsdmxppxpptut15resultstools$ examples/smallDataRun -husage: ./examples/smallDataRun options
OPTIONS:
-r run
-e expname (def: current experiment, example: xppe7815)
-d directory for output files (default: ftc diretory of specified experiment)
-q queue name (default: psanaq if exp specified, psneh(hi)prioq for current exp)
-j
-n number of events (for testing)
-s run locally
arp_scripts/submit_smd.sh -h
submit_smd.sh:
Script to launch a smalldata_tools run analysis
OPTIONS:
-h|--help
Definition of options
-e|--experiment
Experiment name (i.e. cxilr6716)
-r|--run
Run Number
-d|--directory
Full path to directory for output file
-n|--nevents
Number of events to analyze
-q|--queue
Queue to use on SLURM
-c|--cores
Number of cores to be utilized
-f|--full
If specified, translate everything
-D|--default
If specified, translate only smalldata
-i|--image
If specified, translate everything & save area detectors as images
-T|--tiff
If specified, translate everything & save area detectors as images * single-event tiffs
--norecorder
If specified, don't use recorder data
--nparallel
Number of processes per node
--postTrigger
Post that primary processing done to elog to seconndary jobs can start
--interactive
Run the process live w/o batch system |
|
We will usually set up the production to run automatically through the ARP During data taking we typically run a "procServ" process (similar to a cron job) that checks the database for the last run number and if it finds one that has not been processed yet, it will submit a job to the batch queue. The number of jobs is tuned to use as few cores as necessary to process data at data taking speed to keep the time before the files are available to a minimum while keep the the queue as empty as possible. This is useful in cases where the reduction parameters are stable for sets of runs (most of the XPP and XCS experiments fall into this category).How to work with this process is described <here>.