Real-Time Analysis

Because critical experimental decisions must be made while an LCLS experiment is in progress, it is important to be able to analyze your data while it is being taken.  The DAQ supports an online GUI that can handle common tasks that is called AMI.  However, often this is not specialized enough for particular experiments.  To handle this case, psana-python allows you to run your analysis code while data is being taken in two different ways, using MPI-parallelization for performance.

Real-Time Analysis Using "DAQ Small Data"

This approach (supported for data after October 2015) is simplest, but has the disadvantage that it is not guaranteed to be able to keep up with the data (but in most cases it does when using MPI as shown in the example here). It typically uses dedicated "fast feedback" (FFB) resources as described here: Fast Feedback System.

Real-Time Analysis Using "Shared Memory"

This approach is somewhat more complex, but is guaranteed to analyze only "recent" events coming from the DAQ network (no disks involved, so it's fast!).  It has the disadvantage that it is not guaranteed to analyze every event in a run.  This approach is best when you need guaranteed real-time feedback, e.g. for tuning experiment/accelerator settings based on real-time plots.  To use this mode, create a DataSource object this like:

ds = DataSource('shmem=psana.0:stop=no')

Jobs in the shared memory doesn't know what the experiment is, so calibration directory needs to be specified like this (where "exp" is an experiment name like "xpptut15"):

calibDir = '/sdf/data/lcls/ds/%s/%s/calib' % (hutch,exp)
setOption('psana.calib-dir', calibDir)

There are other complications starting the code and gathering results from the MPI processes.  You must use the data gathering pattern here.  It is important not to use MPI collective operations like reduce/gather/bcast etc. during the event loop,  this can cause software hangs.  Contact the analysis group or your hutch Point Of Contact if you wish to use this mode.