...

Development of this application was stimulated by the discussion with Marcin Sikorski, doing xcs experiments.
Experimentalists Users need in real-time algorithm for calculation of image vs time selfauto-correlation function g2(tau) = <I(t)*I(t+tau)> / (<I(t)> * <I(t+tau)>), where I(t) is an intensity at time t, and tau is a delay between two measurements.
Typical experimental condition can be described as follows:

• Run duration is about one hour at frequency up to 120 Hz that gives up to 500-10^6 images.
• Use Currently typical imaging devise is a Princeton camera with 1300x1340 pixels.
• Need to calculate g2(tau) = <I(t)*I(t+tau)> / (<I(t)> * <I(t+tau)>) for each pixel, averaged over all possible image times t with time difference tau between images.
• A set of tau should have about 30-100 points in log scale uniformly covering the run time duration.
• Use for example xcsi0112-r0015: 500 images with 8 sec between images.
  Desired processing time time for evaluation of the auto-correlation function should be comparable with run duration ~<1 hour. Currently this algorithm takes a few hours that can not be used for fast feedback in real time experiment.

Algorithm

Basic idea is (1) to split image vs time for small parts in image, (2) to process each part on separate computer node, (3) to merge results at the end of processing. It is clear tignificant that significant speedup (~T/Nnodes) is achieved at the 2nd stage. These three stages are performed in separate C++ applications. Wrapping python script allows to submit job by a single command. It takes care about numerous file files and sub-process management in this job, as discussed below.

...

psana -c <config-file> <xtc-file-list>

For example:

./CorAnaSubmit.py psana c ana-misc-exp/psana-xcsi0112-r0015-img-auto-correlation.cfg /reg/d/psdm/XCS/xcsi0112/xtc/e167-r0015*

...

img-xcs-r0015-b0000.bin - file with a part of image vs time
img-xcs-r0015-b0001.bin
img-xcs-r0015-b0002.bin
img-xcs-r0015-b0003.bin
img-xcs-r0015-b0004.bin
img-xcs-r0015-b0005.bin
img-xcs-r0015-b0006.bin
img-xcs-r0015-b0007.bin
img-xcs-r0015-time.txt - list of time-record for all events in processed run.
img-xcs-r0015-med.txt - file with metadata. In particular it has the original image size, number of image parts for splitting, number of images in run, etc.

Time correlation processing

...

img-xcs-r0015-tau.txt          - string of tau values for which the auto-correlation function is evaluated
img-xcs-r0015-b0000-result.bin - auto-correlators for the part of the image for all tau values 
img-xcs-r0015-b0001-result.bin
img-xcs-r0015-b0002-result.bin
img-xcs-r0015-b0003-result.bin
img-xcs-r0015-b0004-result.bin
img-xcs-r0015-b0005-result.bin
img-xcs-r0015-b0006-result.bin
img-xcs-r0015-b0007-result.bin

...