...

Development of this application was stimulated by the discussion with Marcin Sikorski (meeting on 2012-08-30), doing xcs experiments.
Users need in real-time algorithm for calculation of image vs time auto-correlation function

...

• Run duration is about one hour at frequency up to 120 Hz that gives up to 10^5-10^6 images.
• Currently typical imaging devise is a Princeton camera with 1300x1340 pixels.
• Need to calculate g2(tau) for each pixel, averaged over all possible image times t with time difference tau between images.
• A set of tau should have about 30-100 points in log scale uniformly covering the run duration.
• Use for example xcsi0112-r0015: 500 images with 8 sec delay between images.
  Desired time for evaluation of the auto-correlation function should be comparable with run duration <1 hour. Currently this algorithm takes a few hours that can not be used for fast feedback in real time experiment.

More details

2012-09-10 meeting: In order to be useful this application should do correct math, accounts for image mask, discard bad events, noisy and "bright" pixels, apply normalization etc, and have a convenient GUI. Below is a list of requirements (marked as R with suggested solutions ( marked as S.

Pedestals

R: "dark" run name should be provided by user and pedestals should be evaluated and applied for all runs until the "dark" run name has not changed.
S: For pedestals evaluation: use available ImgAlgos::ImgAverage psana module for "dark" run, which produces file with averaged over events pedestals (also produces the file with rms values).
S: For pedestals subtraction: use ImgAlgos::ImgCalib psana module right before evaluation of pedestals; the pedestals will be subtracted and corrected image will be retained in the event.

Low level threshold.

R: Image pixel intensity physically can't be negative. Low amplitude noise should be suppressed by the threshold. The threshold amplitude should be provided by user (along with substituting amplitude).
S: Add this feature to the ImgAlgos::ImgCalib psana module, right after pedestal evaluation.

Here is my comments

Algorithm

Basic idea is (1) to split image vs time for small parts in image, (2) to process each part on separate computer node, (3) to merge results at the end of processing. It is clear that significant speedup (about T/N_nodes_) is achieved at the 2nd stage. These three stages are performed in separate C++ applications. Wrapping python script allows to submit job by a single command. It takes care about file and sub-process management in this job, as described below.

...