Motivation

Development of this application was stimulated by the discussion with Marcin Sikorski (meeting on 2012-08-30), doing xcs experiments.
Users need in real-time algorithm for calculation of image vs time auto-correlation function

g2(tau) = <I(t)*I(t+tau)> / (<I(t)> * <I(t+tau)>),

where I(t) is an image intensity at time t, and tau is a delay between two measurements.
Typical experimental condition can be described as follows:

• Run duration is about one hour at frequency up to 120 Hz that gives up to 10^5-10^6 images.
• Currently typical imaging devise is a Princeton camera with 1300x1340 pixels.
• Need to calculate g2(tau) for each pixel, averaged over all possible image times t with time difference tau between images.
• A set of tau should have about 30-100 points in log scale uniformly covering the run duration.
• Use for example xcsi0112-r0015: 500 images with 8 sec delay between images.
  Desired time for evaluation of the auto-correlation function should be comparable with run duration <1 hour. Currently this algorithm takes a few hours that can not be used for fast feedback in real time experiment.

In first approximation this problem was solved, as it is described in the Command Line Interface For Time Correlation Analysis. However, in order to be useful, the command line algorithm needs to be integrated in the global analysis system, which de bene esse called as Integrated Analysis Environment for Time Correlation Experiments and is discussed in IDPE for TCE in XCS - Problems and tentative solutions.

GUI Implementation

Updated on 2013-02-13

Three versions of GUI were implemented for this application. It was decided to use the GUIMainTB
layout, where:

• All input windows are integrated in one with tab-bar for switching and custom tool bar on the top
• Logger and File Browser windows are isolated
• All plots are isolated, although similar plots appear in the same windows.

Code location

Everything resides in CorAna package, which is included in LCLS offline releases beginning from ana-0.7.16.

Modules

Updated on 2012-12-10
Notations:
- is done (at least it is assumed for now...)
- needs more work
- test or deprecated modules
All module names beginning with letters GUI implements different Graphical User Interfaces.

Status of the project

Updated on 2013-02-07

PSANA modules

For dark run pre-processing

Scaner
Runs in batch for the dark file to get preliminary information.
ImgAlgos.ImgTimeStampList

• counts number of events in the file
• makes file with time stamps
• evaluate time intervals between frames (for dark run)

Pedestals
Runs in batch for the dark file to get averaged pedestals.
ImgAlgos.PrincetonImageProducer gets image from event as an ndarray object
ImgAlgos.Tahometer evaluates performance of the batch job
ImgAlgos.ImgAverage produces file with averaged image for given range of events.

For data pre-processing

Scaner
Runs in batch for the data file to get preliminary information.
ImgAlgos.Tahometer evaluates performance of the batch job, counts number of events in the data file
ImgAlgos.ImgTimeStampList makes file with time stamps and time record counters for tau
ImgAlgos.IntensityMonitorsData makes file with intensity monitor records

Average
Runs in batch for the data file to get averaged image
ImgAlgos.Tahometer evaluates performance of the batch job
ImgAlgos.PrincetonImageProducer gets image from event as an ndarray object
ImgAlgos.ImgAverage produces file with averaged image for given range of events
ImgAlgos::ImgMaskEvaluation - module is configured to evaluate masks:

• saturated mask - pixel is considered as saturated if its amplitude exceeds the threshold at least once per run
• noise mask - pixel is considered as noisy if its amplitude exceeds the MEAN+5*RMS in 5% of events. The "noise" MEAN and RMS are evaluated for 8 (or less on the boarder) surrounding pixels.

For data processing

Split
ImgAlgos.Tahometer
ImgAlgos.PrincetonImageProducer
ImgAlgos.ImgVsTimeSplitInFiles
ImgAlgos::ImgCalib module is available, but needs to be updated and properly configured in psana.cfg. This module is intended to:

• subtract pedestals obtained for dark run
• account for threshold (LLD) constant or in number of RMS
  ImgAlgos::??? Intensity monitors

Process
Data processing is implemented in stand alone (non-psana) modules
ImgAlgos.CorAna.cpp
ImgAlgos.CorAnaInputParameters.cpp
ImgAlgos.CorAnaData.cpp

Merge
CorAnaMergeFiles.cpp - saves binary file for float(32) with shape (Ntau,3,rows,cols), where 3 stands for <Ip>, <If>, and <Ip*If>
Can be acessed in python as

    sp.cor_arr = np.fromfile(sp.fname, dtype=np.float32)
    <image-size> = rows * cols
    nptau = <file-size>/<image-size>/3
    sp.cor_arr.shape = (nptau, 3, rows, cols)

GUI

The system of GUIs, consisting of dozens of CorAna.GUI... modules is implemented in the draft approximation. Roughly it reproduces all features of the old program.

Input parameters GUIs

CorAna.GUI... most of them are available. Will be added or extended if necessary.
Files - define input files and do pre-processing
Setup Info
Analysis Info
System
Intensity Monitor needs to be used at processing stage

Run GUI

Input - short summary of input info for data processing
Split - control and monitoring for the 1st stage of processing
Process - ... 2nd stage ...
Merge - ... 3d stage ...
Auto - ... for all 3 stages ...

View Results GUI

Contains a set of control fields for presentation of results

Graphics

PlotArray*.py - for Intensity Monitors
PlotImgSpe*.py - for images, partition maps, masks etc.
PlotTime*.py - for time stamp monitoring
PlotG2*.py - G2 plot peresentation
MaskEditor*.py, Drag*.py - Mask Editor for

• region of interest (ROI-mask)
• blemish mask

Infrastructural modules

Infrastructural modules provide basic infrastructure of the project.
ConfigParameters.py, ConfigParametersCorAna.py, and GUIConfigParameters.py provides convenient approach for maintenance of all configuration parameters.
Infrastructural modules Logger.py and GUILogger.py provides a generic approach to logging system.
Module ConfigFileGenerator.py use current settings of configuration parameters and stub-file scripts from CorAna/data/scripts/ and generates the psana configuration files.
Module FileNameManager.py is a single place which provides a dynamic file names for current version of the configuration parameters.
GlobalUtils.py - global utilities for common operations
BatchJob.py - superclass for batch job submission
BatchJobPedestals.py - pre-processing for dark run files
BatchJobData.py - re-processing for data files
BatchJobCorAna.py - main data processing - calculation algorithm.

Progress

See Weekly progress of the IDPE project for TCE

To-do list

ImgAlgos::ImgCalib - needs in update and configuration in psana.cfg

Processing (at psana Split level?)
Account for thresholds in ImgAlgos::ImgCalib - needs in update and configuration in psana.cfg
Intensity monitors: needs to be used at processing in psana

View Results
geometry for reflected beam
fits

ELog
Add submission in ELog using responce ID and Igor's new interface: Python module for posting message into ELog
GlobalUtils: comment/uncomment submission in ELog