Motivation

Development of this application was stimulated by the discussion with Marcin Sikorski (meeting on 2012-08-30), doing xcs experiments.
Users need in real-time algorithm for calculation of image vs time auto-correlation function

...

{HTMLcomment:hidden}
Here is my comment
{HTMLcomment}

GUI Implementation

Updated on 2013-04-01

As an example of GUI style Marcin suggested to use XPCSGUI - earlier implementation of similar application.

...

h3 Plots for intensity in q-static bins

Intensity monitors

FEEGasDetEnergy, XCS-IPM-02, XCS-IPM-mono, XcsBeamline.1:Ipimb.4, and XcsBeamline.1:Ipimb.5

Reflective scattering geometry

Source code from Marcin: q_functions.txt

Code location

Everything resides in CorAna package, which is included in LCLS offline releases beginning from ana-0.7.16.
References to the specific algorithms can be found in Algorithms for Time Correlation Experiments.
Original data processing code from Marcin is described in Note on XCS code from Marcin.

Modules

In tables below we list modules from packages CorAna and ImgAlgos, which were developed for this project.
Notations:
- is done (at least it is assumed for now...)
- needs more work
- test or deprecated modules

Package CorAna

Updated on 2013-10-14

All module names beginning with letters GUI implements different Graphical User Interfaces.

...

Modules used from package ImgAlgos

Updated on 2013-10-14

List of modules in alphabetic order:

Functional description

Updated on 2013-10-14
In this section modules are listed in functional order.

PSANA modules

Dark run pre-processing

Scaner
Runs in batch for the dark file to get preliminary information.
ImgAlgos.ImgTimeStampList

...

Pedestals
Runs in batch for the dark file to get averaged pedestals.
ImgAlgos.PrincetonImageProducer gets image from event as an ndarray object
ImgAlgos.Tahometer evaluates performance of the batch job
ImgAlgos.ImgAverage produces files with averaged, rms-spread, and hot-pixel mask for images in a given range of events.

Data pre-processing

Scaner
Runs in batch for the data file to get preliminary information.
ImgAlgos.Tahometer evaluates performance of the batch job, counts number of events in the data file
ImgAlgos.ImgTimeStampList makes file with time stamps and time record counters for tau
ImgAlgos.IntensityMonitorsData makes file with intensity monitor records

...

• saturated mask - pixel is considered as saturated if its amplitude exceeds the threshold at least once per run
• noise mask - pixel is considered as noisy if its amplitude exceeds the MEAN+5*RMS in 5% of events. The "noise" MEAN and RMS are evaluated for 8 (or less on the boarder) surrounding pixels. This mask is not used in current analysis.

Data processing

Split
ImgAlgos.Tahometer for performance evaluation.
ImgAlgos.PrincetonImageProducer
ImgAlgos::ImgCalib module is configured to:

...

    sp.cor_arr = np.fromfile(sp.fname, dtype=np.float32)
    <image-size> = rows * cols
    nptau = <file-size>/<image-size>/3
    sp.cor_arr.shape = (nptau, 3, rows, cols)

GUI

The system of GUIs, consisting of dozens of CorAna.GUI... modules is implemented in the draft approximation. Roughly it reproduces all features of the old program.

Input parameters GUIs

CorAna.GUI... most of them are available. Will be added or extended if necessary.
Files - define input files and do pre-processing
Setup Info
Analysis Info
System
Intensity Monitor

Run GUI

Input - short summary of input info for data processing
Split - control and monitoring for the 1st stage of processing
Process - ... 2nd stage ...
Merge - ... 3d stage ...
Auto - ... for all 3 stages ...

View Results GUI

Contains a set of control fields for presentation of results
Direct and reflected beam geometry is implemented in ViewResults.py. Currently the switch between the direct and reflected beam geometry is used from tab status variable: cp.exp_setup_geom.value().
If the PV variable will be used, then the switch should be changed in ViewResults.get_q_map(sp) module.

Graphics

PlotArray*.py - for intensity monitors
PlotImgSpe*.py - for images, partition maps, masks etc.
PlotTime*.py - for time stamp monitoring
PlotG2*.py - G2 plot
PlotGraph*.py - I(q) and I(q,t) plot
MaskEditor*.py, Drag*.py - Mask Editor for

• region of interest (ROI-mask)
• blemish mask
  ViewResults.pyQ - all evaluations for resulting array of correlators
• maps for pixel x,y coordinates, r, phi
• q maps for transmission and reflective geometry
• etc.

Infrastructural modules

Infrastructural modules provide basic infrastructure of the project.
ConfigParameters.py, ConfigParametersCorAna.py, and GUIConfigParameters.py provides convenient approach for maintenance of all configuration parameters.
Infrastructural modules Logger.py and GUILogger.py provides a generic approach to logging system.
Module ConfigFileGenerator.py use current settings of configuration parameters and stub-file scripts from CorAna/data/scripts/ and generates the psana configuration files.
Module FileNameManager.py is a single place which provides a dynamic file names for current version of the configuration parameters.
GlobalUtils.py - global utilities for common operations.
BatchJob.py - superclass for batch job submission.
BatchJobPedestals.py - pre-processing for dark run files.
BatchJobData.py - re-processing for data files.
BatchJobCorAna.py - main data processing - calculation algorithm.
RecordsFromFiles.py - class helps to access data in files.

Status of the project

• This project is in the stage of comparison of results between old Marcin's scripts and this application.

Comparison of results

Conditions

1. dark run: /reg/d/ana12/xcs/xcsi0112/xtc/e167-r0020-s00-c00.xtc use all 75 events
2. data run: /reg/d/ana12/xcs/xcsi0112/xtc/e167-r0015-s00-c00.xtc use all 500 events
3. do not use any intensity monitor selection or correction
4. do not use any mask including hot pixel, saturation, blemish, ROI, and restriction on the image size.
5. use LLD as a constant ADU threshold = 20
6. use a single q-phi static bin
7. use a single q-phi dynamic bin
8. q value is not an issue for current comparison, so geometry does not matter.

...

1 2 3 4 5 6 7 8 9 10 12 14 16 20

See Data Processing for Time Correlations Progress section for more details on comparison.

Progress

Weekly progress of the IDPE project for TCE

To-do list

Processing (at psana Split level)
Get cp.photon_energy, cp.nominal_angle and other PV variables from data scan and setup confiruration

...

• plot for all fits with beta=1 and float,
• tau0 vs q-dynamic
• C vs q-dynamic
• B vs q-dynamic
• beta vs q-dynamic

References

Photon Correlation Spectroscopy, article in wikipedia
Algorithms for Time Correlation Experiments
Note on XCS code from Marcin