Versions Compared

Old Version 23

changes.mady.by.user Mikhail Dubrovin

Saved on

compared with

New Version 24

changes.mady.by.user Mikhail Dubrovin

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Module

Functionality

ImgVsTimeSplitInFiles

splitter

CorAna

base class with common methods

CorAnaData

data processing for split files

CorAnaInputParameters

provides storage for input parameters

CorAnaMergeFiles

merging algorithm

CorAnaProcResults

Example showing how to access results using C++ and produce a table for presentation

CorAnaPars.py

singleton class for parameter storage in the wrapping file manager

CorAnaSubmit.py

global methods for the file manager

app/corana_submit

pythonic script which defines the sequence of procedures

app/corana.cpp

main module for the part of image vs time correlation processing

app/corana_merge.cpp

main module for merging

app/corana_procres.cpp

main module for merging

data/psana-corana...-auto-correlation.cfg

psana configuration file for ImgVsTimeSplitInFiles

data/PlotCorAnaResults.py

example of the python script which plots the resulting graphics

...

Code Block
psana -c ImgAlgos/data/psana-xcsi0112-r0015-img-auto-correlationcorana.cfg  /reg/d/psdm/XCS/xcsi0112/xtc/e167-r0015-*

where ImgAlgos/data/psana-xcsi0112-r0015-img-auto-correlationcorana.cfg is an example of the configuration script for psana and /reg/d/psdm/XCS/xcsi0112/xtc/e167-r0015-* are the input xtc files for particular run.

...

Produces the files:

Code Block
imgcor-xcsana-r0015-b0000.bin - file with a part of image vs time
imgcor-xcsana-r0015-b0001.bin
imgcor-xcsana-r0015-b0002.bin
imgcor-xcsana-r0015-b0003.bin
imgcor-xcsana-r0015-b0004.bin
imgcor-xcsana-r0015-b0005.bin
imgcor-xcsana-r0015-b0006.bin
imgcor-xcsana-r0015-b0007.bin
imgcor-xcsana-r0015-time.txt - list of time-records for all events in processed run.
imgcor-xcsana-r0015-time-ind.txt - list of time-records for all events in processed run with time index.
imgcor-xcsana-r0015-med.txt - file with metadata. In particular it has the original image size, number of image parts for splitting, number of images in run, etc.

...

  • The <int16_t> image data array is split for ordered number of equal parts (by the parameters nfiles_out in psana *-corana.cfg file) number of equal parts and each part is saved in the output imgcor-xcsana-r0015-b####.bin file sequentially for all selected events.
  • The appropriate time record for selected event is saved in the file imgcor-xcsana-r0015-time.txt.
  • At the end of the splitting procedure:
    • the average time difference and its rms between sequential events is evaluated for all recorded time records.
    • The file imgcor-xcsana-r0015-time.txt is re-processed and for each record the time index is evaluated as unsigned value of
      Code Block
      <time-index> = (<event-time> + 0.5 <average-time-between-events>) /  <average-time-between-events>
    • Event record with time index is saved in the file imgcor-xcsana-r0015-time-ind.txt
  • All metadata parameters which are required for further processing, such as input parameters, image size, <average-time-between-events, maximal value of the time index etc., are saved in file imgcor-xcsana-r0015-med.txt.
Note

This approach allows to apply the modest event selection algorithms in psana pre-processing stage.
But, it still based on uniform time indexing...
Q: Is it really good assumption for this kind of experiments?

...

For example the interactive and batch mode commands:

Code Block
corana -f imgcor-xcsana-r0015-b0001.bin -t my-tau.txt 
bsub -q psfehq -o log-file 'corana -f imgcor-xcsana-r0015-b0000.bin'

Where ~/<path-from-home-dir>/ is an arbitrary directory, but this directory should exist before running this script.

Produce files:

Code Block
imgcor-xcsana-r0015-tau.txt          - string of {{tau}} values for which the auto-correlation function is evaluated
imgcor-xcsana-r0015-b0000-result.bin - auto-correlators for the part of the image for all {{tau}} values 
imgcor-xcsana-r0015-b0001-result.bin
imgcor-xcsana-r0015-b0002-result.bin
imgcor-xcsana-r0015-b0003-result.bin
imgcor-xcsana-r0015-b0004-result.bin
imgcor-xcsana-r0015-b0005-result.bin
imgcor-xcsana-r0015-b0006-result.bin
imgcor-xcsana-r0015-b0007-result.bin

Merging results

...

Code Block
corana_merge -f imgcor-xcsana-r0015-b0001-result.bin -t my-tau.txt

This procedure produces file:

Code Block
imgcor-xcsana-r0015-image-result.bin

Example of how to get and process results

...

Code Block
corana_submit -c ImgAlgos/data/psana-xcsi0112-r0015-img-auto-correlationcorana.cfg -t my-tau.txt /reg/d/psdm/XCS/xcsi0112/xtc/e167-r0015-s00-c00.xtc

...

  • File with split-image data for selected events imgcor-xcsana-r0015-b000N.bin:
    Currently this file contains <uint16_t> amplitude for each pixel in binary format for:
    Code Block
    <data-for-img-partN-of-img1> <data-for-img-partN-of-img2> ... <data-for-img-partN-of-imgLast>
  • File with metadata parameters imgcor-xcsana-r0015-med.txt:
    Code Block
    IMAGE_ROWS      1300
IMAGE_COLS      1340
IMAGE_SIZE      1742000
NUMBER_OF_FILES 8
BLOCK_SIZE      217750
REST_SIZE       0
NUMBER_OF_IMGS  500
FILE_TYPE       bin
DATA_TYPE       uint16_t
TIME_SEC_AVE    8.088413
TIME_SEC_RMS    0.063639
TIME_INDEX_MAX       499
  • File with image time records imgcor-xcsana-r0015-time.txt:
    Code Block
         1        0.000000  0.000000  20120616-080236.671607864    5366      0
     2        8.026429  8.026429  20120616-080244.698036743    8255      1
     3       16.144788  8.118359  20120616-080252.816395836   11177      2
     4       24.154835  8.010048  20120616-080300.826443448   14060      3
    ...
    where each record has:
    Code Block
    <image-in-file#> <t(sec)-from-the-1st-event> <dt(sec)> <time-stamp> <fiducials> <event#-since-configure>
  • File with image time records and evaluated time index imgcor-xcsana-r0015-time-ind.txt:
    Code Block
         1        0.000000  0.000000  20120616-080236.671607864    5366      0        0 
     2        8.026429  8.026429  20120616-080244.698036743    8255      1        1 
     3       16.144788  8.118359  20120616-080252.816395836   11177      2        2 
     4       24.154835  8.010048  20120616-080300.826443448   14060      3        3 
     5       32.281937  8.127102  20120616-080308.953545010   16985      4        4 
    ...
    where each record has:
    Code Block
    <image-in-file#>  <t(sec)-from-the-1st-event> <dt(sec)> <time-stamp> <fiducials> <event#-since-configure> <time-index-starting-from-0>
  • File with split-image correlators for each value of tau imgcor-xcsana-r0015-b000N-result.bin:
    Currently it saves <float> correlator for each pixel in binary format for:
    Code Block
    <corr-for-img-partN-of-tau1> <corr-for-img-partN-of-tau2> ... <corr-for-img-partN-of-tauLast>
  • my-tau.txt:
    Code Block
     1 3 5 7 9 10 12 14 16 18 20 24 28 30 32 36 40 ... 160 180 200 240 280 300 320 360 400
    contains the tau values presented in terms of number of ordered images in the file.

...

Code Block
./ImgAlgos/data/PlotCorAnaResults.py work_corana/imgcor-xcsana-r0015-hist.txt

Note

Another option is to use python script for direct processing of the resulting files.
This is not elaborated yet.
Q: What kind of further processing is desired and what tools are going to be used?