Page History
This is a reference manual for algorithms implemented for the Integrated Data Processing Environment for Time Correlation Experiments. All work-flow can be split for main procedures:
- Pre-processing,
- Data processing, and
- Presentation of results,
covered by this note with extensive references to the specific code.
Content
Table of Contents |
---|
Pre-processing
Pre-processing algorithms consist of four procedures
- Dark run scan
...
- ,
- Pedestals averaging,
- Data scan, and
- Data averaging,
described in this section.
Dark run scan
Defines Defines for the dark run
- the number of events
- time records and average time interval
...
Pedestals averaging
Defines for the dark run
- pedestals
- RMS
- hot pixel mask
...
- Input image may be : double, float, int, uint16_t, uint8_t
- Calculation: intensity in double, statistics in unsigned
- Save arrays in txt for
Code Block save2DArrayInFile<double> ( m_aveFile, m_ave, m_rows, m_cols, m_print_bits & 16 ); save2DArrayInFile<double> ( m_rmsFile, m_rms, m_rows, m_cols, m_print_bits & 16 ); if (m_do_mask) save2DArrayInFile<int> ( m_hotFile, m_hot, m_rows, m_cols, m_print_bits & 16 );
...
PSANA module ImgAlgos.Tahometer
...
Code Block |
---|
[psana] #psana -m psana_examples.DumpPrinceton /reg/d/psdm/XCS/xcsi0112/xtc/e167-r0015-s00-c00.xtc #files = /reg/d/ana12/xcs/xcsi0112/xtc/e167-r0015-*.xtc modules = ImgAlgos.Tahometer ImgAlgos.ImgTimeStampList ImgAlgos.IntensityMonitorsData #skip-events = IS_NOT_USED #events = FOR_ALL_EVENTS [ImgAlgos.Tahometer] print_bits = 11 [ImgAlgos.ImgTimeStampList] print_bits = 13 out_file = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-data-scan-tstamp-list.txt [ImgAlgos.IntensityMonitorsData] print_bits = 45 #file_type = bin #file_data = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-data-scan-mons-data.txt file_data = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-data-scan-mons-data.txt file_header = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-data-scan-mons-comments.txt #======EOF====== |
...
PSANA module ImgAlgos.ImgTimeStampList
- ImgAlgos.ImgTimeStampList.cpp
- ImgAlgos.ImgTimeStampList.h
Produces file with time records*data-scan-tstamp-list.txt
:Code Block 0 0.000000 0.000000 20120616-080236.671607864 5366 0 0 1 8.026429 8.026429 20120616-080244.698036743 8255 1 1 2 16.144788 8.118359 20120616-080252.816395836 11177 2 2 3 24.154835 8.010048 20120616-080300.826443448 14060 3 3 4 32.281937 8.127102 20120616-080308.953545010 16985 4 4 5 40.400633 8.118696 20120616-080317.072241060 19907 5 5 ...
...
PSANA module ImgAlgos.IntensityMonitorsData
...
IntensityMonitorsData produces files:
...
Code Block |
---|
[psana] #files = /reg/d/ana12/xcs/xcsi0112/xtc/e167-r0015-*.xtc skip-events = 0 events = 500 modules = ImgAlgos.Tahometer ImgAlgos.PrincetonImageProducer ImgAlgos.ImgAverage ImgAlgos.ImgMaskEvaluation [ImgAlgos.Tahometer] print_bits = 7 [ ImgAlgos.PrincetonImageProducer ] source = DetInfo(:Princeton) key_in = key_out = img print_bits = 1 [ImgAlgos.ImgAverage] source = DetInfo(:Princeton) key = img avefile = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-data-ave.txt rmsfile = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-data-rms.txt print_bits = 25 #evts_stage1 = 100 #evts_stage2 = 100 #gate_width1 = 200 #gate_width2 = 50 [ImgAlgos.ImgMaskEvaluation] source = DetInfo(:Princeton) key = img file_mask_satu = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-satpix-mask-level-65000ADU.txt file_mask_nois = file_mask_comb = file_frac_satu = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-satpix-frac-level-65000ADU.txt file_frac_nois = thre_satu = 65000 frac_satu = 0 dr_SoN_ave = 1 thre_SoN = 5 frac_nois = 0.05 print_bits = 29 #======EOF====== |
...
PSANA module ImgAlgos.ImgMaskEvaluation
Saturated pixel mask
The saturated pixel mask is produced in ImgAlgos.ImgMaskEvaluation
...
Data processing
...
The ImgAlgos.ImgMaskEvaluation
produces the saturated pixel mask; the image size txt file *-satpix-mask-level-65000ADU.txt
, consistiong 0(zero) or 1(one) for each bin.
Data processing
Data processing stage consists of three procedures:
- Spliting,
- Processing, and
- Merging,
described in this section.
Spliting
Submit command
Code Block |
---|
psana -c /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora-xcsi0112-r0015-split.cfg /reg/d/ana12/xcs/xcsi0112/xtc/e167-r0015-*.xtc |
...
Code Block |
---|
[psana] # Command to run this script from release directory: # psana -c ImgAlgos/data/psana-split.cfg <path-to-xtc-file-name-pattern-for-one-run> #files = /reg/d/ana12/xcs/xcsi0112/xtc/e167-r0015-*.xtc skip-events = 0 events = 500 modules = ImgAlgos.Tahometer ImgAlgos.PrincetonImageProducer ImgAlgos.ImgCalib ImgAlgos.ImgIntMonCorr ImgAlgos.ImgIntForBins ImgAlgos.ImgVsTimeSplitInFiles ImgAlgos.ImgAverage [ImgAlgos.Tahometer] print_bits = 7 [ ImgAlgos.PrincetonImageProducer ] source = DetInfo(:Princeton) key_in = key_out = img print_bits = 1 [ImgAlgos.ImgCalib] source = DetInfo(:Princeton) key_in = img key_out = calibrated fname_peds = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0020-peds-ave.txt fname_bkgd = fname_gain = fname_mask = fname_rms = threshold_nrms = 0 do_threshold = true threshold = 20.0 below_thre_value = 0 print_bits = 5 [ImgAlgos.ImgIntMonCorr] source = DetInfo(:Princeton) key_in = calibrated key_out = imon_corrected fname_imon_cfg = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora-xcsi0112-r0015-imon-cfg.txt print_bits = 1 [ImgAlgos.ImgIntForBins] source = DetInfo(:Princeton) key_in = imon_corrected fname_map_bins = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora-r0015-map-static-q.txt fname_int_bins = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora-r0015-int-static-q.txt number_of_bins = 1 print_bits = 33 [ImgAlgos.ImgVsTimeSplitInFiles] source = DetInfo(:Princeton) key = imon_corrected fname_prefix = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora file_type = bin add_tstamp = false ampl_thr = 20.0 ampl_min = 01 nfiles_out = 8 print_bits = 29 #======EOF====== |
PSANA module ImgAlgos.ImgCalib
PSANA module ImgAlgos.ImgIntMonCorr
...
[ImgAlgos.ImgAverage]
source = DetInfo(:Princeton)
key = calibrated
avefile = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-data-ave.txt
rmsfile = /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-xcsi0112-r0015-data-rms.txt
#======EOF======
|
PSANA module ImgAlgos.
...
ImgCalib
- ImgAlgos.ImgIntForBinsImgCalib.cpp
- ImgAlgos.ImgIntForBinsImgCalib.h
PSANA module ImgAlgos.ImgVsTimeSplitInFiles
Processing
Submit command
Code Block |
---|
corana -f /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora-r0015-b0000.bin -t ./tau-list.txt
|
Module ImgAlgos.corana
Module ImgAlgos.CorAnaData.cpp
...
- This module is responsible for basic image intensity transformation. For correlation analysis it applys a couple of corrections:
- subtract pedestals and
- apply LLD threshold.
PSANA module ImgAlgos.ImgIntMonCorr
- ImgAlgos.ImgIntMonCorr.cpp
- ImgAlgos.ImgIntMonCorr.h
Gets intensity monitor configuration fileimon-cfg.txt
produced in GUI:Code Block BldInfo(FEEGasDetEnergy) FEEGasDetEnergy 1 1 1 1 0 0 -1.0000 -1.0000 1.0000 BldInfo(XCS-IPM-02) XCS-IPM-02 1 1 1 1 0 0 -1.0000 -1.0000 1.0000 BldInfo(XCS-IPM-mono) XCS-IPM-mono 1 1 1 1 0 0 -1.0000 -1.0000 -1.0000 DetInfo(XcsBeamline.1:Ipimb.4) Ipimb.4 1 1 1 1 0 0 -1.0000 -1.0000 1.0000 DetInfo(XcsBeamline.1:Ipimb.5) Ipimb.5 1 1 1 1 0 0 -1.0000 -1.0000 1.0000
PSANA module ImgAlgos.ImgIntForBins
Gets the file *-map-static-q.txt
with a map of static q bin numbers
Produces the file *-int-static-q.txt
with intensities averaged over statis q bins:
Code Block |
---|
0 44.460
1 27.982
2 62.282
3 0.875
4 73.732
5 698.901
6 4.923
7 227.669
8 28.298
...
|
PSANA module ImgAlgos.ImgVsTimeSplitInFiles
Gets/produces the file *-tau.txt
with a list of tau intervals in terms of time(event) indexes, for example:
Code Block |
---|
1 2 3 4 5 6 7 8 9 10 12 14 16 20
|
Produces the file *-med.txt
with metadata parameters from splitting algorithm:
Code Block |
---|
IMAGE_ROWS 1300
IMAGE_COLS 1340
IMAGE_SIZE 1742000
NUMBER_OF_FILES 8
BLOCK_SIZE 217750
REST_SIZE 0
NUMBER_OF_IMGS 500
FILE_TYPE bin
DATA_TYPE f
TIME_SEC_AVE 8.088413
TIME_SEC_RMS 0.063639
TIME_INDEX_MAX 499
|
Processing
Submit command
Code Block |
---|
corana -f /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora-r0015-b0000.bin -t ./tau-list.txt
|
Module ImgAlgos.corana
Module ImgAlgos.CorAnaData
Evaluation of correlators:
Code Block |
---|
ImgAlgos/include/CorAna.h: typedef float cor_t;
void CorAnaData::evaluateCorTau(unsigned tau) // tau in number of frames between images
{
m_log << "\nCorAnaData::evaluateCorTau(tau): tau=" << tau;
std::fill_n(m_sum_gi, m_blk_size, double(0));
std::fill_n(m_sum_gf, m_blk_size, double(0));
std::fill_n(m_sum_g2, m_blk_size, double(0));
std::fill_n(m_sum_st, m_blk_size, unsigned(0));
std::fill_n(m_cor_gi, m_blk_size, cor_t(0));
std::fill_n(m_cor_gf, m_blk_size, cor_t(0));
std::fill_n(m_cor_g2, m_blk_size, cor_t(0));
for (unsigned ti=0; ti<m_tind_size-tau; ti++) {
unsigned tf=ti+tau;
if ( ! (tf<m_tind_size) ) break;
// get the event index in array for time index
int evi = m_tind_to_evind[ti];
int evf = m_tind_to_evind[tf];
// If the event does not exist for specified time index -> skip it in sum
if(evi<0) continue;
if(evf<0) continue;
sumCorTau((unsigned)evi,(unsigned)evf);
}
}
//----------------
void CorAnaData::sumCorTau(unsigned i, unsigned f) // i and f are the event indexes
{
data_t* p_i = &m_data[i*m_blk_size];
data_t* p_f = &m_data[f*m_blk_size];
double Ii, If;
for(unsigned pix=0; pix<m_blk_size; pix++) {
Ii = p_i[pix];
If = p_f[pix];
m_sum_gi[pix] += Ii;
m_sum_gf[pix] += If;
m_sum_g2[pix] += Ii*If;
m_sum_st[pix] ++;
}
}
|
Averaging:
Code Block |
---|
for(unsigned pix=0; pix<m_blk_size; pix++) {
if(m_sum_st[pix]<1) continue;
m_cor_gi[pix] = cor_t( m_sum_gi[pix] / m_sum_st[pix] );
m_cor_gf[pix] = cor_t( m_sum_gf[pix] / m_sum_st[pix] );
m_cor_g2[pix] = cor_t( m_sum_g2[pix] / m_sum_st[pix] );
}
|
Module ImgAlgos.CorAna
...
Code Block |
---|
corana_merge -f /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora-r0015-b0000-result.bin -t /reg/neh/home1/dubrovin/LCLS/PSANA-V01/work/t1-cora-r0015-tau.txt
|
Module ImgAlgos.corana_merge
PSANA module ImgAlgos.CorAnaMergeFiles
tau.txt
|
Module ImgAlgos.corana_merge
Module ImgAlgos.CorAnaMergeFiles
Presentation of results
Presentation of results is implemented in the GUIViewResults.py
, which basically is a panel of buttons for different plots.
Summary
In this note we presented a functional description of implemented procedures for time correlation analysis. All modules have convenient reference to the latest version of the code.
References
- Module
ImgAlgos.GlobalMethods
- ImgAlgos.GlobalMethodsImgAlgos.CorAnaMergeFiles.cpp
- ImgAlgos.CorAnaMergeFilesGlobalMethods.h
Presentation of results
<!--
...
- Modules of
ImgAlgos
- ImgAlgos.*.cpp
- ImgAlgos.*.h
-->
...