Analysis of cxif5315
These commands setup the psana environment, get the code, analyze 5 events with 2 separate hit finders in the equator/arc regions, display the found hits, and save the resulting hit information to a "small-data" text file (similar to cheetah):
ssh -X pslogin.slac.stanford.edu ssh -X psana source /reg/g/psdm/etc/ana_env.sh cp -r /reg/g/psdm/tutorials/cxi/cxif5315 . cd cxif5315 python psana-cxif5315-r0169-cspad-ds2-NDArrDropletFinder.py |
This command reads the resulting small-data text file and can select specific events for reanalysis. The name of the txt file changes for different runs, so insert the name of your .txt file in the work/ subdirectory:
python readSmallData.py exp=cxif5315:run=169 work/PutNameOfYourTxtFileHere.txt |
This same script can also read cheetah-produced text files, and only analyze specific events
Use my local calibration store
[psana] calib-dir = /reg/neh/home1/dubrovin/LCLS/rel-mengning/calib |
Dark files were obtained and deployed using Calibration Management Tool
exp=cxif5315:run=147 dark average, rms, and average difference between runs 147-198:
Difference of average dark images between runs 147-198 is small, 0.3 ADU, difference rms=0.6 ADU, this proves that CxiDs2.0:Cspad.0 is stable during 50 runs.
Image array averaging is useful procedure for many purpose;
1) Download configuration file psana-cxif5315-cspad-ds2-NDArrAverage.cfg
2) To get average/rms/max over 10000 events data of run 169 use command:
psana -c psana-cxif5315-cspad-ds2-NDArrAverage.cfg -n 1000 exp=cxif5315:run=169 |
This command produces files with arrays of size (32x185x388)
3) To get average/rms/max over 1000 events (skipping 5000) data for water sample of run 162 use command:
psana -c psana-cxif5315-cspad-ds2-NDArrAverage.cfg -s 5000 -n 1000 exp=cxif5315:run=162 |
This command produces files with arrays of size (32x185x388)
In accordance with CSPAD Alignment, use geometry file:
/reg/g/psdm/detector/alignment/cspad/calib-cxi-ds2-2015-01-20/calib/CsPad::CalibV1/CxiDs2.0:Cspad.0/geometry/1-end.data
and array with image from file: cspad-ndarr-ave-cxif5315-r0169.dat
Apparently, this geometry file had wrong indexes associated with quads (cable swap) that gives misaligned 2x1s in quads. This problem should be fixed in the file
|
Tuning of geometry can be done with Detector alignment tool running command:
geo -i cspad-ndarr-ave-cxif5315-r0169.dat -g geo-cxif5315-r0169-swap-tuned.data |
Thus obtained geo-cxif5315-r0169-swap-tuned.data gives image, radial, and spectral histograms for averaged array:
or see image array with maximal intensities by the command
geo -i cspad-ndarr-max-cxif5315-r0169.dat -g geo-cxif5315-r0169-swap-tuned.data |
looks like: ,
and similar for water background from run 162:
Using image array in cspad-ndarr-max-cxif5315-r0169.dat and geometry file geo-cxif5315-r0169-swap-tuned.data, one can generate masks using the Mask Editor embedded in Calibration Manager.
Run command calibman
, select tab ROI
and performing all steps of the mask creation procedure, one can draw ROI on image, create the mask for this image, convert this mask to ndarray, and use this ndarray for data processing. This ndarray can be used later for image reconstruction with the same geometry file.
For data processing of exp=cxif5315:run=169 two masks were produced for arc and equatorial regions:
Files with mask ndarray for arc and equator regions:
Peak finding algorithm is based on psana module ImgAlgos::NDArrDropletFinder with preceded CSPAD ndarray producer CSPadPixCoords::CSPadNDArrProducer and its calibration ImgAlgos::NDArrCalib. Detector geometry information is provided by the module ImgAlgos::PixCoordsProducer. Configuration file psana-cxif5315-r0169-cspad-ds2-NDArrDropletFinder.cfg (download) defines parameters for these modules. Two instances of modules are used in order to process two region of interests for Arc and Equator.
For each event psana modules are executed first and save the list of found peaks in the event store. Then, python script psana-cxif5315-r0169-cspad-ds2-NDArrDropletFinder.py (download) works with lists of peaks, performs additional selection, plots image with peaks, and saves selected peaks in the file.
On 2015-04-16:
Release ana-0.14.2
is up to date.
Until ana-current
is ana-0.14.1
, one has to use sit_setup
script with release tag parameter:
sit_setup ana-0.14.2 |
In order to run examples create directory with some arbitrary name my_analysis
and work
directory in it:
mkdir my_analysis mkdir my_analysis/work cd my_analysis sit_setup ana-0.14.2 |
Download files in my_analysis
directory:
Configuration script expects a few files in the local directory work/
; two files with masks and file with a shape of background needs to be downloaded in the my_analysis/work directory:
In the local or standard calib/
directory the files for calibration should be available for geometry, pedestals, pixel_status, common_mode
etc., whatever is going to be used in the modules.
Use command:
python psana-cxif5315-r0169-cspad-ds2-NDArrDropletFinder.py |
Image with peaks for separate and both regions can be plotted:
File with table of parameters for found peaks has an unique name like work/peaks-2015-04-14-15:57:44-cxif5315-r0169.txt
and contains records with peak parameters:
# Exp Run Event Date Time time(sec) time(nsec) fiduc Reg Seg Row Col Amax Atot Npix X(um) Y(um) cxif5315 169 3 2015-02-22 02:20:47 1424600447 503058551 104427 equ 1 145 27 165.8 2970.8 70 -224 9450 cxif5315 169 3 2015-02-22 02:20:47 1424600447 503058551 104427 equ 17 155 47 164.0 3130.3 58 506 -9674 cxif5315 169 7 2015-02-22 02:20:47 1424600447 536405573 104439 arc 8 10 20 500.9 10085.8 74 -48264 1186 cxif5315 169 7 2015-02-22 02:20:47 1424600447 536405573 104439 equ 1 161 13 483.6 3238.8 56 1318 7695 cxif5315 169 7 2015-02-22 02:20:47 1424600447 536405573 104439 equ 16 152 8 191.7 3393.9 74 -3801 -32359 cxif5315 169 7 2015-02-22 02:20:47 1424600447 536405573 104439 equ 17 30 69 295.3 2644.8 50 2903 -23418 cxif5315 169 7 2015-02-22 02:20:47 1424600447 536405573 104439 equ 17 167 48 153.0 2723.5 59 618 -8355 cxif5315 169 7 2015-02-22 02:20:47 1424600447 536405573 104439 equ 17 171 32 283.2 3447.1 59 -1139 -7913 cxif5315 169 8 2015-02-22 02:20:47 1424600447 544737897 104442 equ 17 168 47 201.7 2797.8 54 508 -8245 cxif5315 169 12 2015-02-22 02:20:47 1424600447 578091436 104454 equ 1 152 27 159.8 2374.2 58 -222 8681 cxif5315 169 14 2015-02-22 02:20:47 1424600447 594757102 104460 equ 1 153 29 160.8 3308.4 62 -442 8571 ... |
The 1st line in this file is a (commented) header with colon-titles.
ImgAlgos.NDArrDropletFinder(:Arc/:Equ)
, andpeaks_filter(...)
method.
Selection of peaks and events is defined in few stages, as listed below.
1. Peak selection parameters in module ImgAlgos.NDArrDropletFinder:
[ImgAlgos.NDArrDropletFinder:Arc] ... threshold_low = 10 threshold_high = 150 peak_radius = 3 ... |
2. Peak list selection parameters:
nda_peaks_arc = peaks_filter(nda_droplets_arc, atot_thr=2000, npix_thr=20, npeaks_min=1, npeaks_max=10) |
3. Peak/event selection parameters at peak-list file readout
def peakIsSelected() : """Apply peak selection criteria to each peak from file """ amax, atot, npix, x, y, r, phi = sp.amax, sp.atot, sp.npix, sp.x, sp.y, sp.r, sp.phi if amax<150 : return False if atot<2500 : return False if npix<30 : return False if r<434 : return False if r>444 : return False if phi<150 and phi>-150 : return False return True |
Few sets of parameters listed in table were tested in order to justify selection parameters.
Parameter | Sel.1 (rpeak5) | Sel.2 (rpeak7) | Sel.3 (rpeak3) |
---|---|---|---|
in module NDArrDropletFinder:Arc | |||
threshold_low | 10 | 10 | 10 |
threshold_high | 100 | 100 | 150 |
peak_radius | 5 | 7 | 3 |
in method peaks_filter(...) | |||
atot_thr | 2000 | 2000 | 2000 |
npix_thr | 20 | 30 | 20 |
npeaks_min | 1 | 1 | 1 |
npeaks_max | 10 | 20 | 10 |
Save selected peaks in file |
Process peaks from file and use selector method peakIsSelected():
Parameter | Sel.1 (rpeak5) | Sel.2 (rpeak7) | Sel.3 (rpeak3) |
---|---|---|---|
Process peaks from file | |||
amax | 150 | 150 | 150 |
atot | 2500 | 2500 | 2500 |
npix | 20 | 40 | 20 |
rmin | 434 | 434 | 434 |
rmax | 444 | 444 | 444 |
phi | <-150 & >150 | <-150 & >150 | <-150 & >150 |
Distribution for selection parameters are shown below. The 1st group of plots is accumulated for all peaks saved in the file with list of peaks:
atan2(y,x)
peak_radius
(peak region)threshold_low
Two other plots were accumulated after method peakIsSelected()
is applied
These distributions were obtained for "reasonable-by-eye" selection criteria. Now the question is how optimal the selection parameters are?
These plots look noisy, because it was allowed to save up to 20 peaks in the event.
Tight parameters in this case reduce a number of signal events in the Distance distribution.
npix -it is safe to require >20 or >30.
So far original Selection 1 looks optimal.
Set in psana.cfg file peak selection parameters pretty much the same as in Arc region
[ImgAlgos.NDArrDropletFinder:Whi] source = DetInfo(CxiDs2.0:Cspad.0) key = nda_clb key_droplets = nda_droplets_whi key_smeared = nda_sme_whi threshold_low = 10 threshold_high = 150 sigma = 0 smear_radius = 3 peak_radius = 3 low_value = 0. mask = work/roi_mask_nda_whi.txt masked_value = 0 windows = 1 0 185 0 388 \ 17 0 185 0 388 |
save peaks in file without additional selector.
Event 03: - normal, single hit event
Event 07: - multi-hit event
Event 37: - multi-hit event
Left plot - peak phi<0, right plot for phi>0.
In the matrix frame x-axis points down... |
Number of peaks
Peak azimuth angle
Peak radius
Average over peaks azimuth angle
Difference between maximal and minimal values of peak azimuth angle
Standard deviation of azimuth angle spread
Difference between averaged azimuth angles in two regions
Orientation angle
Peak radius equal to 3 or 5 pixel with low and high thresholds 10 and 150ADU, respectively, work fine.
phi_std<0.5
or phi_ptp<1
can be used to select good single-hit events.
[ImgAlgos.NDArrDropletFinder:Equ] source = DetInfo(CxiDs2.0:Cspad.0) key = nda_clb key_droplets = nda_droplets_equ key_smeared = nda_sme_equ threshold_low = 10 threshold_high = 150 sigma = 0 smear_radius = 3 peak_radius = 3 low_value = 0. mask = work/roi_mask_nda_equ.txt masked_value = 0 windows = 0 0 185 0 388 \ 1 0 185 0 388 \ 3 0 185 0 388 \ 8 0 185 0 388 \ 9 0 185 0 388 \ 11 0 185 0 388 \ 16 0 185 0 388 \ 17 0 185 0 388 \ 19 0 185 0 388 \ 24 0 185 0 388 \ 25 0 185 0 388 \ 27 0 185 0 388 fname_prefix = print_bits = 3 |
Where additional peak selection parameters in the python script:
if npix>42 : return False if r<100 : return False if r>460 : return False |
In psana event processing the same peak definition parameters were used for Arc and Equ regions:
[ImgAlgos.NDArrDropletFinder:Equ] ... threshold_low = 10 threshold_high = 150 peak_radius = 3 |
All peaks found in the event were saved in the peak table file without additional selection.
Arc region selection
if atot<2500 : return False if npix>42 : return False if r<434 : return False if r>444 : return False if phi<150 and phi>-150 : return False |
Equ region selection
if npix>42 : return False if r<100 : return False if r>420 : return False |
Event selection
def eventIsSelected() : sp.event_is_selected = False if sp.count_arc_pks_sel != 2 : return False if sp.count_equ_pks_sel > 5 : return False if sp.count_equ_pks_sel < 1 : return False sp.event_is_selected = True return True |
For events with exactly two peaks in Arc region orientation is evaluated as an average angle: orient = (phi1 + phi2) / 2:
Orientation distribution shows that in many ~2/3 (left and right peaks) of events two peaks are close to each other... Presumably single peak is split for two or more peaks. Further we select clean events from central peak only:
abs(orient) < 20 : |
- azimuth angle of peaks in Equ region
- orientation corrected azimuth angle of peaks in Equ region
- radial distribution of peaks in Equ region
- number of peaks in Equ region (constrain on sp.count_equ_pks_sel is not applied).
Current version of scripts with comments is collected here. It is assumed that you work on one of psana nodes which has access to data, for example:
ssh -Y psana cd <your-analysis-directory> sit_setup |
Download files in <your-analysis-directory>
Beside automatically loaded calibration files (for pedestals, common mode, masks, geometry, etc.) this configuration file uses four external files with region masks and background
These files are expected to be in the <your-analysis-directory>/work
directory.
psana-cxif5315-r0169-cspad-ds2-NDArrDropletFinder.py.txt (download) - python script, which used *.cfg file to produce all ndarrays in the event store, gets these ndarrays, does preliminary processing, saves list of peaks in the output text file, draw events (if requested).
This script can be run interactively or in batch by commands:
// interactive command: psana -c psana-cxif5315-r0169-cspad-ds2-NDArrDropletFinder.cfg exp=cxif5315:run=169 // batch job submission command: bsub -q psfehq -o log-r169.log python psana-cxif5315-r0169-cspad-ds2-NDArrDropletFinder.py |
At least 2 parameters need to be adjusted in this script for each mode:
do_plot = True - should be = False in the batch mode #events_max = 10000000 - for batch mode loop over all events events_max = 1000 - for loop over 1000 events in interactive mode |
This job produces text file with all found peaks with name like work/peaks-2015-05-09-23:52:30-cxif5315-r0169-all.txt
, which can be used for further analysis
proc-peaks-from-file.py (download) - python script for processing of the text file with list of peaks.
Input file name and output file path prefix should be set at the end of this script:
procPeaksFromFiles (('work/peaks-2015-05-09-23:52:30-cxif5315-r0169-all.txt',), '2015-05-14-figs-cxif5315/plot-cxif5315-r0169-all') |
After that this script can be executed by the command:
python proc-peaks-from-file.py |
which loops over peaks in file, does analysis, and plots requested histograms at the end.