This note is about n-d array processing algorithms implemented in ImgAlgos.PyAlgos. Algorithms can be called from python but low level implementation is done on C++ with boost/python wrapper. All examples are shown for python level interface.

Content

Table of Contents

Common

...

features of algorithms

n-d arrays

LCLS detector data come from DAQ as n-d arrays (ndarray in C++ or numpy.array in Python). In simple case camera data is an image presented by the 2-d array. For composite detectors like CSPAD, CSPAD2X2, EPIX, PNCCD, etc. data comes from a set of sensors as 3-d or 4-d arrays. If relative sensors' positions are known, then sensors can be composed in 2-d image. But this image contains significant portion of "fake" empty pixels, that may be up to ~20-25% in case of CSPAD. Most efficient data processing algorithms should be able to work with n-d arrays.

...

Code Block

import psana
from ImgAlgos.PyAlgos import PyAlgos 

# create object:
alg = PyAlgos(windows=winds, mask=mask, pbits=0)

# set peak-selector parameters:
alg.set_peak_selection_pars(npix_min=5, npix_max=5000, amax_thr=0, atot_thr=0, son_min=10)

Define ROI using windows and/or mask

...

Code Block

# List of windows
winds = None # entire size of all segments will be used for peak finding
winds = (( 0, 0, 185, 0, 388),              
         ( 1, 20,160, 30,300),
         ( 7, 0, 185, 0, 388))

# Mask
mask = None                   # (default) all pixels in windows will be used for peak finding
mask = det.mask()             # see class Detector.PyDetector
mask = np.loadtxt(fname_mask) # 
mask.shape = <should be the same as shape of data n-d array>

Hit finders

Hit finders return simple values for decision on event selection. Two algorithms are implemented in ImgAlgos.PyAlgos. They count number of pixels and intensity above threshold in the Region Of Interest (ROI) defined by windows and mask parameters in object constructor.

Both hit-finders receive input n-d drray data and threshold thr parameters and return a single value in accordance with method name.

Number of pixels above thresholdNumber of pixels above threshold

number_of_pix_above_thr

Code Block
npix = alg.number_of_pix_above_thr(data, thr=10)

Total intensity above threshold

intensity_of_pix_above_thr

Code Block
intensity = alg.intensity_of_pix_above_thr(data, thr=12)

Peak finders

_thr

Code Block
intensity = alg.intensity_of_pix_above_thr(data, thr=12)

Peak finders

Peak selection parameters

Internal peak selection can be done at the end of each peak finder, but all peak selection parameters need to be defined right after algorithm object is created, for example, as follows:

Code Block
# create object: alg = PyAlgos(windows=winds, mask=mask) # set peak-selector parameters: alg.set_peak_selection_pars(npix_min=5, npix_max=5000, amax_thr=0, atot_thr=0, son_min=10)

All peak finders have a few algorithm-dependent parameters

nda - calibrated n-d array of data, pedestals and background should be subtracted, common mode - corrected
thr_* - different type of thresholds
rank - peak rank as explained in section .
r0, dr - ring internal radius and width to evaluate background and noise rms as explained in section

...

Code Block
peaks = alg.peak_finder_v1(nda, thr_low=10, thr_high=150, radius=5, dr=0.05)

Flood filling algorithm

define peaks for regions of connected pixels above threshold

...

Code Block
peaks = alg.peak_finder_v2(nda, thr=10, r0=5, dr=0.05)

...

Local maximums search algorithm

...

Code Block
peaks = alg.peak_finder_v3(nda, rank=2, r0=5, dr=0.05)

...

makes a map of pixels with local maximums of requested rank for data ndarray and mask, pixel code in the map may have bits 0/1/2/4 standing for not-a-maximum / maximum-in-row / maximum-in-column / maximum-in-rectangular-region of radius=rank.
for each pixel with local maximal intensity in the region defined by the rank radius counts a number of pixels with intensity above zero, total positive intensity, center of gravity coordinates and rms,
using parameters r0(ex.=5.0), dr(ex.=0.05) evaluates background level, rms of noise, and S/N for the pixel with maximal intensity.
makes list of peaks which pass selector with parameters set in set_peak_selection_pars, for example

...

rank	2-d region	fraction	time, ms
1	3x3	0.1062	5.4
2	5x5	0.0372	5.2
3	7x7	0.0179	5.1
4	9x9	0.0104	5.2
5	11x11	0.0066	5.2

Anchor
bkg_rms
bkg_rms

Evaluation of the background level, rms, and S/N ratio

...

Code Block

r0=3  dr=0.1 (4 pixels)   r0=3  dr=0.5 (12 pixels)   r0=3  dr=1 (24 pixels) 
0 0 0 0 0 0 0 0 0         0 0 0 0 0 0 0 0 0          0 0 0 0 1 0 0 0 0       
0 0 0 0 1 0 0 0 0         0 0 0 1 1 1 0 0 0          0 0 1 1 1 1 1 0 0       
0 0 0 0 0 0 0 0 0         0 0 0 0 0 0 0 0 0          0 1 0 0 0 0 0 1 0       
0 0 0 0 0 0 0 0 0         0 1 0 0 0 0 0 1 0          0 1 0 0 0 0 0 1 0       
0 1 0 0 + 0 0 1 0         0 1 0 0 + 0 0 1 0          1 1 0 0 + 0 0 1 1       
0 0 0 0 0 0 0 0 0         0 1 0 0 0 0 0 1 0          0 1 0 0 0 0 0 1 0       
0 0 0 0 0 0 0 0 0         0 0 0 0 0 0 0 0 0          0 1 0 0 0 0 0 1 0       
0 0 0 0 1 0 0 0 0         0 0 0 1 1 1 0 0 0          0 0 1 1 1 1 1 0 0       
0 0 0 0 0 0 0 0 0         0 0 0 0 0 0 0 0 0          0 0 0 0 1 0 0 0 0

r0=4  dr=0.2 (12 pixels)   r0=4  dr=0.3 (16 pixels)    r0=4  dr=0.5 (24 pixels)
0 0 0 0 0 0 0 0 0 0 0      0 0 0 0 0 0 0 0 0 0 0       0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 1 1 0 0 0 0      0 0 0 0 1 1 1 0 0 0 0       0 0 0 1 1 1 1 1 0 0 0
0 0 0 0 0 0 0 0 0 0 0      0 0 1 0 0 0 0 0 1 0 0       0 0 1 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 0 0 0      0 0 0 0 0 0 0 0 0 0 0       0 1 0 0 0 0 0 0 0 1 0
0 1 0 0 0 0 0 0 0 1 0      0 1 0 0 0 0 0 0 0 1 0       0 1 0 0 0 0 0 0 0 1 0
0 1 0 0 0 + 0 0 0 1 0      0 1 0 0 0 + 0 0 0 1 0       0 1 0 0 0 + 0 0 0 1 0
0 1 0 0 0 0 0 0 0 1 0      0 1 0 0 0 0 0 0 0 1 0       0 1 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0 0      0 0 0 0 0 0 0 0 0 0 0       0 1 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0 0      0 0 1 0 0 0 0 0 1 0 0       0 0 1 0 0 0 0 0 1 0 0
0 0 0 0 1 1 1 0 0 0 0      0 0 0 0 1 1 1 0 0 0 0       0 0 0 1 1 1 1 1 0 0 0
0 0 0 0 0 0 0 0 0 0 0      0 0 0 0 0 0 0 0 0 0 0       0 0 0 0 0 0 0 0 0 0 0

Matrices of pixels for r0=5 and 6 and different dr values

Code Block

r0=5  dr=0.05 (12 pixels)      r0=5  dr=0.5  (28 pixels)            
0 0 0 0 0 0 0 0 0 0 0 0 0      0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 0 0 0 0 0 0      0 0 0 0 1 1 1 1 1 0 0 0 0
0 0 0 1 0 0 0 0 0 1 0 0 0      0 0 0 1 0 0 0 0 0 1 0 0 0
0 0 1 0 0 0 0 0 0 0 1 0 0      0 0 1 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0      0 1 0 0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0 0 0 0      0 1 0 0 0 0 0 0 0 0 0 1 0
0 1 0 0 0 0 + 0 0 0 0 1 0      0 1 0 0 0 0 + 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0 0 0 0      0 1 0 0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0 0 0 0      0 1 0 0 0 0 0 0 0 0 0 1 0
0 0 1 0 0 0 0 0 0 0 1 0 0      0 0 1 0 0 0 0 0 0 0 1 0 0
0 0 0 1 0 0 0 0 0 1 0 0 0      0 0 0 1 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 1 0 0 0 0 0 0      0 0 0 0 1 1 1 1 1 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0      0 0 0 0 0 0 0 0 0 0 0 0 0

r0=6  dr=0.2 (12 pixels)        r0=6  dr=0.5 (28 pixels)      
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 1 1 0 0 0 0 0 0     0 0 0 0 0 1 1 1 1 1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 0 0 1 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 0 1 0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 1 0 0 0 0 0 0 0 0 0 0 0 1 0
0 1 0 0 0 0 0 0 0 0 0 0 0 1 0     0 1 0 0 0 0 0 0 0 0 0 0 0 1 0
0 1 0 0 0 0 0 + 0 0 0 0 0 1 0     0 1 0 0 0 0 0 + 0 0 0 0 0 1 0
0 1 0 0 0 0 0 0 0 0 0 0 0 1 0     0 1 0 0 0 0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 1 0 0 0 0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 0 1 0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 0 0 1 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 1 1 1 0 0 0 0 0 0     0 0 0 0 0 1 1 1 1 1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

...

References

ImgAlgos.PyAlgos - code example in Sphinx documentation
Peak Finding - short announcement about peak finders
Hit and Peak Finders - examples in Chris' tutorial
Peak Finding Module - (depricated) psana module, it demonstaration examples and results
Psana Module Catalog - (depricated) peak finding psana modules
Psana Module Examples - (depricated) peak finding examples in psana modules
GUI for tuning peak finding - Chun's page in development

...

Versions Compared

Old Version 13

New Version 14

Key

Content

Common

features of algorithms

n-d arrays

Define ROI using windows and/or mask

Hit finders

Number of pixels above thresholdNumber of pixels above threshold

number_of_pix_above_thr

Total intensity above threshold

intensity_of_pix_above_thr

Peak finders

_thr

Peak finders

Peak selection parameters

Flood filling algorithm

Local maximums search algorithm

Anchor
bkg_rms
bkg_rms

Evaluation of the background level, rms, and S/N ratio

Matrices of pixels for r0=5 and 6 and different dr values

References

Page History

Versions Compared

Old Version 13

New Version 14

Key

Content

Common

features of algorithms

n-d arrays

Define ROI using windows and/or mask

Hit finders

Number of pixels above thresholdNumber of pixels above threshold

number_of_pix_above_thr

Total intensity above threshold

intensity_of_pix_above_thr

Peak finders

_thr

Peak finders

Peak selection parameters

Flood filling algorithm

Local maximums search algorithm

Anchorbkg_rmsbkg_rms

Evaluation of the background level, rms, and S/N ratio

Matrices of pixels for r0=5 and 6 and different dr values

References

Anchor
bkg_rms
bkg_rms