Content
Common mode correction
Most of pixel array detectors produce imaging data that can not directly be used in analysis and need in corrections. Most popular corrections are
- dark rate (pedestal) subtraction,
- bad pixel masking,
- common mode correction,
- gain correction,
- etc.
In this note we discuss common mode correction algorithms. Common mode is a hardware effect of collective pixel intensity variation due to synchronous variation of potentials on sensor chip or ADC at readout process. This effect can be corrected at low sensor illumination, where the number of pixels with energy deposition from photons is small. The spectrum of pixel intensities without photons should be grouped in the peak with small offset of the average from zero (if the dark rate correction is already applied) due to the common mode effect. This offset can be evaluated and subtracted from all pixel intensities in the group of synchronously fluctuating pixels. The number and structure of commonly behaving pixel groups depend on detector hardware.
Implementation
Below we describe common mode correction algorithms used in
- psana module
ImgAlgos::NDArrCalib
. This correction along with others can be applied to raw ndarray under control of the psana configuration file for specified detector, for example for Epix100a:
[ImgAlgos.NDArrCalib] source = DetInfo(:Epix100a) key_in = ndarray-raw key_out = ndarray-clb do_peds = yes do_cmod = yes do_stat = yes ...
where keywords do_peds
and do_cmod
should have value yes
or true
in order to turn on pedestal subtraction and common mode correction algorithms.
Detector.PyDetector
package which provides uniform access to calibration and data for all detectors.
Low level implementation of the common mode correction algorithms is done in class ImgAlgos::CommonModeCorrection
which uses methods from ImgAlgos::CommonMode
.
Each algorithm may load file with parameters from calibration directory, which by default accounts for experimnet, calibration version, data source, calibration type and run range:
/reg/d/psdm/<INS>/<experiment>/calib/<calib-version>/<data-source>/<calibration-type>/<run-range>.data For example: /reg/d/psdm/XPP/xppi0614/calib/Epix100a::CalibV1/NoDetector.0:Epix100a.0/pedestals/0-end.data /reg/d/psdm/XPP/xppi0614/calib/Epix100a::CalibV1/NoDetector.0:Epix100a.0/common_mode/0-end.data
Content of this file depends on detector, calibration type, and algorithm, as shown below.
Algorithms
We use algorithms earlier developed for CSPAD and other detectors and currently residing in ImgAlgos
and psalg
packages.
Selection of algorithm of particular type is controlled by the parameters in file for common_mode
calibration type.
#1 - common mode peak finding algorithm
Valid for: CsPad and CsPad2x2
This algorithm is similar to one developed by Andy Salnikov and implemented in cspad_mod.CsPadCalib;
- for each cspad2x1 sensor the pixels' intensity histogram is filled for natural ADU bins, pixels with bad status are ignored,
- search for the peak in the histogram corresponding to the dark pixels using thresholds,
- iterate over bins around the peak region and improve the peak location precision, that defines the common mode correction
- if correction is in allowed range then apply it to all sensor pixels.
Control parameters for this algorithm resides in the file for common_mode
calibration type;
parameters for CSPAD and CSPAD2x2
1 50 50 100
- par[0] - algorithm #
- par[1] - maximal deviation of the peak mean from 0
- par[2] - maximal allowed value of the peak RMS
- par[3] -threshold on number of pixels in the ADU bin in the peak finding algorithm
parameters for other detectors
- par[4] - number of segments for common mode evaluation
- par[5] - segment size (number of pixels for common mode evaluation)
par[6] - stride (step for jump to the next pixel)
For example:
1 50 50 100 8192 128 1
#2 - MEAN algorithm
Not valid for: CsPad and CsPad2x2
It was developed by Philip Hart for test purpose;
- for each group of pixels intensity histogram is filled for natural ADU bins, pixels with bad status are ignored,
- a simple mean below threshold is considered as a common mode correction,
- if correction is in the allowed range, it is applied to all pixels from the group.
Control parameters for this algorithm resides in the file for common_mode
calibration type;
- par[0] - algorithm #
- par[1] - maximal threshold on intensity to evaluate mean for low intensities
- par[2] - maximal allowed common mode correction
par[3] - length of consecutive pixel array for common mode evaluation
For example, for pnCCD one can evaluate common-mode for one came chip (128 channels).Control file: /reg/d/psdm/amo/<exp-name>/calib/PNCCD::CalibV1/Camp.0:pnCCD.0/calib/PNCCD::CalibV1/Camp.0:pnCCD.0/common_mode/0-end.data
2 1000 1000 128
- The algorithm loops over the data and evaluates consecutive arrays of specified length (which might represent for example a row of pixels in a readout chip) and finds the mean value for values below a threshold, ignoring masked pixels. It corrects all the data if the calculated common mode is less than the maximal allowed correction. The median algorithm is more stable in many cases and is recommended.
#3 - MEDIAN algorithm
Not valid for: CsPad and CsPad2x2
It was developed by Philip Hart as a replacement for the mean (#2) algorithm;
- for each group of pixels intensity histogram is filled for natural ADU bins, pixels with bad status are ignored,
- a half of statistics counted from bin with low intensities below threshold is considered as a common mode correction,
- if correction is in the allowed range, it is applied to all pixels from the group.
Parameters are the same as in #2. The algorithm is as above, except that it calculates the median, or the average of the two median points if there are an even number passing the selection criteria.
3 100 100 128
#4 - MEDIAN algorithm - detector dependent
Not valid for: CsPad and CsPad2x2
It is pretty similar to one developed by Matthew Weaver that is implemented in ami/event/FrameCalib
In ImgAlgos::NDArrCalib
it is implemented for Epix100a and Fccd960. The algorithm is detector-dependent, executing different code depending on whether an Epix or Fccd "Source" is given to the NDArrCalib module.
The difference from algorithm #3 is quite minor; it starts to fill the intensity histogram in a quite narrow range relative to zero, but if the half of statistics is not found in this range it is extended by 1/4 of pixels (see parameter #3 below for initial guess for range of histogram). This iterations are repeated until the half of statistics is in the range, or the number of bins exceeds 10000.
Typical common_mode
control parameters:
4 1 30 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
where
- par[0] (4) - algorithm number; 4 stands for this median algorithm,
- par[1] (1) - type of regions for median algorithm; 3 regions (banks/rows/columns) for Epix100a, and 2 regions (banks/rows) for Fccd960:
EPIX100A has an option to turn on up to 3 regions for common mode correction, controlled by the bitword parameter #2: bit#1 - common mode for 352x96-pixel 16 banks, bit#2 - common mode for 96-pixel rows in 16 banks, bit#3 - common mode for 352-pixel columns in 16 banks
FCCD960 has 2 regions, also selected by parameter #2: bit#1 - common mode correction for 1x160-pixel rows with stride 2, bit#2 - common mode correction for 480x10-pixel 96*2 supercolumns
- par[2] (30) - initial guess for range of histogram (relative to 0) in ADU (see above description of intensity histogram). This number can grow in subsequent iterations if half of the statistics is not found in this range. It should probably start at around 3 sigma of the noise value.
- par[3] (10) - maximal allowed absolute value of the common mode correction in ADU. If = 0 - no-limit is applied directly, indirect limit = 10000ADU (from limit on intensity histogram described above).
Other parameters are not used.
#5 - Unbond pixels common mode correction
Only valid for: CsPad and CsPad2x2
In latest version of CSPAD (including 2x2) detectors a group of pixels is not bound to relevant electronic channels. Signal from these channels can be used to evaluate common mode variation of the sensor. Typical common_mode
control parameters:
5 50
where
- par[0] - algorithm #,
- par[1] - maximal allowed correction value.
Test of the common mode correction for pnCCD
To test implementation of algorithms in ImgAlgos::NDArrCalib
we use the same data sets as in 2014-03-25-Ankush-CommonModeNoise.pdf
Use data from experiment amob5114
High gain pnCCD run 121
2(or 3) 1000 1000 128
Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #2 and 4) algorithm #3:
Images 1) for subtracted pedestals and 2) common mode correction algorithm #2:
High gain pnCCD run 329
2(or 3) 1000 1000 128
Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #2 and 4) algorithm #3:
Images 1) for subtracted pedestals and 2) common mode correction algorithm #2:
Summary for pnCCD
Common mode correction for pnCCD
- gives significant effect in low gain mode and is negligible in high gain mode
- algorithm #2 gives the best results, #3 a little bit worse, #1 - does not work for pnCCD
Test of common mode correction for CSPAD
Use cxi83714-r0136 with
1 10 10 100
Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #1:
Images 1) for subtracted pedestals and 2) common mode correction algorithm #1:
Summary for CSPAD
Common mode correction for CSPAD works with algorith #1 and shows significant effect.
Test of common mode correction for CSPAD2x2
Use meca1113-r0045 with
1 50 10 (100) - last parameter is set by default
Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #1:
Images 1) raw data and 2) subtracted pedestals with common mode correction algorithm #1:
Summary for CSPAD2x2
Common mode correction for CSPAD2x2 in this example shows minor improvement.
References
2014-03-25-Ankush-CommonModeNoise.pdf - stand-alone test of common mode correction for pnCCD
psana - Module Catalog - Module ImgAlgos::NDArrCalib
Fccd960-Detector