...

In this note we discuss common mode correction algorithms. Common mode is a hardware effect of collective pixel intensity variation due to synchronous variation of potentials on sensor chip or ADC at readout process. This effect can be corrected at low sensor illumination, where the number of pixels with energy deposition from photons is small. The spectrum of pixel intensities without photons should be grouped in the peak with small offset of the average from zero (if the dark rate correction is already applied) due to the common mode effect. This offset can be evaluated and subtracted from all pixel intensities in the group of synchronously fluctuating pixels. The number and structure of commonly behaving pixel groups depend on detector hardware.

Implementation

Below we describe common Common mode correction algorithms used in

• psana module ImgAlgos::NDArrCalib. This correction along with others can be applied to raw ndarray under control of the psana configuration file for specified detector, for example for  Epix100a:

[ImgAlgos.NDArrCalib]
source  = DetInfo(:Epix100a)
key_in  = ndarray-raw
key_out = ndarray-clb
do_peds = yes
do_cmod = yes
do_stat = yes
...

where keywords do_peds and do_cmod should have value yes or true in order to turn on pedestal subtraction and common mode correction algorithms.

• Detector.PyDetector package which provides uniform access to calibration and data for all detectors.

Low level implementation of the common mode correction algorithms is done in class ImgAlgos::CommonModeCorrection which uses methods from ImgAlgos::CommonMode.

Each algorithm may load file with parameters from calibration directory, which by default accounts for experimnet, calibration version, data source, calibration type and run range:

is applied in the Detector interface method calib described in AreaDetector:<PATH>

arr = det.calib(evt, cmpars=None)

and is controlled by the list/tuple of parameters cmpars=(<algorithm>, <mode>, <algorithm-specific parameters>,...).

Common mode correction can be turned off using cmpars=(0,0).

If this list of parameters is not specified cmpars=None, it is searched under the experimental calib directory or substituted by default.

Each algorithm may load file with parameters from calibration directory, which by default accounts for experimnet, calibration version, data source, calibration type and run range, for example

<PATH>/<INS>

/reg/d/psdm/<INS>/<experiment>/calib/<calib-version>/<data-source>/<calibration-type>/<run-range>.data

For example:
/reg/d/psdm 
<PATH>/XPP/xppi0614/calib/Epix100a::CalibV1/NoDetector.0:Epix100a.0/pedestals/0-end.data
<PATH>/reg/d/psdm/XPP/xppi0614/calib/Epix100a::CalibV1/NoDetector.0:Epix100a.0/common_mode/0-end.data

Content of this file depends on detector, calibration type, and algorithm, as shown below.

Algorithms

We use algorithms earlier developed for CSPAD and other detectors and currently residing in ImgAlgos and  psalg packagesavailable through the Detector package.

Selection of algorithm of particular type is controlled by the list of cmpars or similar list of parameters in file for common_mode calibration type.

...

#0 - common mode

...

correction is turned off

The 1-st parameter in the list of common mode parameters represents algorithm number. If it is set to 0 common mode correction is not applied.

#1 - common mode peak finding algorithm

Valid for: CsPad-style detectors (CsPad with 32 tiles and smaller CsPad2x2 with 2 tiles).

This algorithm is

Valid for: CsPad and CsPad2x2

This algorithm is similar to one developed by Andy Salnikov and implemented in cspad_mod.CsPadCalib;in https://github.com/lcls-psana/pdscalibdata/blob/master/src/CsPadCommonModeSubV1.cpp:

1. for each cspad tile an for each cspad2x1 sensor the pixels' intensity histogram is filled for natural ADU bins, with pedestal-subtracted ADU values (1 bin for each possible ADU value). pixels with bad status are ignored,
2. search for the peak in the histogram corresponding to the dark pixels using thresholds,
3. iterate over bins around the peak region and improve the peak location precision, that defines the common mode correction
4. if correction is in allowed range then apply it to all sensor pixels.

Control parameters for this algorithm resides in the file for common_mode calibration type;

...

2. software finds where histogram crosses threshold (parameter 3)
3. measure mean/rms of region that is above the threshold
4. if mean/rms is in allowed range (parameters 1 and 2) then common mode value is set to the mean.
5. if the correction is not in the allowed range, and par[4]>0.1 (see below) then the common mode value is computed from unbonded pixels (see algorithm #5)
6. subtract the common mode value from all pixels in the cspad tile

Parameters for this algorithm resides in the calibration directory in the directory named common_mode or can be set with the python Detector interface.

Default parameters for CSPAD and CSPAD2x2

1 5025 5025 100 1

• par[0] - algorithm #
• par[1] - maximal deviation of the peak mean from 0
• par[2] - maximal allowed value of the peak RMS
• par[3] -threshold on number of pixels in the ADU bin in the peak finding algorithm
• par[4] - if set to 1 then use unbonded pixel common-mode values (algorithm 5) for those panels where the standard correction failed.  do not do this if set to 0

parameters for other detectors

...

It was developed by Philip Hart for test purposepurposes;

1. for each group of pixels intensity histogram is filled for natural ADU bins, pixels with bad status are ignored,
2. a simple mean below threshold is considered as a common mode correction,
3. if correction is in the allowed range, it is applied to all pixels from the group.

...

• par[0] - algorithm #
• par[1] - maximal threshold on intensity to evaluate mean for low intensities
• par[2] - maximal allowed common mode correction

• par[3] - length of consecutive pixel array for common mode evaluation
  For example, for pnCCD one can evaluate common-mode for one came chip (128 channels).

  Control file:  

<PATH>/reg/d/psdm/amo/<exp-name>/calib/PNCCD::CalibV1/Camp.0:pnCCD.0/calib/PNCCD::CalibV1/Camp.0:pnCCD.0/common_mode/0-end.data

2 1000 1000 128

• The algorithm loops over the data and evaluates consecutive arrays of specified length (which might represent for example a row of pixels in a readout chip) and finds the mean value for values below a threshold, ignoring masked pixels.  It corrects all the data if the calculated common mode is less than the maximal allowed correction.  The median algorithm is more stable in many cases and is recommended.

...

#5 - Unbond pixels common mode correction

Only valid Valid for : CsPad and CsPad2x2CsPad-style detectors only!

In latest version of CSPAD (including 2x2) detectors a group of pixels is not bound bonded to relevant electronic channels, so they see no signal. For each tile of the detector an averaged signal from these unbonded pixel-channels is used as a common mode value. It is subtracted from all other pixels.

. Signal from these channels can be used to evaluate common mode variation of the sensor. Typical common_mode  control parameters:

5 50

where

• par[0] - algorithm #,
• par[1] - maximal allowed correction value.

 50

Parameters for this algorithm:

• par[0] - algorithm #
• par[1] - maximal allowed common-mode correction value (i.e. don't apply common-mode if computed value exceeds this number).  In other common-mode algorithms this parameter is used to make sure that the common-mode has not been computed using pixels with signal in them.  For this algorithm (that uses unbonded pixels) this parameter is less necessary, since there should never be signal in these pixels, unless there are electronic effects.  One example of such electronic effects: experiments in the MEC hutch often see an electromagnetic-pulse from their high-powered lasers that causes unusual things to happen in the CSPAD readout.  But this is rare in non-MEC experiments.

#6 - Epix neighbor exclusion

This algorithm ignores pixels above histoRange and, unlike algorithm #4, their neighbors. The statistics used to compute the common-mode values will be poorer because there will be fewer pixels used, but possible energy leakage from the pixels containing photons will not be included. The parameters are:

1. img - image on which the common mode is applied
2. rms - array of noise values for each pixel with same shape as img

3. maxCorr - (default = 30) maximum correction applied. If common mode correction is larger than this value, no correction will be applied

4. histoRange - (default = 30) all pixels above this parameter are masked

5. colrow - (default = 3) decides what is corrected. If 1, only the columns are corrected. If 2, only the rows are corrected. And if 3, both are corrected

6. minFrac - (default = 0.25) the minimum fraction of pixels required to be left in a row or column after applying the mask and rejecting high pixels and their neighbors

7. normAll - (default = False) if true, will subtract the mean from the full image with the masked applied

Below is an example block of code showing how to call this algorithm and pass it arguments.

for nevent, evt in enumerate(ds.events()):
	if nevent == 10:
		break
 
	# cmpars = [6] because this is the 6th common mode algorithm and it must be in a list format
	# Add arguments on as seen fit
    nda = det.calib(evt, cmpars=[6], rms=det.rms(evt), maxCorr=25)

#7 - MEDIAN algorithm for jungfrau and epix10ka

This algorithm evaluates common mode offset for specified by par[1] groups of pixels using method numpy.median.

• par[0] - algorithm # 7
• par[1] - mode 0/ +1/ +2/ +4 - turned off /  in rows /  in columns / in banks. Each correction consumes extra time.
• par[2] - maximal allowed common mode correction in det.raw ADU to protect from over-correction at high illumination - needs to be adjusted by users per experiment
• par[3] - minimal number of good pixels in mask and gain mode to evaluate median offset (optional, default =10)

• common mode correction is applied to pixels in high gain mode only for jungfrau and H or M modes (fixed or auto-switching) only for epix10ka.
• common mode is evaluated for good pixels only (marked by 1 in mask).
• mask is combined of 2 sources (if available)
  • det.calib(..., mbits=1,...) which is used in det.mask_comb(..., mbits=1,...). Parameter mbits controls mask of many sources, including pixel_status, borders, unbonded pixels (cspad only), and user-defined pixel_mask.
  • det.calib(..., mask=None,...) user specified mask shaped as det.raw

Default parameters:

7 2 10 10 0 0 0 0 0 0 0 0 0 0 0 0 

Code:

cmpars=None # for default
cmpars=(7,0,10,10) # turn off common mode correction
cmpars=(7,1,10,10) # correction in rows
cmpars=(7,2,10,10) # correction in columns - works the best for epix10ka
cmpars=(7,4,10,10) # correction in banks
cmpars=(7,3,10,10) # correction in rows and columns - works the best for jungfrau
cmpars=(7,7,10,10) # correction in banks, rows and columns (in this order - banks first, then rows, then columns)

Example:
nda = det.calib(evt, cmpars=(7,3,10,10), mbits=1, mask=None, **kwargs)

#8 - MEDIAN algorithm for pnCCD

Accounts for pixel mask and apply median correction for different group of pixels, selected by parameter 1:

• par[0] - algorithm # 8
• par[1] - mode of median algorithm for pixel groups: 0 - turned off | +1 - rows in banks (128 pix) | +2 - for rows (512 pix) |  +4 columns (512 pix) |  +8 banks (512x128 pix)
• par[2] - maximal allowed common mode correction

For each group of pixels common mode is evaluated using pixels defined by mask (1), then applied to all pixels in the group.

Code example:

# Call algorithm directly:
from Detector.UtilsPNCCD import common_mode_pnccd
common_mode_pnccd(nda, mask, cmp=(8,5,500))   # median in rows 128 and columns 512

# call through the Detector interface:
pnccd.common_mode_apply(evt, nda, (8,5,500), mask=mask) # new py
# OR
nda = pnccdd.calib(evt, (8,5,500), mask=mask) # generic calib method with mask parameter 

Test of the common mode correction for pnCCD

...

Images 1) for subtracted pedestals and 2) common mode correction algorithm #2:

High gain pnCCD run 329

2(or 3) 1000 1000 128

...

• gives significant effect in low gain mode and is negligible in high gain mode
• algorithm #2 gives the best results, #3 a little bit worse, #1 - does not work for pnCCD

Test of common mode correction for CSPAD

Use cxi83714-r0136 with

Test of common mode correction for CSPAD

Here we use cxi83714-r0136: lysozyme crystallography data with bright bragg spots on top of a varying water background.  We use these common-mode parameters (turning off the fall-back unbonded correction):

1 10 10 100

Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #1:

...

This is the same plot for an event with many photons.

...

 This is for exp=cxid9114:run=96, and the event has unix-timestamp 6025277111415285948.  There is one entry in the plot for the common-mode value for each tile of the CSPAD.

The common-mode methods still correlate, but the default method perhaps over-subtracts because of leakage into the zero-photon peak from neighbor-pixels of real photons.

...

Common mode histogram using algorithm 5

Thoughts from Phil on complexities involved with CsPad Common Mode:

• statistics of unbonded pixels is a problem
• poisson statistics on bragg spots is bigger than the common mode
• don't correct common-mode for low-gain pixels
• the larger shifts seen in bright events are "cross-talk" caused by the "ramp" where pixels nearer the mean-pedestal have a lower gain
• should do per-asic common-mode
• should do algorithm 5, then algorithm 1 if it fails

Summary for CSPAD

Common mode correction for CSPAD works with algorithm #1 and shows significant effect.

• have a lower gain
• should do per-asic common-mode

Test of common mode correction for CSPAD2x2

...

Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #1:

Images 1) raw data and 2) subtracted pedestals with common mode correction algorithm #1:

...

Common mode correction for CSPAD2x2 in this example shows minor improvement.

References

• AreaDetector - Detector Interface is described in the head of this file.

...

• 2014-03-25-Ankush-CommonModeNoise.pdf - stand-alone test of common mode correction for pnCCD
• psana - Module Catalog - Module ImgAlgos::NDArrCalib
• Epix100a
• Fccd960-Detector

• numpy.median
• <PATH>

...