Most of pixel array detectors produce imaging data that can not directly be used in analysis and need in corrections. Most popular corrections are
In this note we discuss common mode correction algorithms. Common mode is a hardware effect of collective pixel intensity variation due to synchronous variation of potentials on sensor chip or ADC at readout process. This effect can be corrected at low sensor illumination, where the number of pixels with energy deposition from photons is small. The spectrum of pixel intensities without photons should be grouped in the peak with small offset of the average from zero (if the dark rate correction is already applied) due to the common mode effect. This offset can be evaluated and subtracted from all pixel intensities in the group of synchronously fluctuating pixels. The number and structure of commonly behaving pixel groups depend on detector hardware.
Common mode correction is applied in the Detector interface method calib described in AreaDetector:
arr = det.calib(evt, cmpars=None)
and is controlled by the list/tuple of parameters cmpars=(<algorithm>, <mode>, <algorithm-specific parameters>,...).
Common mode correction can be turned off using cmpars=(0,0).
If this list of parameters is not specified cmpars=None, it is searched under the experimental calib directory or substituted by default.
Each algorithm may load file with parameters from calibration directory, which by default accounts for experimnet, calibration version, data source, calibration type and run range, for example
/reg/d/psdm/<INS>/<experiment>/calib/<calib-version>/<data-source>/<calibration-type>/<run-range>.data For example: /reg/d/psdm/XPP/xppi0614/calib/Epix100a::CalibV1/NoDetector.0:Epix100a.0/pedestals/0-end.data /reg/d/psdm/XPP/xppi0614/calib/Epix100a::CalibV1/NoDetector.0:Epix100a.0/common_mode/0-end.data |
Content of this file depends on detector, calibration type, and algorithm, as shown below.
We use algorithms earlier developed for CSPAD and other detectors and currently available through the Detector package.
Selection of algorithm of particular type is controlled by the list of cmpars or similar list of parameters in file for common_mode calibration type.
The 1-st parameter in the list of common mode parameters represents algorithm number. If it is set to 0 common mode correction is not applied.
Valid for: CsPad-style detectors (CsPad with 32 tiles and smaller CsPad2x2 with 2 tiles).
This algorithm is similar to one developed by Andy Salnikov and implemented in https://github.com/lcls-psana/pdscalibdata/blob/master/src/CsPadCommonModeSubV1.cpp:
Parameters for this algorithm resides in the calibration directory in the directory named common_mode or can be set with the python Detector interface.
1 25 25 100 1 |
par[6] - stride (step for jump to the next pixel)
For example:
1 50 50 100 8192 128 1 |
Not valid for: CsPad and CsPad2x2
It was developed by Philip Hart for test purposes;
Control parameters for this algorithm resides in the file for common_mode calibration type;
par[3] - length of consecutive pixel array for common mode evaluation
For example, for pnCCD one can evaluate common-mode for one came chip (128 channels).
Control file: /reg/d/psdm/amo/<exp-name>/calib/PNCCD::CalibV1/Camp.0:pnCCD.0/calib/PNCCD::CalibV1/Camp.0:pnCCD.0/common_mode/0-end.data
2 1000 1000 128 |
Not valid for: CsPad and CsPad2x2
It was developed by Philip Hart as a replacement for the mean (#2) algorithm;
Parameters are the same as in #2. The algorithm is as above, except that it calculates the median, or the average of the two median points if there are an even number passing the selection criteria.
3 100 100 128 |
Not valid for: CsPad and CsPad2x2
It is pretty similar to one developed by Matthew Weaver that is implemented in ami/event/FrameCalib
In ImgAlgos::NDArrCalib it is implemented for Epix100a and Fccd960. The algorithm is detector-dependent, executing different code depending on whether an Epix or Fccd "Source" is given to the NDArrCalib module.
The difference from algorithm #3 is quite minor; it starts to fill the intensity histogram in a quite narrow range relative to zero, but if the half of statistics is not found in this range it is extended by 1/4 of pixels (see parameter #3 below for initial guess for range of histogram). This iterations are repeated until the half of statistics is in the range, or the number of bins exceeds 10000.
Typical common_mode control parameters:
4 6 30 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 |
where
EPIX100A has an option to turn on up to 3 regions for common mode correction, controlled by the bitword parameter #2: bit#1 - common mode for 352x96-pixel 16 banks, bit#2 - common mode for 96-pixel rows in 16 banks, bit#3 - common mode for 352-pixel columns in 16 banks
FCCD960 has 2 regions, also selected by parameter #2: bit#1 - common mode correction for 1x160-pixel rows with stride 2, bit#2 - common mode correction for 480x10-pixel 96*2 supercolumns
Other parameters are not used.
Valid for CsPad-style detectors only!
In latest version of CSPAD detectors a group of pixels is not bonded to relevant electronic channels, so they see no signal. For each tile of the detector an averaged signal from these unbonded pixel-channels is used as a common mode value. It is subtracted from all other pixels.
Typical common_mode control parameters:
5 50 |
Parameters for this algorithm:
This algorithm ignores pixels above histoRange and, unlike algorithm #4, their neighbors. The statistics used to compute the common-mode values will be poorer because there will be fewer pixels used, but possible energy leakage from the pixels containing photons will not be included. The parameters are:
maxCorr - (default = 30) maximum correction applied. If common mode correction is larger than this value, no correction will be applied
colrow - (default = 3) decides what is corrected. If 1, only the columns are corrected. If 2, only the rows are corrected. And if 3, both are corrected
minFrac - (default = 0.25) the minimum fraction of pixels required to be left in a row or column after applying the mask and rejecting high pixels and their neighbors
normAll - (default = False) if true, will subtract the mean from the full image with the masked applied
Below is an example block of code showing how to call this algorithm and pass it arguments.
for nevent, evt in enumerate(ds.events()):
if nevent == 10:
break
# cmpars = [6] because this is the 6th common mode algorithm and it must be in a list format
# Add arguments on as seen fit
nda = det.calib(evt, cmpars=[6], rms=det.rms(evt), maxCorr=25) |
This algorithm evaluates common mode offset for specified by par[1] groups of pixels using method numpy.median.
rows and columns are one dimensional set of pixels limited by the shape of bank.
|
Default parameters:
7 2 10 10 0 0 0 0 0 0 0 0 0 0 0 0 |
Code:
cmpars=None # for default cmpars=(7,0,10,10) # turn off common mode correction cmpars=(7,1,10,10) # correction in rows cmpars=(7,2,10,10) # correction in columns - works the best for epix10ka cmpars=(7,4,10,10) # correction in banks cmpars=(7,3,10,10) # correction in rows and columns - works the best for jungfrau cmpars=(7,7,10,10) # correction in banks, rows and columns (in this order - banks first, then rows, then columns) Example: nda = det.calib(evt, cmpars=(7,3,10,10), mbits=1, mask=None, **kwargs) |
Accounts for pixel mask and apply median correction for different group of pixels, selected by parameter 1:
For each group of pixels common mode is evaluated using pixels defined by mask (1), then applied to all pixels in the group.
Code example:
# Call algorithm directly: from Detector.UtilsPNCCD import common_mode_pnccd common_mode_pnccd(nda, mask, cmp=(8,5,500)) # median in rows 128 and columns 512 # call through the Detector interface: pnccd.common_mode_apply(evt, nda, (8,5,500), mask=mask) # new py # OR nda = pnccdd.calib(evt, (8,5,500), mask=mask) # generic calib method with mask parameter |
To test implementation of algorithms in ImgAlgos::NDArrCalib we use the same data sets as in 2014-03-25-Ankush-CommonModeNoise.pdf
Use data from experiment amob5114
2(or 3) 1000 1000 128 |
Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #2 and 4) algorithm #3:
Images 1) for subtracted pedestals and 2) common mode correction algorithm #2:
2(or 3) 1000 1000 128 |
Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #2 and 4) algorithm #3:
Images 1) for subtracted pedestals and 2) common mode correction algorithm #2:
Common mode correction for pnCCD
Here we use cxi83714-r0136: lysozyme crystallography data with bright bragg spots on top of a varying water background. We use these common-mode parameters (turning off the fall-back unbonded correction):
1 10 10 100 |
Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #1:
Images 1) for subtracted pedestals and 2) common mode correction algorithm #1:
This is for a dark run, and was generated using this script:
import psana
from matplotlib import pyplot as plt
import numpy as np
dataset_name = "exp=cxil5316:run=1"
ds = psana.DataSource(dataset_name)
psana_det = psana.Detector("DscCsPad")
evt = ds.events().next()
pedestal = psana_det.pedestals(evt)
data = psana_det.raw_data(evt) - pedestal
data_cm = psana_det.raw_data(evt) - pedestal
unbond_cm = psana_det.common_mode_correction(evt, data_cm, [5, 50])
default_cm = psana_det.common_mode_correction(evt, data_cm, [1, 50, 10, 100])
unbond=unbond_cm[:,0,0]
default=default_cm[:,0,0]
plt.subplot(2,1,1)
plt.title('Default vs. Unbond Correction (ADU)')
plt.plot(unbond,default,'o')
plt.subplot(2,1,2)
plt.title('Default-Unbond Correction (ADU)')
plt.hist(default-unbond)
plt.show() |
This yields the following plots comparing the size of the two corrections:
This is the same plot for an event with many photons. This is for exp=cxid9114:run=96, and the event has unix-timestamp 6025277111415285948. There is one entry in the plot for the common-mode value for each tile of the CSPAD.
The common-mode methods still correlate, but the default method perhaps over-subtracts because of leakage into the zero-photon peak from neighbor-pixels of real photons.
Here are the common mode corrections applied to a crystal diffraction pattern with a strong water ring using algorithms 1 vs 5. Algorithm 5 seems to perform better in this case.
Common mode histogram using algorithm 1
Common mode histogram using algorithm 5
Use meca1113-r0045 with
1 50 10 (100) - last parameter is set by default |
Spectra for 1) raw data, 2) subtracted pedestals, 3) subtracted common mode correction algorithm #1:
Images 1) raw data and 2) subtracted pedestals with common mode correction algorithm #1:
Common mode correction for CSPAD2x2 in this example shows minor improvement.