Page History
...
Dark data processing algorithms include pixel status evaluation which is saved in calibration files of type pixel_status. This note explains how to find definition and statistics of the bad pixel bits.
Algorithm
Dark data processing
...
Dark data processing is originaly intended for base signal level a.k.a. pedestal evaluation. In addition quality of pixel response is also evaluated. This algorithm evolved for a long time. The most advanced version implemented for calibration of jungfrau panels is described here.
- dark raw data is accumulated in array
block[nrecs1,<shape-of-data>]for a portion of events nrecs1=50. It would be nice to accumulate it for entire volume of events (nrecs=1000), but some of detectors are too big and causes problem with memory. - pre-process data from block, use median and quantile for fraclo=0.05 and fcachi=0.95 fractions of spectral events to estimate mean value and gate intensity limits, respectively.
- use gated average algorithm to process events in
blockand all other events (nrecs=1000) requested for dark processing. - at the end of the event loop evaluate mean, rms, max/min values and process a few statistical cumulative arrays which produce information about bad pixels.
Bad pixel status
...
For arrays of per-pixel mean intensity and rms, obtained in gated average algorithm, we use parameters of absolute limits and number of sigma for low and hight range
...
Due to algorithms evolution precise the bad pixel bit assignment may be different from description above. The best source of true information about bad pixel content and statistics is the log file. Another option is to look at code. Tables of this note contain references to both sources of information for different type of commands/detectors.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
Raw data for XcsEndstation.0:Epix100a.1 found/selected in 1001 events , begin data summary stage evaluate_limits: RMS ave, std = 3.411, 0.276 low, high limits = 1.500, 7.000 evaluate_limits: AVE ave, std = 4515.065, 203.890 low, high limits = 10.000, 10000.000 Bad pixel status: status 1: 134 pixel rms > 7.000 status 8: 5 pixel rms < 1.500 status 2: 6 pixel intensity > 10000 in more than 0.1 fraction of events status 4: 0 pixel intensity < 10 in more than 0.1 fraction of events status 16: 0 pixel average > 10000 status 32: 5 pixel average < 10 |
LCLS-II
Command det_raw_pixel_status
Script det_raw_pixel_status is specifically designed to search for bad pixels in dark and light data using different algorithms, named as "features 1-6" after publication [Journal of Applied Crystallography - 2022 - Sadri]. Algorithms of det_raw_pixel_status in for features 1-6 closely follow to this publication. Feature 11 is a new development to search for bad pixels with too low or too high gain factors.
Meaning and application of features in det_raw_pixel_status can be briefly listed as
| Feature # | Data type | Description | # of events | status type | Comments |
|---|---|---|---|---|---|
| 1 | dark | mean intensity of frames should be in good range | ~1000 | status_dark | |
| 2 | dark | pixel mean intensity should be in good range | ~1000 | status_dark | |
| 3 | dark | pixel rms intensity should be in good range | ~1000 | status_dark | |
| 4 | gain stage indicators between stages | for gain switching detectors only, TBD | |||
| 5 | gain stage indicators for each gain stage | for gain switching detectors only, TBD | |||
| 6 | light | average SNR of pixels over time | ~1000 | status_light | |
| 11 | light | pedestal subtracted pixel intensity max value for entire run should be in good range | entire run sample >10k | status_max | [Bad pixel mask for epix100a xpplw3319] |
where "good range" is equivalent to the "robust statistics" in publication to the bulk of data.
LCLS-II
| Detector | Dark processing command | Log file directory | Log file name | Code |
|---|---|---|---|---|
| epix10ka | epix10ka_pedestals_calibration | / | ||
| Detector | Dark processing command | Log file directory | Log file name | Code |
| epix10ka | epix10ka_pedestals_calibration | /cds/group/psdm/detector/gains2/epix10ka/panels/logs/<year>/ | <time-stamp>_log_epix10ka_pedestals_calibration_<uid>txt | UtilsEpix10kaCalib.py#L355 |
| epix100a/opal/etc | det_dark_proc | /cds/group/psdm/detector/calib2/epix100/logs/<year>/ | <time-stamp>_log_det_dark_proc_<uid>.txt | UtilsCalib.py#L281 |
...
- Calibration Scripts Repository and Logging
- calibrun - LCLS CLI for dark processing
- Dark processing for LCLS2 area detectors
- Jungfrau and Epix10ka Calibration
- save_constants_in_repository - user's helper to save constants in repository
- det_raw_pixel_status - generates status_data - new algorithms in development
- deploy_constants - deploys constants from repository to DB
- Bad pixel mask - new development
- Paper in JAC: Automatic bad-pixel mask maker for X-ray pixel detectors with application to serial crystallography,
- Journal of Applied Crystallography - 2022 - Sadri - Automatic bad‐pixel mask maker for X‐ray pixel detectors with.pdf
- Bad pixel mask for epix100a xpplw3319
Overview
Content Tools