Content
CommandsScript name and its features
det_pixel_status
This script process dark or light data and produces pixel status files. Implemented algorithms, named here as Features after Ref.2,3, are discussed in R&D with Chuck briefly documented in Ref.1.
Not all features can be applied simultaneously. Suggested combinations for -F parameters are shown in for three examples below 1) 1,2,3; 2) 1,6; 3) 11, respectively.
Get help
ana-4.0.51-py3 [dubrovin@sdfiana001:~/LCLS/con-py3]$ det_pixel_status -h
usage:
det_pixel_status works with 2-d raw arrays ONLY! For 3-d the segment index --segind needs to be specified
det_pixel_status -k <dataset> -d <detname> ...
Ex.1: det_pixel_status -k exp=xpplw3319:run=287 -d XppGon.0:Epix100a.3 -F 1,2,3 -n 1000 # dark processing for Feat.1 & 2 & 3
Ex.2: det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 1,6 -n 1000 # light processing for Feat.1 & 6
Ex.3: det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 11, -n 1000000 # light processing for Feat.11
Feature # stands for
1: mean intensity of frames in good range
2: dark mean in good range
3: dark rms in good range
4: NOT-IMPLEMENTED for gain stage indicator values
5: NOT-IMPLEMENTED for gain stage indicator values
6: light average SNR of pixels over time
11: light intensity max-peds in good range - should be processed separately from all other features on entire/large set of events.
Help: det_pixel_status -h
Evaluates pixel status of any detector/segment using raw dark/light data
optional arguments:
-h, --help show this help message and exit
-k DSKWARGS, --dskwargs DSKWARGS
(str) DataSource parameters, default = exp=xpplw3319:run=293
-d DETNAME, --detname DETNAME
(str) detector/detname name, default = epix_alc3
-n EVENTS, --events EVENTS
(int) maximal number of events total (in runs, steps), default = 1000
-m EVSKIP, --evskip EVSKIP
(int) number of events to skip in the beginning of each step, default = 0
-N STEPS, --steps STEPS
(int) maximal number of steps or all by default, default = None
-M STSKIP, --stskip STSKIP
(int) number of steps to skip before processing or process all by default, default = None
-c EVCODE, --evcode EVCODE
(str) comma separated event codes for selection as OR combination; any negative code inverts selection, default = None
-i SEGIND, --segind SEGIND
(int) segment index for multi-panel detectors with raw.shape.ndim>2, default = 0
-S SLICE, --slice SLICE
(str) FOR DEBUGGING ONLY (str) slice of the panel image 2-k array selected for plots and pixel status, ex. "0:144,0:192", default = None
-r NRECS, --nrecs NRECS
(int) number of records to collect data, default = 1000
-w SHWIND, --shwind SHWIND
(str) window shape for feature 6 fitting to plane, ex. "15,15", default = 15,15
-R SNRMAX, --snrmax SNRMAX
(int) width of the good region in terms on number of spread/rms, default = 8.0
-t CTYPE, --ctype CTYPE
(str) type of calibration constants to save, default = status_data
-L LOGMODE, --logmode LOGMODE
(str) logging mode, one of CRITICAL, FATAL, ERROR, WARN, WARNING, INFO, DEBUG, NOTSET, default = INFO
-o DIRREPO, --dirrepo DIRREPO
(str) repository for calibration results, default = /sdf/group/lcls/ds/ana/detector/calib/constants/
-F FEATURES, --features FEATURES
(str) comma-separated list of Features [JAC-2022, Sadri, Automatic bad pixel mask maker...] from 1 to 6 to evaluate bad pixels, default = 1,2,3
--dirmode DIRMODE (int) mode for all mkdir, default = 0o2775
--filemode FILEMODE (int) mode for all saved files, default = 0o664
--group GROUP (str) group ownership for all files, default = ps-users
--databits DATABITS (int) data bits in ADC for code of intensity, default = 0o37777
--gainbits GAINBITS (int) gain mode switch bits in ADC, default = 0o0
--gmode GMODE (str) gain mode name-suffix for multi-gain detectors with raw.shape.ndim>3, ex: AHL-H, default = None
Dark processing
dark data processing for Feat.1 & 2 & 3, Ref.1
det_pixel_status -k exp=xpplw3319:run=287 -d XppGon.0:Epix100a.3 -F 1,2,3 -n 1000
almost equivalent to pixel_status evaluarion in psana dark processing, but use different approach to definition of limits good value range.
It works on collected block of frames shaped as (<number-of-frames>, <2-d-frame-size>) which implies limits on consumed memory.
[I] UtilsPixelStatus.py L0389
Summary of the bad pixel status evaluation for SNR=8.00, status_dark array
0o1: 0 / 540672 ( 0.000%) pixels Feat.2 mean <= 2933.437
0o2: 33 / 540672 ( 0.006%) pixels Feat.2 mean >= 5029.965
0o4: 0 / 540672 ( 0.000%) pixels Feat.3 std <= 1.525
0o10: 5314 / 540672 ( 0.983%) pixels Feat.3 std >= 4.053
Any bad status bit: 5319 / 540672 ( 0.984%) pixels
[I] RepoManager.py L0176 panel directory: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424
[I] UtilsCalib.py L0150 saved: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424/status_dark/epix100a_0009_20210716061106_xpplw3319_r0287_status_dark.dat
[I] UtilsPixelStatus.py L0639 Consumed time 19.674 sec
[I] RepoManager.py L0255
move logfile: /sdf/group/lcls/ds/ana/detector/calib/constants/scripts/det_pixel_status/logs/2023/2023-10-09T115328_log_det_pixel_status_dubrovin.txt
to: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/logs/2023/2023-10-09T115328_log_det_pixel_status_dubrovin.txt
and create link
Light processing with fitting light background shape
light data processing for Feat.1 & 6, Ref.1
det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 1,6 -n 1000
It works on collected block of frames shaped as (<number-of-frames>, <2-d-frame-size>) which implies limits on consumed memory.
[I] UtilsPixelStatus.py L0389
Summary of the bad pixel status evaluation for SNR=8.00, status_light array
0o20: 0 / 540672 ( 0.000%) pixels Feat.6 res_med <= -881.291
0o40: 33 / 540672 ( 0.006%) pixels Feat.6 res_med >= 882.400
0o100: 0 / 540672 ( 0.000%) pixels Feat.6 res_spr <= 0.954
0o200: 4925 / 540672 ( 0.911%) pixels Feat.6 res_spr >= 2.613
Any bad status bit: 4930 / 540672 ( 0.912%) pixels
[I] RepoManager.py L0176 panel directory: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424
[I] UtilsCalib.py L0150 saved: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424/status_light/epix100a_0009_20210716073950_xpplw3319_r0293_status_light.dat
[I] UtilsPixelStatus.py L0639 Consumed time 3443.990 sec
[I] RepoManager.py L0255
move logfile: /sdf/group/lcls/ds/ana/detector/calib/constants/scripts/det_pixel_status/logs/2023/2023-10-09T101656_log_det_pixel_status_dubrovin.txt
to: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/logs/2023/2023-10-09T101656_log_det_pixel_status_dubrovin.txt
and create link
light data processing for Feat.11, Ref.1
det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 11, -n 1000000
Should be processed separately from all other features on entire/large set of events, -n 1000000
.
References
- Bad pixel mask for epix100a xpplw3319
- Paper in JAC: Automatic bad-pixel mask maker for X-ray pixel detectors with application to serial crystallography,
- Journal of Applied Crystallography - 2022 - Sadri - Automatic bad‐pixel mask maker for X‐ray pixel detectors with.pdf
- Bad Pixel Status