Content

CommandsScript name and its features

det_pixel_status

This script process dark or light data and produces pixel status files. Implemented algorithms, named here as Features after Ref.2,3, are discussed in R&D with Chuck briefly documented in Ref.1.

Not all features can be applied simultaneously. Suggested combinations for -F parameters are shown in for three examples below 1) 1,2,3; 2) 1,6; 3) 11, respectively.

Get help

det_pixel_status -h # on 2023-10-09

ana-4.0.51-py3 [dubrovin@sdfiana001:~/LCLS/con-py3]$ det_pixel_status -h
usage: 
det_pixel_status works with 2-d raw arrays ONLY! For 3-d the segment index --segind needs to be specified
det_pixel_status -k <dataset> -d <detname> ...
Ex.1:  det_pixel_status -k exp=xpplw3319:run=287 -d XppGon.0:Epix100a.3 -F 1,2,3 -n 1000   # dark  processing for Feat.1 & 2 & 3
Ex.2:  det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 1,6 -n 1000     # light processing for Feat.1 & 6
Ex.3:  det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 11, -n 1000000  # light processing for Feat.11
 Feature # stands for
   1: mean intensity of frames in good range
   2: dark mean in good range
   3: dark rms in good range
   4: NOT-IMPLEMENTED for gain stage indicator values
   5: NOT-IMPLEMENTED for gain stage indicator values
   6: light average SNR of pixels over time
  11: light intensity max-peds in good range - should be processed separately from all other features on entire/large set of events.

Help:  det_pixel_status -h

Evaluates pixel status of any detector/segment using raw dark/light data

optional arguments:
  -h, --help            show this help message and exit
  -k DSKWARGS, --dskwargs DSKWARGS
                        (str) DataSource parameters, default = exp=xpplw3319:run=293
  -d DETNAME, --detname DETNAME
                        (str) detector/detname name, default = epix_alc3
  -n EVENTS, --events EVENTS
                        (int) maximal number of events total (in runs, steps), default = 1000
  -m EVSKIP, --evskip EVSKIP
                        (int) number of events to skip in the beginning of each step, default = 0
  -N STEPS, --steps STEPS
                        (int) maximal number of steps or all by default, default = None
  -M STSKIP, --stskip STSKIP
                        (int) number of steps to skip before processing or process all by default, default = None
  -c EVCODE, --evcode EVCODE
                        (str) comma separated event codes for selection as OR combination; any negative code inverts selection, default = None
  -i SEGIND, --segind SEGIND
                        (int) segment index for multi-panel detectors with raw.shape.ndim>2, default = 0
  -S SLICE, --slice SLICE
                        (str) FOR DEBUGGING ONLY (str) slice of the panel image 2-k array selected for plots and pixel status, ex. "0:144,0:192", default = None
  -r NRECS, --nrecs NRECS
                        (int) number of records to collect data, default = 1000
-w SHWIND, --shwind SHWIND
                        (str) window shape for feature 6 fitting to plane, ex. "15,15", default = 15,15
  -R SNRMAX, --snrmax SNRMAX
                        (int) width of the good region in terms on number of spread/rms, default = 8.0
  -t CTYPE, --ctype CTYPE
                        (str) type of calibration constants to save, default = status_data
  -L LOGMODE, --logmode LOGMODE
                        (str) logging mode, one of CRITICAL, FATAL, ERROR, WARN, WARNING, INFO, DEBUG, NOTSET, default = INFO
  -o DIRREPO, --dirrepo DIRREPO
                        (str) repository for calibration results, default = /sdf/group/lcls/ds/ana/detector/calib/constants/
  -F FEATURES, --features FEATURES
                        (str) comma-separated list of Features [JAC-2022, Sadri, Automatic bad pixel mask maker...] from 1 to 6 to evaluate bad pixels, default = 1,2,3
  --dirmode DIRMODE     (int) mode for all mkdir, default = 0o2775
  --filemode FILEMODE   (int) mode for all saved files, default = 0o664
  --group GROUP         (str) group ownership for all files, default = ps-users
  --databits DATABITS   (int) data bits in ADC for code of intensity, default = 0o37777
  --gainbits GAINBITS   (int) gain mode switch bits in ADC, default = 0o0
  --gmode GMODE         (str) gain mode name-suffix for multi-gain detectors with raw.shape.ndim>3, ex: AHL-H, default = None

Dark processing

dark data processing for Feat.1 & 2 & 3, Ref.1

det_pixel_status -k exp=xpplw3319:run=287 -d XppGon.0:Epix100a.3 -F 1,2,3 -n 1000

almost equivalent to pixel_status evaluarion in psana dark processing, but use different approach to definition of limits good value range.

It works on collected block of frames shaped as (<number-of-frames>, <2-d-frame-size>) which implies limits on consumed memory.

Summary for det_pixel_status ... -F 1,2,3

[I] UtilsPixelStatus.py L0389 
Summary of the bad pixel status evaluation for SNR=8.00, status_dark array
                   0o1:        0 / 540672 ( 0.000%) pixels Feat.2 mean <= 2933.437
                   0o2:       33 / 540672 ( 0.006%) pixels Feat.2 mean >= 5029.965
                   0o4:        0 / 540672 ( 0.000%) pixels Feat.3 std <= 1.525
                  0o10:     5314 / 540672 ( 0.983%) pixels Feat.3 std >= 4.053
    Any bad status bit:     5319 / 540672 ( 0.984%) pixels
[I] RepoManager.py L0176 panel directory: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424
[I] UtilsCalib.py L0150 saved:  /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424/status_dark/epix100a_0009_20210716061106_xpplw3319_r0287_status_dark.dat
[I] UtilsPixelStatus.py L0639 Consumed time 19.674 sec
[I] RepoManager.py L0255 
  move logfile: /sdf/group/lcls/ds/ana/detector/calib/constants/scripts/det_pixel_status/logs/2023/2023-10-09T115328_log_det_pixel_status_dubrovin.txt
  to: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/logs/2023/2023-10-09T115328_log_det_pixel_status_dubrovin.txt
  and create link

Light processing with fitting light background shape

light data processing for Feat.1 & 6, Ref.1

det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 1,6 -n 1000

It works on collected block of frames shaped as (<number-of-frames>, <2-d-frame-size>) which implies limits on consumed memory.

Summary of det_pixel_status ... -F1,6

[I] UtilsPixelStatus.py L0389 
Summary of the bad pixel status evaluation for SNR=8.00, status_light array
                  0o20:        0 / 540672 ( 0.000%) pixels Feat.6 res_med <= -881.291
                  0o40:       33 / 540672 ( 0.006%) pixels Feat.6 res_med >= 882.400
                 0o100:        0 / 540672 ( 0.000%) pixels Feat.6 res_spr <= 0.954
                 0o200:     4925 / 540672 ( 0.911%) pixels Feat.6 res_spr >= 2.613
    Any bad status bit:     4930 / 540672 ( 0.912%) pixels
[I] RepoManager.py L0176 panel directory: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424
[I] UtilsCalib.py L0150 saved:  /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424/status_light/epix100a_0009_20210716073950_xpplw3319_r0293_status_light.dat
[I] UtilsPixelStatus.py L0639 Consumed time 3443.990 sec
[I] RepoManager.py L0255 
  move logfile: /sdf/group/lcls/ds/ana/detector/calib/constants/scripts/det_pixel_status/logs/2023/2023-10-09T101656_log_det_pixel_status_dubrovin.txt
  to: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/logs/2023/2023-10-09T101656_log_det_pixel_status_dubrovin.txt
  and create link

Light processing for max values

light data processing for Feat.11, Ref.1

det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 11, -n 1000000

Should be processed separately from all other features on entire/large set of events, -n 1000000.

Bad pixel status evaluation using park and light data