You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Content

Example off commands

Script and its features

Script det_pixel_status processes dark or light data and produces 2-d image or per-panel pixel status files. Implemented algorithms, named here as Features after Ref.2,3, are developed in R&D with Chuck briefly documented in Ref.1.

All algorithms/features are wrapped under the common hood of event processing loop and files' I/O.  But, not all algorithms/features can be applied simultaneously due to different data types (dark or light) and different processing algorithms. Suggested combinations for -F parameters are shown in three examples below features

  1. -F 1,2,3
  2. -F 1,6
  3. -F 11

Get help

det_pixel_status -h # on 2023-10-09 
ana-4.0.51-py3 [username@sdfiana001:~/]$ det_pixel_status -h
usage: 
det_pixel_status works with 2-d raw arrays ONLY! For 3-d the segment index --segind needs to be specified
det_pixel_status -k <dataset> -d <detname> ...
Ex.1:  det_pixel_status -k exp=xpplw3319:run=287 -d XppGon.0:Epix100a.3 -F 1,2,3 -n 1000   # dark  processing for Feat.1 & 2 & 3
Ex.2:  det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 1,6 -n 1000     # light processing for Feat.1 & 6
Ex.3:  det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 11, -n 1000000  # light processing for Feat.11
 Feature # stands for
   1: mean intensity of frames in good range
   2: dark mean in good range
   3: dark rms in good range
   4: NOT-IMPLEMENTED for gain stage indicator values
   5: NOT-IMPLEMENTED for gain stage indicator values
   6: light average SNR of pixels over time
  11: light intensity max-peds in good range - should be processed separately from all other features on entire/large set of events.

Help:  det_pixel_status -h

Evaluates pixel status of any detector/segment using raw dark/light data

optional arguments:
  -h, --help            show this help message and exit
  -k DSKWARGS, --dskwargs DSKWARGS
                        (str) DataSource parameters, default = exp=xpplw3319:run=293
  -d DETNAME, --detname DETNAME
                        (str) detector/detname name, default = epix_alc3
  -n EVENTS, --events EVENTS
                        (int) maximal number of events total (in runs, steps), default = 1000
  -m EVSKIP, --evskip EVSKIP
                        (int) number of events to skip in the beginning of each step, default = 0
  -N STEPS, --steps STEPS
                        (int) maximal number of steps or all by default, default = None
  -M STSKIP, --stskip STSKIP
                        (int) number of steps to skip before processing or process all by default, default = None
  -c EVCODE, --evcode EVCODE
                        (str) comma separated event codes for selection as OR combination; any negative code inverts selection, default = None
  -i SEGIND, --segind SEGIND
                        (int) segment index for multi-panel detectors with raw.shape.ndim>2, default = 0
  -S SLICE, --slice SLICE
                        (str) FOR DEBUGGING ONLY (str) slice of the panel image 2-k array selected for plots and pixel status, ex. "0:144,0:192", default = None
  -r NRECS, --nrecs NRECS
                        (int) number of records to collect data, default = 1000
-w SHWIND, --shwind SHWIND
                        (str) window shape for feature 6 fitting to plane, ex. "15,15", default = 15,15
  -R SNRMAX, --snrmax SNRMAX
                        (int) width of the good region in terms on number of spread/rms, default = 8.0
  -t CTYPE, --ctype CTYPE
                        (str) type of calibration constants to save, default = status_data
  -L LOGMODE, --logmode LOGMODE
                        (str) logging mode, one of CRITICAL, FATAL, ERROR, WARN, WARNING, INFO, DEBUG, NOTSET, default = INFO
  -o DIRREPO, --dirrepo DIRREPO
                        (str) repository for calibration results, default = /sdf/group/lcls/ds/ana/detector/calib/constants/
  -F FEATURES, --features FEATURES
                        (str) comma-separated list of Features [JAC-2022, Sadri, Automatic bad pixel mask maker...] from 1 to 6 to evaluate bad pixels, default = 1,2,3
  --dirmode DIRMODE     (int) mode for all mkdir, default = 0o2775
  --filemode FILEMODE   (int) mode for all saved files, default = 0o664
  --group GROUP         (str) group ownership for all files, default = ps-users
  --databits DATABITS   (int) data bits in ADC for code of intensity, default = 0o37777
  --gainbits GAINBITS   (int) gain mode switch bits in ADC, default = 0o0
  --gmode GMODE         (str) gain mode name-suffix for multi-gain detectors with raw.shape.ndim>3, ex: AHL-H, default = None

Dark processing

dark data processing for Feat.1 & 2 & 3, Ref.1

det_pixel_status -k exp=xpplw3319:run=287 -d XppGon.0:Epix100a.3 -F 1,2,3 -n 1000

These algorithms almost equivalent to pixel_status evaluarion in psana regular dark processing, command calibrun. It has different approach to definition of the good value range limits.

It works on collected block of frames shaped as (<number-of-frames>, <2-d-frame-size>) which implies limits on consumed memory.

Summary for det_pixel_status ... -F 1,2,3 
[I] UtilsPixelStatus.py L0389 
Summary of the bad pixel status evaluation for SNR=8.00, status_dark array
                   0o1:        0 / 540672 ( 0.000%) pixels Feat.2 mean <= 2933.437
                   0o2:       33 / 540672 ( 0.006%) pixels Feat.2 mean >= 5029.965
                   0o4:        0 / 540672 ( 0.000%) pixels Feat.3 std <= 1.525
                  0o10:     5314 / 540672 ( 0.983%) pixels Feat.3 std >= 4.053
    Any bad status bit:     5319 / 540672 ( 0.984%) pixels
[I] RepoManager.py L0176 panel directory: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424
[I] UtilsCalib.py L0150 saved:  /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424/status_dark/epix100a_0009_20210716061106_xpplw3319_r0287_status_dark.dat
[I] UtilsPixelStatus.py L0639 Consumed time 19.674 sec
[I] RepoManager.py L0255 
  move logfile: /sdf/group/lcls/ds/ana/detector/calib/constants/scripts/det_pixel_status/logs/2023/2023-10-09T115328_log_det_pixel_status_dubrovin.txt
  to: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/logs/2023/2023-10-09T115328_log_det_pixel_status_dubrovin.txt
  and create link

Light processing with fitting light background shape

light data processing for Feat.1 & 6, Ref.1

det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 1,6 -n 1000

It works on collected block of frames shaped as (<number-of-frames>, <2-d-frame-size>) which implies limits on consumed memory.

Summary for det_pixel_status ... -F1,6 
[I] UtilsPixelStatus.py L0389 
Summary of the bad pixel status evaluation for SNR=8.00, status_light array
                  0o20:        0 / 540672 ( 0.000%) pixels Feat.6 res_med <= -881.291
                  0o40:       33 / 540672 ( 0.006%) pixels Feat.6 res_med >= 882.400
                 0o100:        0 / 540672 ( 0.000%) pixels Feat.6 res_spr <= 0.954
                 0o200:     4925 / 540672 ( 0.911%) pixels Feat.6 res_spr >= 2.613
    Any bad status bit:     4930 / 540672 ( 0.912%) pixels
[I] RepoManager.py L0176 panel directory: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424
[I] UtilsCalib.py L0150 saved:  /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424/status_light/epix100a_0009_20210716073950_xpplw3319_r0293_status_light.dat
[I] UtilsPixelStatus.py L0639 Consumed time 3443.990 sec
[I] RepoManager.py L0255 
  move logfile: /sdf/group/lcls/ds/ana/detector/calib/constants/scripts/det_pixel_status/logs/2023/2023-10-09T101656_log_det_pixel_status_dubrovin.txt
  to: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/logs/2023/2023-10-09T101656_log_det_pixel_status_dubrovin.txt
  and create link

Light processing for max values

light data processing for Feat.11, Ref.1

det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 11, -n 1000000

This algorithm accumulates array of max values over large number of events and analyzes distribution of max values over the frame. Should be processed separately from all other features on entire/large set of events, -n 1000000.

Summary for det_pixel_status ... -F 11, -n 1000000 
[I] UtilsPixelStatus.py L0389 
Summary of the bad pixel status evaluation for SNR=8.00, status_max array
                 0o400:     7737 / 540672 ( 1.431%) pixels Feat.11 max-peds <= 114.447
                0o1000:     3106 / 540672 ( 0.574%) pixels Feat.11 max-peds >= 216.111
    Any bad status bit:    10843 / 540672 ( 2.005%) pixels
[I] RepoManager.py L0176 panel directory: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424
[I] UtilsCalib.py L0150 saved:  /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424/status_max/epix100a_0009_20210716073950_xpplw3319_r0293_status_max.dat
[I] UtilsPixelStatus.py L0639 Consumed time 113.834 sec
[I] RepoManager.py L0255 
  move logfile: /sdf/group/lcls/ds/ana/detector/calib/constants/scripts/det_pixel_status/logs/2023/2023-10-09T120040_log_det_pixel_status_dubrovin.txt
  to: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/logs/2023/2023-10-09T120040_log_det_pixel_status_dubrovin.txt
  and create link

References

  1. Bad pixel mask for epix100a xpplw3319
  2. Paper in JAC: Automatic bad-pixel mask maker for X-ray pixel detectors with application to serial crystallography,
  3. Journal of Applied Crystallography - 2022 - Sadri - Automatic bad‐pixel mask maker for X‐ray pixel detectors with.pdf
  4. Bad Pixel Status


  • No labels