Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Content

Table of Contents

...

Command examples

Script and its features

  • Script det_pixel_status processes dark or light data and produces 2-d image or per-panel pixel status files.
  • Implemented algorithms, named here as Features after Ref.2,3, are developed in R&D with Chuck briefly documented in Ref.1.
  • Tested for epix100a - other detectors not tested...

All algorithms/features are wrapped under the common hood of event processing loop and files' I/O.  But, not all algorithms/features can be applied simultaneously due to different data types (dark or light) and different processing algorithms. Suggested combinations for -F parameters are shown in three examples below features with associated type of generated status constants in repository

  1. -F 1,2,3 # status_dark
  2. -F 1,6 # status_light
  3. -F 11

Get help

  1. # status_max

Location of repository and calibration files can be found in the log file, see also Ref.5.

Get help

Code Block
titledet_pixel
Code Block
titledet_pixel_status -h # on 2023-10-09
collapsetrue
ana-4.0.51-py3 [username@sdfiana001:~/]$ det_pixel_status -h
usage: 
det_pixel_status works with 2-d raw arrays ONLY! For 3-d the segment index --segind needs to be specified
det_pixel_status -k <dataset> -d <detname> ...
Ex.1:  det_pixel_status -k exp=xpplw3319:run=287 -d XppGon.0:Epix100a.3 -F 1,2,3 -n 1000   # dark  processing for Feat.1 & 2 & 3
Ex.2:  det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 1,6 -n 1000     # light processing for Feat.1 & 6
Ex.3:  det_pixel_status -k exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -F 11, -n 1000000  # light processing for Feat.11
 Feature # stands for
   1: mean intensity of frames in good range
   2: dark mean in good range
   3: dark rms in good range
   4: NOT-IMPLEMENTED for gain stage indicator values
   5: NOT-IMPLEMENTED for gain stage indicator values
   6: light average SNR of pixels over time
  11: light intensity max-peds in good range - should be processed separately from all other features on entire/large set of events.

Help:  det_pixel_status -h

Evaluates pixel status of any detector/segment using raw dark/light data

optional arguments:
  -h, --help            show this help message and exit
  -k DSKWARGS, --dskwargs DSKWARGS
                        (str) DataSource parameters, default = exp=xpplw3319:run=293
  -d DETNAME, --detname DETNAME
                        (str) detector/detname name, default = epix_alc3
  -n EVENTS, --events EVENTS
                        (int) maximal number of events total (in runs, steps), default = 1000
  -m EVSKIP, --evskip EVSKIP
                        (int) number of events to skip in the beginning of each step, default = 0
  -N STEPS, --steps STEPS
                        (int) maximal number of steps or all by default, default = None
  -M STSKIP, --stskip STSKIP
                        (int) number of steps to skip before processing or process all by default, default = None
  -c EVCODE, --evcode EVCODE
                        (str) comma separated event codes for selection as OR combination; any negative code inverts selection, default = None
  -i SEGIND, --segind SEGIND
                        (int) segment index for multi-panel detectors with raw.shape.ndim>2, default = 0
  -S SLICE, --slice SLICE
                        (str) FOR DEBUGGING ONLY (str) slice of the panel image 2-k array selected for plots and pixel status, ex. "0:144,0:192", default = None
  -r NRECS, --nrecs NRECS
                        (int) number of records to collect data, default = 1000
-w SHWIND, --shwind SHWIND
                        (str) window shape for feature 6 fitting to plane, ex. "15,15", default = 15,15
  -R SNRMAX, --snrmax SNRMAX
                        (int) width of the good region in terms on number of spread/rms, default = 8.0
  -t CTYPE, --ctype CTYPE
                        (str) type of calibration constants to save, default = status_data
  -L LOGMODE, --logmode LOGMODE
                        (str) logging mode, one of CRITICAL, FATAL, ERROR, WARN, WARNING, INFO, DEBUG, NOTSET, default = INFO
  -o DIRREPO, --dirrepo DIRREPO
                        (str) repository for calibration results, default = /sdf/group/lcls/ds/ana/detector/calib/constants/
  -F FEATURES, --features FEATURES
                        (str) comma-separated list of Features [JAC-2022, Sadri, Automatic bad pixel mask maker...] from 1 to 6 to evaluate bad pixels, default = 1,2,3
  --dirmode DIRMODE     (int) mode for all mkdir, default = 0o2775
  --filemode FILEMODE   (int) mode for all saved files, default = 0o664
  --group GROUP         (str) group ownership for all files, default = ps-users
  --databits DATABITS   (int) data bits in ADC for code of intensity, default = 0o37777
  --gainbits GAINBITS   (int) gain mode switch bits in ADC, default = 0o0
  --gmode GMODE         (str) gain mode name-suffix for multi-gain detectors with raw.shape.ndim>3, ex: AHL-H, default = None

...

Code Block
titleSummary for det_pixel_status ... -F 11, -n 1000000
collapsetrue
[I] UtilsPixelStatus.py L0389 
Summary of the bad pixel status evaluation for SNR=8.00, status_max array
                 0o400:     7737 / 540672 ( 1.431%) pixels Feat.11 max-peds <= 114.447
                0o1000:     3106 / 540672 ( 0.574%) pixels Feat.11 max-peds >= 216.111
    Any bad status bit:    10843 / 540672 ( 2.005%) pixels
[I] RepoManager.py L0176 panel directory: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424
[I] UtilsCalib.py L0150 saved:  /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/3925999620-0996513537-2080374794-1794135040-0940361739-2398406657-0419430424/status_max/epix100a_0009_20210716073950_xpplw3319_r0293_status_max.dat
[I] UtilsPixelStatus.py L0639 Consumed time 113.834 sec
[I] RepoManager.py L0255 
  move logfile: /sdf/group/lcls/ds/ana/detector/calib/constants/scripts/det_pixel_status/logs/2023/2023-10-09T120040_log_det_pixel_status_dubrovin.txt
  to: /sdf/group/lcls/ds/ana/detector/calib/constants/epix100a/logs/2023/2023-10-09T120040_log_det_pixel_status_dubrovin.txt
  and create link

References

Deployment of status_extra constants

For detail see Ref.6 Detector Calibration Constants Deployment, e.g.:

deploy_constants -x exp=xpplw3319:run=293 -d XppGon.0:Epix100a.3 -C status_extra -D # -c ./calib  -R 1-end

merges all status_* constants to status_merged in repository and deploys under /sdf/data/lcls/ds/xpp/xpplw3319/calib/Epix100a::CalibV1/XppGon.0:Epix100a.3/status_extra/

Release

Code available since ana-4.0.53-py3

Code Block
titleWhile ana-4.0.53-py3 is not a default...
# on sdf psana node:
source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh
conda deactivate
conda activate ana-4.0.53-py3

References

  1. Bad pixel mask for epix100a xpplw3319
  2. Paper in JAC: Automatic bad-
  3. Bad pixel mask for epix100a xpplw3319
  4. Paper in JAC: Automatic bad-pixel mask maker for X-ray pixel detectors with application to serial crystallography,
  5. Journal of Applied Crystallography - 2022 - Sadri - Automatic bad‐pixel mask maker for X‐ray pixel detectors with.pdf
  6. Bad Pixel Status
  7. Calibration Scripts Repository and Logging
  8. Detector Calibration Constants Deployment