Content

Intro

In April 2021 we introduce a new approach to dark runs processing for Jungfrau detectors. This approach is pretty much similar to epix10ka dark calibration based on per-panel processing of constants. Entire dark calibration is split for two stages. At first stage specialized dark runs for three gain modes or regular runs with "drop-shots" are processed and results per-panel are saved in the repository. At second stage, per-panel constants are merged for particular detector or set of panels (for DRP) and deployed in the regular calibration directory. This approach allows to manage each panel and its gain ranges separately.

Repository organization

/reg/g/psdm/detector/gains/jungfrau/panels/ # repository for jungfrau per-panel constants
- 190408-181206-50c246df50010d/ # e.g. - for one of the panel ids
  - rms/ # subdirectory for calibration constant's type
  - status/
  - dark_max/
  - dark_min/
  - pedestals/
    - jungfrau_0001_20201201073354_cxilu9218_r0230_pedestals_gm0-Normal.dat # e.g. - panel constants for gain mode 0: Normal
      jungfrau_0001_20201201073425_cxilu9218_r0231_pedestals_gm1-ForcedGain1.dat
      jungfrau_0001_20201201073453_cxilu9218_r0232_pedestals_gm2-ForcedGain2.dat
      jungfrau_0001_20201201085333_cxilu9218_r0238_pedestals_gm0-Normal.dat
      jungfrau_0001_20201201085333_cxilu9218_r0238_pedestals.txt # e.g. - per-panel. constants merged for three gain modes valid for runs >=238
- merge_tmp/
  - CxiDs1.0:Jungfrau.0-20201201085333-cxilu9218-r0238-pedestals.txt # merged constants for detector in run 238
- logs/
  - 2021_log_jungfrau_dark_proc.txt # log files with single record per job for each script
    2021_log_jungfrau_deploy_constants.txt
  - 2021/ # anual directory for job submission commands
    - 2021-04-23T142354_log_jungfrau_dark_proc_dubrovin.txt # log file for each job
      2021-04-23T154102_log_jungfrau_deploy_constants_dubrovin.txt

Calibration file naming conventions

File name example: jungfrau_0001_20201201073354_cxilu9218_r0230_pedestals_gm0-Normal.dat

Fields meaning: <panel-type>_<panel-unique-numeric-alias>_<run-time-stamp>_<experiment>_r<run-number>_<constants-type>_gm<gain-mode-index-and-name>.dat

Log file naming conventions

File name example: 2021-04-23T142354_log_jungfrau_dark_proc_dubrovin.txt

Fields meaning: <job-submission-time-stamp>_<script-name>_<login-name>.txt

Dark processing

jungfrau_dark_processing

Proceses jungfrau dark data

jungfrau_dark_proc -d <dataset> -s <source> ...
Ex.1: jungfrau_dark_proc -d exp=cxilu9218:run=242,243,244:smd -s CxiDs1.0:Jungfrau.0 # regular dark processing
Ex.2: jungfrau_dark_proc -d exp=cxilu9218:run=242,243,244:smd -s CxiDs1.0:Jungfrau.0 -I1 # speed-up for single panel
Ex.3: jungfrau_dark_proc -d exp=cxilu9218:run=238:smd -s CxiDs1.0:Jungfrau.0 --evcode 162 # use drop-shots as dark events

Help: jungfrau_dark_proc -h

Merging and deployment of constants

jungfrau_merge_constants

Deployment jungfrau calibration parameters

jungfrau_deploy_constants -e <experiment> -d <detector> -r <run-number> [-D] [-L <logging-mode>] [...]

TEST COMMAND:
Ex.1: jungfrau_deploy_constants -e cxilu9218 -d CxiDs1.0:Jungfrau.0 -r238
Ex.2: jungfrau_deploy_constants -e cxilu9218 -d CxiDs1.0:Jungfrau.0 -r238 -D -c ./calib

REGULAR COMMAND:
Ex.3: jungfrau_deploy_constants -e cxilu9218 -d CxiDs1.0:Jungfrau.0 -r238 -D

Help: jungfrau_deploy_constants -h

Example for parallel processing with time comparison

Dataset exp=xpplw4319:run=1 contains three calib-cycles with dark data for 2-panel XppEndstation.0:Jungfrau.0

process all steps (all gain ranges) and all panels takes 306 sec on psanagpu103 with command

jungfrau_dark_proc -d exp=xpplw4319:run=1:smd:stream=0-79 -s XppEndstation.0:Jungfrau.0

process all steps for a single panel (--segind=1 is selected from 0,1) takes 103 sec

jungfrau_dark_proc -d exp=xpplw4319:run=1:smd:stream=0-79 -s XppEndstation.0:Jungfrau.0 --segind=1

process a single step (--stepnum=2 is selected from 0,1,2) for all panels takes 83 sec with command

jungfrau_dark_proc -d exp=xpplw4319:run=1:smd:stream=0-79 -s XppEndstation.0:Jungfrau.0 --stepnum=2

process a single step (--stepnum=2) for a single panel (--segind=1) takes 44 sec with command

jungfrau_dark_proc -d exp=xpplw4319:run=1:smd:stream=0-79 -s XppEndstation.0:Jungfrau.0 --stepnum=2 --segind=1

Then results have to be deployed in the calib directory using command

jungfrau_deploy_constants -e xpplw4319 -d XppEndstation.0:Jungfrau.0 -r1 -D

Optional parameters on 2021-12-10

jungfrau_dark_proc -h

Proceses jungfrau dark data

Options:
  -h, --help            show this help message and exit
  -d DSNAME, --dsname=DSNAME
                        dataset name, default = None
  -s SOURCE, --source=SOURCE
                        input ndarray source name, default = None
  -n EVENTS, --events=EVENTS
                        maximal number of events total (in runs, steps),
                        default = 100000
  -m EVSKIP, --evskip=EVSKIP
                        number of events to skip in the beginning of each
                        step, default = 0
  -e EVSTEP, --evstep=EVSTEP
                        maximal number of events to process in each step,
                        default = 2000
  -b INT_LO, --int_lo=INT_LO
                        intensity low limit, default = 1
  -t INT_HI, --int_hi=INT_HI
                        intensity high limit, default = 16000
  -B RMS_LO, --rms_lo=RMS_LO
                        rms low limit, default = 0.001
  -T RMS_HI, --rms_hi=RMS_HI
                        rms high limit, default = 16000
  -F FRACLM, --fraclm=FRACLM
                        allowed fraction limit, default = 0.1
  -p PLOTIM, --plotim=PLOTIM
                        control bit-word to plot images, default = 0
  -D INTNLO, --intnlo=INTNLO
                        number of sigma from mean for low limit on INTENSITY,
                        default = 6.0
  -U INTNHI, --intnhi=INTNHI
                        number of sigma from mean for high limit on INTENSITY,
                        default = 6.0
  -L RMSNLO, --rmsnlo=RMSNLO
                        number of sigma from mean for low limit on RMS,
                        default = 6.0
  -H RMSNHI, --rmsnhi=RMSNHI
                        number of sigma from mean for high limit on RMS,
                        default = 6.0
  -c EVCODE, --evcode=EVCODE
                        comma separated event codes for selection as OR
                        combination, any negative code inverts selection,
                        default = None
  -u, --upload          upload files in calib directory, default = False
  -N STEPNUM, --stepnum=STEPNUM
                        step/calibcycle number [0,2] or all by default,
                        default = None
  -M STEPMAX, --stepmax=STEPMAX
                        maximal number of steps or all by default, default =
                        None
  -I SEGIND, --segind=SEGIND
                        segment index to process, default = None
  --nrecs=NRECS         number of records to collect data, default = 1000
  --nrecs1=NRECS1       number of records for 1st stage processing, default =
                        50
  --fraclo=FRACLO       fraction of statistics [0,1] below low  limit of the
                        gate, default = 0.050000
  --frachi=FRACHI       fraction of statistics [0,1] below high limit of the
                        gate, default = 0.950000
  -o DIRREPO, --dirrepo=DIRREPO
                        repository for calibration results, default =
                        /reg/g/psdm/detector/gains/jungfrau/panels
  --logmode=LOGMODE     logging mode, one of INFO, CRITICAL, WARN, WARNING,
                        ERROR, DEBUG, NOTSET, default = INFO

jungfrau_deploy_constants -h

Deployment jungfrau calibration parameters

Options:
  -h, --help            show this help message and exit
  -e EXP, --exp=EXP     experiment name, default = None
  -d DET, --det=DET     detector name, default = None
  -r RUN, --run=RUN     run number for beginning of the validity range,
                        default = None
  -t TSTAMP, --tstamp=TSTAMP
                        non-default time stamp (<YYYYmmddHHMMSS) or run
                        number(<10000) for constants selection in repo. By
                        default run time is used, default = None
  -x DIRXTC, --dirxtc=DIRXTC
                        non-default xtc directory which is used to access run
                        start time, default = None
  -o DIRREPO, --dirrepo=DIRREPO
                        non-default repository of calibration results, default
                        = /reg/g/psdm/detector/gains/jungfrau/panels
  -c DIRCALIB, --dircalib=DIRCALIB
                        deployment calib directory if different from standard
                        one, default = None
  -I PANINDS, --paninds=PANINDS
                        comma-separated (str) panel indexds to generate
                        constants for subset of panels (ex. 2 panel of 8 in
                        the detector), default = None
  -D, --deploy          deploy constants to the calib dir, default = False
  -L LOGMODE, --logmode=LOGMODE
                        logging mode, one of INFO, CRITICAL, WARN, WARNING,
                        ERROR, DEBUG, NOTSET, default = INFO

Jungfrau dark processing