You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 30 Next »

Intro

Evolution of calibration system of LCLS creates new opportunities and increase complexity. Large number of detectors and scripts for calibration processing makes it hard to track down what is going on. This note is intended to collect in one spot references to location of all intermediate repositories and log files.

Environment variables

psana/psdm

lcls1

source /cds/sw/ds/ana/conda1/manage/bin/psconda.sh, setup_testrel

export SIT_ROOT=/reg/g/psdm/
export SIT_DATA=/cds/group/psdm/data/
export SIT_PSDM_DATA=/cds/data/psdm/

lclc2

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh

export DIR_PSDM=/cds/group/psdm

DIR_ROOT = os.getenv('DIR_PSDM')

DIR_LOG_AT_START = /cds/group/psdm/detector/logs/atstart/

DIR_REPO = /cds/group/psdm/detector/calib2/constants/

s3df

lcls1

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh, setup_testrel

export SIT_ROOT=/sdf/group/lcls/ds/ana/
export SIT_DATA=/sdf/group/lcls/ds/ana/data/
export SIT_PSDM_DATA=/sdf/data/lcls/ds/

lclc2

source /sdf/group/lcls/ds/ana/sw/conda2/manage/bin/psconda.sh

LCLS scripts

scriptrecord at startrepositorylogscontrol parameterscomments
area detectors




calibrun

>ana-4.0.46-py3:

$SIT_ROOT/detector/logs/atstart/2022/2022_lcls1_calibrun.txt

>ana-4.0.46-py3:

$SIT_ROOT/detector/calib/constants/calibrun/

/reg/g/psdm/logs/calibman/<year>/<month>/<repository>/logs/2022/2022-10-12T103606_log_calibrun_dubrovin.txt

>ana-4.0.46-py3:

<repository>/logs/<year>/<time-stamp>_log_calibrun_<login>.txt

<repository>/logs/2022/2022-10-12T103606_log_calibrun_dubrovin.txt

N/ACLI dark processing for generic detector
calibfile

/reg/g/psdm/logs/calibman/<year>/<month>/calibfile.txt

>ana-4.0.46-py3:

$SIT_ROOT/detector/logs/atstart/2022/2022_lcls1_calibfile.txt

>ana-4.0.46-py3:

$SIT_ROOT/detector/calib/constants/calibfile/

>ana-4.0.46-py3:

<repository>/logs/<year>/<time-stamp>_log_calibfile_<login>.txt

<repository>/logs/2022/2022-10-13T170608_log_calibfile_dubrovin.txt

-l, --lfnameCLI for calib file deployment under .../<experiment>/calib/...
geometry_deploy_constants

/reg/g/psdm/logs/atstart/<year>/<year>_lcls1_geometry_deploy_constants.txt

>ana-4.0.46-py3:

$SIT_ROOT/detector/logs/atstart/2022/2022_lcls1_geometry_deploy_constants.txt

/reg/g/psdm/detector/calib/geometry/

>ana-4.0.46-py3:

$SIT_ROOT/detector/calib/geometry

<repository>/<dettype>/<dettype>_<full-lcls1-detector-name>_<time-stamp>.data

>ana-4.0.46-py3:

<repository>/logs/<year>/<time-stamp>_log_geometry_deploy_constants_<login>.txt

<repository>/logs/2022/2022-10-14T094805_log_geometry_deploy_constants_dubrovin.txt

-o, --dirrepoGeometry calibration constants deployment CLI
dcs

/reg/g/psdm/logs/atstart/<year>/<year>_lcls1_dcs.txt

>ana-4.0.46-py3:

$SIT_ROOT/detector/logs/atstart/<year>/<year>_lcls1_dcs.txt

/reg/g/psdm/<INS>/<experiment>/calib/<dettype> # default experiment-detector repo

or parameter -c <path to local calib>

e.g.:

-c /reg/d/psdm/detector/calib # detector-dependent repo

>ana-4.0.46-py3:

$SIT_PSDM_DATA/XPP/xpptut15/calib/epix100a
$SIT_PSDM_DATA/detector/calib/epix100a

run-time only
Interaction with dcs-store - hdf5 file calibration constants.

deploy_constants

for releases > ana-4.0.42

/reg/g/psdm/logs/atstart/<year>/<year>_lcls1_deploy_constants.txt

>ana-4.0.46-py3:

$SIT_ROOT/detector/logs/atstart/2022/2022_lcls1_deploy_constants.txt

/reg/g/psdm/detector/calib/constants/

>ana-4.0.46-py3:

$SIT_ROOT/detector/calib/constants/

<repository>/logs/<year>/<time-stamp>_log_deploy_constants_<uid>.txt

<repository>/logs/2022/2022-10-14T105634_log_deploy_constants_dubrovin.txt

-o , --dirrepo Deployment of default constants for all generic (one gain range) detectors. Special case of deployment with merging at -t status_extra
det_raw_pixel_status

/reg/g/psdm/detector/logs/atstart/2023/2023_lcls1_det_raw_pixel_status.txt

$SIT_ROOT/detector/calib/constants//reg/g/psdm/detector/calib/constants/logs/<year>/<time-stamp>_log_det_raw_pixel_status_<uid>.txt
/reg/g/psdm/detector/calib/constants/epix100a/logs/<year>/<time-stamp>_log_det_raw_pixel_status_<uid>.txt
-o, --dirrepo Light data processing for bad pixel status. See: Bad pixel mask ...






epix10ka
epix10ka_offset_calibration

<repository>/logs/<year>_epix10ka_offset_calibration.txt

>ana-4.0.37:

/cds/group/psdm/logs/atstart/<year>/<year>_lcls1_epix10ka_offset_calibration.txt

/reg/g/psdm/detector/gains/epix10k/panels/

<repository>/logs/<year>/<time-stamp>_log_epix10ka_offset_calibration_<login>.txt ????

-o, --dirrepoProcessing of the charge injection runs with 103 steps/calib-cycles
epix10ka_pedestals_calibration

<repository>/logs/<year>_epix10ka_pedestals_calibration.txt

>ana-4.0.37:

/cds/group/psdm/logs/atstart/<year>/<year>_lcls1_epix10ka_offset_calibration.txt

/reg/g/psdm/detector/gains/epix10k/panels/

<repository>/logs/<year>/<time-stamp>_log_epix10ka_pedestals_calibration_<login>.txt

-o, --dirrepoDark processing of runs with 5 steps/calib-cycles
epix10ka_deploy_constants

<repository>/logs/<year>_epix10ka_deploy_constants.txt

>ana-4.0.37:

/cds/group/psdm/logs/atstart/<year>/<year>_lcls1_epix10ka_deploy_constants.txt

/reg/g/psdm/detector/gains/epix10k/panels/

<repository>/logs/<year>/<time-stamp>_log_epix10ka_deploy_constants_<login>.txt

-o, --dirrepoMerging and deployment ofper-panel constants.






Jungfrau
jungfrau_dark_proc

<repository>/logs/<year>_log_jungfrau_dark_proc.txt

>ana-4.0.37:

/cds/group/psdm/logs/atstart/<year>/<year>_lcls1_jungfrau_dark_proc.txt

/reg/g/psdm/detector/gains/jungfrau/panels/

contains per-panel constants

<repository>/logs/<year>/<time-stamp>_jungfrau_dark_proc_<login>.txt

-o, --dirrepo

replacement for jungfrau_ndarr_dark_proc

intended for per-panel calibration constants.

Dark processing of runs with 3 steps/calib-cycles

jungfrau_deploy_constants

<repository>/logs/<year>_log_jungfrau_deploy_constants.txt

>ana-4.0.37:

/cds/group/psdm/logs/atstart/<year>/<year>_lcls1_jungfrau_deploy_constants.txt

/reg/g/psdm/detector/gains/jungfrau/panels/

<repository>/logs/<year>/<time-stamp>_jungfrau_deploy_constants_<login>.txt

-o, --dirrepoworks after jungfrau_dark_proc to merge and deploy constants under .../<experiment>/calib/...
jungfrau_gain_constants

>ana-4.0.44

/cds/group/psdm/logs/atstart/<year>/<year>_lcls1_jungfrau_gain_constants.txt

/reg/g/psdm/detector/gains/jungfrau/ - reference to dirs with gain constants for panel-ids


/reg/g/psdm/logs/calibman/jungfrau_gain/<year>/<time-stamp>-<experiment>-<run>-<login>.txt

for >ana-4.0.44:

/reg/g/psdm/detector/gains/jungfrau/logs/<year>/<time-stamp>_log_jungfrau_gain_constants_<experiment>-r<run>_<login>.txt

-R, --repdir

DEPRECATED

jungfrau_ndarr_dark_proc



/reg/g/psdm/logs/calibman/<year>/<month>/jungfrau_ndarr_dark_proc.txt - log record  at start
Dark processing and deployment of calibration constants for etire detector. It was DEPRECATED due to new per-panel approach to calibration constants. Replaced by jungfrau_dark_proc and jungfrau_deploy_constants

LCLS2 scripts

scriptrecord at startrepositorylogscontrol parameterscomments
Generic calibration manager
calibman

/cds/group/psdm/detector/logs/atstart/2023/2023_lcls2_calibman.txt

DEPRECATED:

/cds/group/psdm/logs/atstart/2022/2022_logrec_calibman.txt

/cds/group/psdm/detector/calib2/constants/

DEPRECATED: /cds/group/psdm/logs/calibman/lcls2

<repository>/<script-or-dettype>/logs/<year>/<time-stamp>-<script-name>-<login>.txt

e.g. /cds/group/psdm/logs/calibman/lcls2/2022/20220202T081010-calibman-dubrovin.txt

-L, --logdirInteraction with MongoDB etc






epix10ka
epix10ka_pedestals_calibration

/cds/group/psdm/logs/atstart/<year>/<year>_logrec_epix10ka_pedestals_calibration.txt

DEPRECATED:

<repository>/panels/logs/log_pedestals_calibration_<year>.txt

/cds/group/psdm/detector/gains2/epix10ka/panels/

<repository>/logs/<year>/<time-stamp>_log_epix10ka_pedestals_calibration_<login>.txt

e.g. /cds/group/psdm/detector/gains2/epix10ka/panels/logs/2022/2022-02-02T115201_log_epix10ka_pedestals_calibration_dubrovin.txt

-o, --dirrepo
epix10ka_deploy_constants

/cds/group/psdm/logs/atstart/<year>/<year>_logrec_epix10ka_deploy_constants.txt

e.g.

/cds/group/psdm/logs/atstart/2022/2022_logrec_epix10ka_deploy_constants.txt

DEPRECATED:

<repository>/panels/logs/log_deploy_constants_<year>.txt

/cds/group/psdm/detector/gains2/epix10ka/panels/

<repository>/logs/<year>/<time-stamp>_log_epix10ka_deploy_constants_<login>.txt

e.g. /cds/group/psdm/detector/gains2/epix10ka/panels/logs/2022/2022-02-02T120758_log_epix10ka_deploy_constants_dubrovin.txt

-o, --dirrepo






epix100, opal

det_dark_proc


/cds/group/psdm/detector/logs/atstart/2023/2023_lcls2_det_dark_proc.txt

DEPRECATED:

/cds/group/psdm/logs/atstart/<year>/<year>_log_det_dark_proc.txt

/cds/group/psdm/detector/calib2/constants/

<repository>/<script-or-dettype>/logs/<year>/<time-stamp>_log_det_dark_proc_<login>.txt

e.g. /cds/group/psdm/detector/calib2/constants/epix100/logs/2022/2022-02-02T080749_log_det_dark_proc_dubrovin.txt









Tentative logging for LCLS2

Q: is it ok to save logs in repo?

A: if repodir is changed path will be loss along with local location. It would be nice to have record at start in well known static directory. Each record points to the main log file etc.

Logging at start records

Is intended to keep all records at start for all scripts in a single place.

Root directory is intended to be hardwired, e.g. 

/cds/group/psdm/logs/atstart/

Further path

/cds/group/psdm/logs/atstart/<year>/<year>-logrec-<script-name>.txt - single file per year per script.

Repository

Root directory

-o, --dirrepo

/cds/group/psdm/detector/gains2/<detector-type>/panels/

/cds/group/psdm/detector/gains2/<detector-type>/panels/<panel-id>,

where <detector-type> e.g. epix10ka

Logging files

/cds/group/psdm/detector/gains2/<detector-type>/panels/logs/<year>/<time-stamp>-<script-name>-<login>.txt

References

  • No labels