...

Results with upgraded algorithm for pedestals calibration

Dark run

...

101 old vs new pedestals comparison

This is a demo of how user might be easily screwed up applying their own algorithm for pedestal calculation.

Five images of the raw-peds for the same 5-step dark run 101 were obtained by command like 

commands:

python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r168 -N100 r101 -Sraw-peds -g2 J1 -o 2021-02-05K500 -N510 -g2 -M2 -o 2021-03-12-figs/r168r101-raw-peds-FL-newpython -old-dark-ev0510-st2-gn2

then old constants were removed from DB with calibman

new constants were generated with new dark processing algorithm and similar images for new pedestals were generated:

lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r168 r101 -N100 Sraw-Scalibcm8 peds -o 2021J1 -02K500 -05N510 -figs/r168-calibcm8-FL-newpython lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r168 -N100 -Scalib -g2 -M2 -o 2021-0203-0512-figs/r168r101-calibraw-FLpeds-new-dark-ev0510-st2-gn2

Images :of raw-

...

peds in pairs for old and new constants for FH, FM, FL, AHL-H, and AML-M:

FH step 0

Image AddedImage Added

FM step 1

Image AddedImage Added

FL step2

Image AddedImage Added

AHL-H step3

Image AddedImage Added

AML-M step 4

Image AddedImage Added

Dark run 101 using pedestals from wrong gain range

Another demo of how user can be screwed-up using pedestals from wrong gain range.

lcls2/psana/

Image RemovedImage RemovedImage RemovedImage Removed

Dark 5-step dark run 134

Old pedestals

...

psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -

...

r101 -Sraw-peds -J1 -K500 -N510 -g0 -M2

...

-o 2021-

...

03-

...

12-figs/

...

r101-raw-peds-new-dark-ev0510-st2-gn0

raw(FL

...

) - peds(FH)

Image Added

...

lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -

...

r101 -

...

Sraw-

...

peds -

...

J1 -K500 -N510 -g2 -M0 -o 2021-

...

03-

...

12-figs/

...

r101-raw-peds-new-dark-ev0510-st1-gn2

raw(FH) - peds(FL)

Image Added

Dark run 168 with single step in FL

commands:

python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -

...

r168 -N100 -

...

Sraw-

...

peds -

...

g2 -o 2021-02-05-figs/

...

r168-

...

raw-

...

peds-

...

FL-

...

new

python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -

...

r168 -N100 -Scalibcm8 -

...

o 2021-02-05-figs/

...

r168-calibcm8-

...

FL-new

python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image

Image RemovedImage RemovedImage RemovedImage Removed

Pedestals calibration and deployment commands

...

-e ueddaq02 -d epixquad -

...

New pedestals

...

r168 -N100 -Scalib -o 2021-02-05-figs/

...

r168-

...

calib-

...

FL-new

Images:

• raw-pedestals(FLpython lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r134 -N100 -Scalib -M2 -o 2021-02-05-figs/r134-calib-new-FL) old
• raw-pedestals(FL) new
• calib
• calib with cmpars = (8,7,50,10) the same works with cmpars = (8,7,10,10) but not for outlying events where correction is larger than 10 ADU...

Image AddedImage AddedImage AddedImage Added

Dark 5-step dark run 134

Old pedestals

• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r134 -N1 N100 -Scalibcm8 Sraw-peds -M2 -g2 -o 2021-02-05-figs/r134-calibcm8raw-7-10peds-10-FL-ev1-new
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r134 -N100 -Scalibcm8 Scalib -M2 -o 2021-02-05-figs/r134-calibcm8-7-10-10-new-FL

Image RemovedImage RemovedImage RemovedImage Removed

Consitency check

Runs 134, 166, 211 cumulative image of 1000 frames of raw-peds descarding events with median intensity 5% in low and high tails:

• calib-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r134 -

...

• N1 -

...

• Scalibcm8 -M2 -o 2021-02-05-figs/r134-calibcm8-7-10-10-FL-ev1
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r134 -N100 -Scalibcm8 -M2 -o 2021-02-05-figs/r134-calibcm8-7-10-10-FL etc...

Image AddedImage AddedImage AddedImage Added

Pedestals calibration and deployment commands

• epix10ka_pedestals_calibration -e ueddaq02 -d epixquad -r134
• epix10ka_deploy_constants -e ueddaq02 -d epixquad -r134 -D
• use calibman to remove old constants for run 134

New pedestals

Image RemovedImage RemovedImage Removed

Mean value of all images is consistent with 0. Visible variation of color causes by variation of intensity spread along surface of the detecor.

1-step FL signal run 137

Old pedestals

• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 r134 -N100 -Sraw-peds -M2 -g2 -o 2021-02-05-figs/r137r134-raw-peds-new-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 r134 -N100 -Scalib -M2 -o 2021-02-05-figs/r137r134-calib-new-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 r134 -N100 N1 -Scalibcm8 -M2 -o 2021-02-05-figs/r137r134-calibcm8-7-10-10-FL-ev1-new
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 r134 -N100 -Scalibcm8 -M2 -o 2021-02-05-figs/r137r134-calibcm8-67-10-10-FL (script edited for cmpars...)python new-FL

Image AddedImage AddedImage AddedImage Added

Consitency check

Runs 134, 166, 211 cumulative image of 1000 frames of raw-peds descarding events with median intensity 5% in low and high tails:

lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -

...

r134 -

...

Sraw-

...

peds -

...

g2 -

...

M2 -

...

J1 -

...

C -

...

-

...

thrmin -

...

0.344 -

...

-thrmax 0.582 --thrpix -10000 -N1000

Image AddedImage AddedImage Added

Mean value of all images is consistent with 0. Visible variation of color causes by variation of intensity spread along surface of the detecor.

1-step FL signal run 137

Old pedestals

• python lcls2/psana/

Image RemovedImage RemovedImage RemovedImage RemovedImage Removed

New pedestals

• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Sraw-peds -g2 -o 2021-02-05-figs/r137-raw-peds-new-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Scalib -o 2021-02-05-figs/r137-calib-new-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Scalibcm8 -o 2021-02-05-figs/r137-calibcm8-7-10-10-new-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Scalibcm8 -o 2021-02-05-figs/r137-calibcm8-6-10-10-new-FL FL (script edited for cmpars...)
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Scalibcm8 -o 2021-02-05-figs/r137-calibcm8-4-10-10-new-FL (script edited for cmpars...)

Image RemovedImage RemovedImage RemovedImage RemovedImage Removed

Threshold 100 keV

Image Removed

cumulative 1000 events raw-peds with pixel intensit threshold 20,50,100ADU:

Image AddedImage AddedImage AddedImage AddedImage Added

New pedestals

• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Sraw-peds -g2 -o 2021-02-05-figs/r137-raw-peds-new-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Scalib -o 2021-02-05-figs/r137-calib-new-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Scalibcm8 -o 2021-02-05-figs/r137-calibcm8-7-10-10-new-FL
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Scalibcm8 -o 2021-02-05-figs/r137-calibcm8-6-10-10-new-FL (script edited for cmpars...)
• python lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r137 -N100 -Scalibcm8 -o 2021-02-05-figs/r137-calibcm8-4-10-10-new-FL (script edited for cmpars...)

Image AddedImage AddedImage AddedImage AddedImage Added

Threshold 100 keV

Image Added

cumulative 1000 events raw-peds with pixel intensit threshold 20,50,100ADU:

Summary for runs 134 and 137

...

Run 167 presumably signal but not so much intensity events 297-300

...

much intensity events 297-300

Commands:

• lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r167 -Sraw-peds -J1 -K295 -N297 -o 2021-03-08-figs/r167-raw-peds-signal-normal-ev0297
• lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r167 -Sraw-peds -J1 -K295 -N297 N298 -o 2021-03-08-figs/r167-raw-peds-signal-normaloutlier-ev0297ev0298
• lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r167 -Sraw-peds -J1 -K295 -N298 N299 -o 2021-03-08-figs/r167-raw-peds-signal-outlier-ev0298ev0299
• lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r167 -Sraw-peds -J1 -K295 -N300 -o 2021-03-08-figs/r167-raw-peds -J1 -K295 -N299 -o 2021-03-08-figs/r167-raw-peds-signal-outlier-ev0299-signal-normal-ev0300

run 167 event 297 - normal, mean ~ 0 ADU

Image AddedImage Added

run 167 event 298 - outlier, mean ~ -25 ADU

Image AddedImage Added

run 167 event 299 - outlier "recovering", mean ~ -4 ADU

Image AddedImage Added

run 167 event 300 - normal, mean ~ 0 ADU

Image AddedImage Added

Dark run 263 vs signal run 264 in FH ueddaq02

For all events plot raw-pedestals  in unusual gain mode FH

Dark run 263

events 200-205, cumulative image of 1000 events, and bad event 94

lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -

...

r263 -Sraw-peds -J1 -K190 -

...

N200 -g0 -

...

M0 -o 2021-03-

...

10-figs/

...

r263-raw-peds-dark-gr0-step0-ev0200

Image AddedImage AddedImage AddedImage AddedImage AddedImage AddedImage AddedImage Added

Signal run 264 in gain mode FH

events 0-6, bad event 0

run 167 event 297 - normal, mean ~ 0 ADU

Image RemovedImage Removed

run 167 event 298 - outlier, mean ~ -25 ADU

Image RemovedImage Removed

run 167 event 299 - outlier "recovering", mean ~ -4 ADU

Image RemovedImage Removed

run 167 event 300 - normal, mean ~ 0 ADU

Image RemovedImage Removed

Dark run 263 vs signal run 264 in FH ueddaq02

For all events plot raw-pedestals  in unusual gain mode FH

Dark run 263

events 200-205, cumulative image of 1000 events, and bad event 94

lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r263 r264 -Sraw-peds -J1 -N6 -K190 -gramin -N200 20 -g0 -M0 gramax 80 -o 2021-03-10-figs/r263r264-raw-peds-dark-gr0-step0-ev0200

Image RemovedImage RemovedImage RemovedImage RemovedImage RemovedImage RemovedImage RemovedImage Removed

Signal run 264 in gain mode FH

signal-ev0006

Image AddedImage AddedImage AddedImage AddedImage AddedImage AddedImage Added

events 300-304, bad event 302events 0-6, bad event 0

lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r264 -Sraw-peds -J1 -K299 -N6 N300 --gramin -20 --gramax 80 -o 2021-03-10-figs/r264-raw-peds-signal-ev0006

Image RemovedImage RemovedImage RemovedImage RemovedImage RemovedImage RemovedImage Removed

events 300-304, bad event 302

lcls2/psana/psana/detector/test_01_epix10ka_raw_calib_image.py image -e ueddaq02 -d epixquad -r264 -Sraw-peds -J1 -K299 -N300 --gramin -20 --gramax 80 -o 2021-03-10-figs/r264-raw-peds-signal-ev0300

Image RemovedImage RemovedImage RemovedImage RemovedImage Removed

Issues for runs 263 and 264 FH

...

1. mean intensity raising during the run, see cumulative image of 1000 events
2. limits on intensity are different comparing to dark processing

...

-o 2021-03-10-figs/r264-raw-peds-signal-ev0300

Image AddedImage AddedImage AddedImage AddedImage Added

Issues for runs 263 and 264 FH

• in 263 mean(raw-peds) <0 for all events ~300
  1. mean intensity raising during the run, see cumulative image of 1000 events
  2. limits on intensity are different comparing to dark processing
• event-by-event variation of the image mean intensity is larger in FH than in FL
• outliers are still there both in dark and signal runs
• in 264 signal is well separated from base level

Correction which solves jitter of the mean value

ueddaq quad shows fluctuation of the baseline-mean image intensity from event to event.

Correction like common mode applied to entire image would solve this problem:

arr = det.raw.calib(evt)

med = np.median(arr)
if fabs(med>thr_med_max): continue # discard deviation of median exceeding some threshold
arr -= med
<here apply per-pixel thresholds etc to arr>

# generate image from arr, if necessary
img = det.raw.image(evt, nda=arr)

...