Page History
...
Feature 2 and 3 for dark run
TBDImplemented on 2023-04-06 exactly as described. Median is used for robust statistics parameters evaluation.
Feature 4 and 5 for gain modes
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
...
[I] UtilsRawPixelStatus.py L0240 frame: 0991 res_med: 1.282 res_spr: 110.094 frame residuals shape:(704, 768) size:540672 dtype:float64 [-236.87714286 103.30887218 -49.50511278...]
[I] UtilsRawPixelStatus.py L0240 frame: 0992 res_med: 1.294 res_spr: 110.065 frame residuals shape:(704, 768) size:540672 dtype:float64 [-235.11755952 106.95056078 -54.98131892...]
[I] UtilsRawPixelStatus.py L0240 frame: 0993 res_med: 1.253 res_spr: 110.069 frame residuals shape:(704, 768) size:540672 dtype:float64 [-237.62863095 108.57688283 -45.21760338...]
[I] UtilsRawPixelStatus.py L0240 frame: 0994 res_med: 1.267 res_spr: 110.062 frame residuals shape:(704, 768) size:540672 dtype:float64 [-238.30857143 107.88390977 -47.92360902...]
[I] UtilsRawPixelStatus.py L0240 frame: 0995 res_med: 1.218 res_spr: 110.053 frame residuals shape:(704, 768) size:540672 dtype:float64 [-239.62184524 108.59725251 -37.18364975...]
[I] UtilsRawPixelStatus.py L0240 frame: 0996 res_med: 1.251 res_spr: 110.064 frame residuals shape:(704, 768) size:540672 dtype:float64 [-237.58547619 109.59983709 -43.21484962...]
[I] UtilsRawPixelStatus.py L0240 frame: 0997 res_med: 1.307 res_spr: 110.067 frame residuals shape:(704, 768) size:540672 dtype:float64 [-240.9114881 104.21362469 -50.66126253...]
[I] UtilsRawPixelStatus.py L0240 frame: 0998 res_med: 1.253 res_spr: 110.062 frame residuals shape:(704, 768) size:540672 dtype:float64 [-233.00535714 106.10161028 -47.79142231...]
[I] UtilsRawPixelStatus.py L0240 frame: 0999 res_med: 1.289 res_spr: 110.099 frame residuals shape:(704, 768) size:540672 dtype:float64 [-241.53529762 109.61091792 -48.24286654...]
[I] UtilsRawPixelStatus.py L0265 block of residuals: shape:(998, 704, 768) size:539590656 dtype:float64
[-239.3222619 106.86876566 -48.94020677 48.2508208 -174.55815163
111.63287594 -128.17609649 -21.98506892 -17.79404135 80.39698622
-86.41198622 61.77904135 -116.02993108 118.16109649 -199.64787594
46.54315163 -238.2658208 95.92520677 -148.88376566 16.3072619 ...]
[I] UtilsRawPixelStatus.py L0266 median over frames per-pixel residuals: shape:(704, 768) size:540672 dtype:float64
[-237.54059524 107.78383145 -46.79742951 51.89206767 -174.04665414
109.26853227 -124.40214912 -22.8108537 -14.74468515 80.47136278
-87.92196429 64.17849154 -117.18580984 119.22582237 -198.74469768
49.81222744 -235.53807174 93.36754543 -146.50276942 16.18511905...]
[I] UtilsRawPixelStatus.py L0267 median over frames per-pixel spread of res: shape:(704, 768) size:540672 dtype:float64
[1.91488095 1.90238878 1.80436404 1.79662124 1.58172462 1.79079887
1.86366228 1.83122337 1.75326128 1.94361685 1.74409305 1.90914474
1.7539317 1.96598684 1.92817356 2.05658521 1.84223058 2.07546366
1.7810354 1.90964286...]
[I] UtilsRawPixelStatus.py L0114 ====================
Feat.6 res_med shape:(704, 768) size:540672 dtype:float64 [...]
0 / 540672 ( 0.000%) pixels Feat.6 res_med <= -16383.000
0 / 540672 ( 0.000%) pixels Feat.6 res_med >= 16383.000
Feat.6 res_med selected 540672 of 540672 pixels in range (-16383, 16383) med: 1.225 spr: 110.081
re-defined range for med ± 8.0*spr: (-879.424, 881.874)
0 / 540672 ( 0.000%) pixels Feat.6 res_med <= -879.424
33 / 540672 ( 0.006%) pixels Feat.6 res_med >= 881.874
[I] UtilsRawPixelStatus.py L0114 ====================
Feat.6 res_spr shape:(704, 768) size:540672 dtype:float64 [...]
0 / 540672 ( 0.000%) pixels Feat.6 res_spr <= -16383.000
0 / 540672 ( 0.000%) pixels Feat.6 res_spr >= 16383.000
Feat.6 res_spr selected 540672 of 540672 pixels in range (-16383, 16383) med: 1.788 spr: 0.105
re-defined range for med ± 8.0*spr: (0.950, 2.626)
0 / 540672 ( 0.000%) pixels Feat.6 res_spr <= 0.950
4877 / 540672 ( 0.902%) pixels Feat.6 res_spr >= 2.626
[I] UtilsRawPixelStatus.py L0284
Summary of the bad pixel status evaluation, pixel_status_extra array
0o1: 0 / 540672 ( 0.000%) pixels Feat.6 res_med <= -879.424
0o2: 33 / 540672 ( 0.006%) pixels Feat.6 res_med >= 881.874
0o4: 0 / 540672 ( 0.000%) pixels Feat.6 res_spr <= 0.950
0o10: 4877 / 540672 ( 0.902%) pixels Feat.6 res_spr >= 2.626
Any bad status bit: 4882 / 540672 ( 0.903%) pixels
[I] UtilsCalib.py L0150 saved: pixel_status_extra.dat
[I] UtilsRawPixelStatus.py L0428 Consumed time 1831.462 sec |
...
2023-04-06 Feature 2 and 3 for dark data vs our standard dark processing
- dark data set: exp=xpplw3319:run=287
...
- detector: XppGon.0:Epix100a.3, alias: epix_alc3
- This data set contains ~1200 events, of which initial 1000 is used in processing.
- By default features 1,2,3 are used. Feature 4 and 5 for gain factors are not implemented and are irrelevant to epix100a.
- Feature 1 - good frame selection discards 1/0/0 frames for SNR=3/5/8, respectively
| Code Block | ||||
|---|---|---|---|---|
| ||||
det_raw_pixel_status -d exp=xpplw3319:run=287 -s XppGon.0:Epix100a.3 -n 1000 -R3 --gmode SNR3
Summary of the bad pixel status evaluation for SNR = 3.00, status_data array
0o1: 6445 / 540672 ( 1.192%) pixels Feat.2 mean <= 3588.608
0o2: 13484 / 540672 ( 2.494%) pixels Feat.2 mean >= 4374.797
0o4: 5600 / 540672 ( 1.036%) pixels Feat.3 std <= 2.314
0o10: 34098 / 540672 ( 6.307%) pixels Feat.3 std >= 3.263
Any bad status bit: 57409 / 540672 (10.618%) pixels
det_raw_pixel_status -d exp=xpplw3319:run=287 -s XppGon.0:Epix100a.3 -n 1000 -R5 --gmode SNR5
Summary of the bad pixel status evaluation for SNR=5.00, status_data array
0o1: 5 / 540672 ( 0.001%) pixels Feat.2 mean <= 3326.536
0o2: 87 / 540672 ( 0.016%) pixels Feat.2 mean >= 4636.866
0o4: 878 / 540672 ( 0.162%) pixels Feat.3 std <= 1.999
0o10: 12739 / 540672 ( 2.356%) pixels Feat.3 std >= 3.579
Any bad status bit: 13675 / 540672 ( 2.529%) pixels
det_raw_pixel_status -d exp=xpplw3319:run=287 -s XppGon.0:Epix100a.3 -n 1000 -R8 --gmode SNR8
Summary of the bad pixel status evaluation for SNR=8.00, status_data array
0o1: 0 / 540672 ( 0.000%) pixels Feat.2 mean <= 2933.437
0o2: 33 / 540672 ( 0.006%) pixels Feat.2 mean >= 5029.965
0o4: 0 / 540672 ( 0.000%) pixels Feat.3 std <= 1.525
0o10: 5314 / 540672 ( 0.983%) pixels Feat.3 std >= 4.053
Any bad status bit: 5319 / 540672 ( 0.984%) pixels |
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
det_ndarr_raw_proc -d exp=xpplw3319:run=287 -s XppGon.0:Epix100a.3 -n 1000 -o work raw data found/selected in 1000 events [I] UtilsCalib.py L0683 begin data summary stage [I] UtilsCalib.py L0112 evaluate_limits RMS: ave=2.317 std=0.370 limits low=0.097 high=4.536 [I] UtilsCalib.py L0112 evaluate_limits AVE: ave=3983.565 std=188.830 limits low=2850.584 high=5116.546 [I] UtilsCalib.py L0721 bad pixel status: status 1: 643 pixel rms > 4.536 status 2: 0 pixel rms < 0.097 status 4: 0 pixel intensity > 16000 in more than 0.1 fraction (99/999) of non-empty events status 8: 0 pixel intensity < 1 in more than 0.1 fraction (99/999) of non-empty events status 16: 31 pixel average > 5116.55 status 32: 0 pixel average < 2850.58 |
REFERENCES
2023-04-06 Feature 6 for regular data for SNR=3 and 5
- dark data set: exp=xpplw3319:run=287
- detector: XppGon.0:Epix100a.3, alias: epix_alc3
| Code Block | ||||
|---|---|---|---|---|
| ||||
det_raw_pixel_status -d exp=xpplw3319:run=293 -s XppGon.0:Epix100a.3 -n 1000 -R3 --gmode SNR3-F6 -F 1,6
Summary of the bad pixel status evaluation for SNR=3.00, status_data array
0o1: 449 / 540672 ( 0.083%) pixels Feat.6 res_med <= -330.137
0o2: 379 / 540672 ( 0.070%) pixels Feat.6 res_med >= 331.245
0o4: 8305 / 540672 ( 1.536%) pixels Feat.6 res_spr <= 1.472
0o10: 34625 / 540672 ( 6.404%) pixels Feat.6 res_spr >= 2.095
Any bad status bit: 43648 / 540672 ( 8.073%) pixels
det_raw_pixel_status -d exp=xpplw3319:run=293 -s XppGon.0:Epix100a.3 -n 1000 -R5 --gmode SNR5-F6 -F 1,6
Summary of the bad pixel status evaluation for SNR=5.00, status_data array
0o1: 0 / 540672 ( 0.000%) pixels Feat.6 res_med <= -550.599
0o2: 45 / 540672 ( 0.008%) pixels Feat.6 res_med >= 551.708
0o4: 1005 / 540672 ( 0.186%) pixels Feat.6 res_spr <= 1.265
0o10: 12484 / 540672 ( 2.309%) pixels Feat.6 res_spr >= 2.302
Any bad status bit: 13501 / 540672 ( 2.497%) pixels |
Interim summary
- Number of bad pixels depends on SNR parameter
- Stripe of bad pixels in the middle of the detector can be caught by Feature 3 and 6 at SNR=5.
- "Scratch" observed by Chuck is not seen detected ...
2023-04-07 mtg summary
- Chuck: try to search for scratch in ASIC ONLY!
- Chris: try to change gate limits in dark processing to reproduce bad pixel in the middle stripe.
2023-05-12 low gain pixels
det_calib_ave_and_max -d exp=xpplw3319:run=293 -s XppGon.0:Epix100a.3 -n 100000
generates max-peds for 35998 events of xpplw3319-r0293
img-XppGon-0-Epix100a-3-xpplw3319-r0293-e035998-mode-p-max.npy
use test algorithm test-2023-04-26-chuck.py
from img-XppGon-0-Epix100a-3-xpplw3319-r0293-e035998-mode-p-max.npy
find median and +/- 6 * (median of difference)
max-peds: shape:(704, 768) size:540672 dtype:float32 [135.05005 135.3999 177.94995 180.53003 176.6001 ...]
Summary of the bad pixel status evaluation
0o1: 10021 / 540672 ( 1.853%) pixels max-peds <= 127.788
0o2: 3689 / 540672 ( 0.682%) pixels max-peds >= 203.811
saved arr_status_med-peds.npy
arr_status_med-peds.txt
cp arr_status_med-peds.txt /cds/data/psdm/xpp/xpplw3319/calib/Epix100a::CalibV1/XppGon.0:Epix100a.3/status_extra/0-end.data
see Bad Pixel Status
REFERENCES
- Paper in JAC: Automatic bad-pixel mask maker for X-ray pixel detectors with application to serial crystallography,
- Journal of Applied Crystallography - 2022 - Sadri - Automatic bad‐pixel mask maker for X‐ray pixel detectors with.pdf
- Bad Pixel Status
- Paper in JAC: Automatic bad-pixel mask maker for X-ray pixel detectors with application to serial crystallography,
- Journal of Applied Crystallography - 2022 - Sadri - Automatic bad‐pixel mask maker for X‐ray pixel detectors with.pdf
Overview
Content Tools