Versions Compared

Old Version 41

changes.mady.by.user Mikhail Dubrovin

Saved on

compared with

New Version 42

changes.mady.by.user Mikhail Dubrovin

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Examples for Package ImgAlgos

See Package ImgAlgos

Example for Module ImgAlgos::

...

PnccdImageProducer

See Module ImgAlgos::ImgAveragePnccdImageProducer

Example of the psana configuration file:

Code Block
[psana]                                                               
files         = /reg/d/psdm/AMO<INSTRUMENT>/amo42112<experiment>/xtc/e138<file-r0065name-s00-c001>.xtc\
modules    = ImgAlgos.CameraImageProducer \
          ImgAlgos.ImgAverage
events  = 1000/reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/<file-name-2>.xtc
#skip-events = 100
events      =  5
modules = ImgAlgos.PnccdImageProducer ImgAlgos.ImgSaveInFile

[ImgAlgos.CameraImageProducerPnccdImageProducer] 
source          = DetInfo(:Opal1000pnCCD)
key_in inkey         = 
key_out   
outimgkey      = img
subtract_offset = trueimgpnccd
print_bits      = 1

[ImgAlgos.ImgAverageImgSaveInFile] 
source          = DetInfo(:Opal1000pnCCD)
key             = imgimgpnccd
avefilefname         = img-ave.dat
rmsfile         = img-rms.dat
print_bits      = 31
evts_stage1pnccd-img-ev
saveAll       = 100true
evts_stage2#eventSave     = 10082
gateprint_width1 bits    = 200
gate_width2     = 20
1

This script saves text files with images like pnccd-img-ev-<run-date-time.nsec>.txt, which can be presented as:
Image Added

Example for Module ImgAlgos::

...

CameraImageProducer

See Module ImgAlgos::ImgCalib:CameraImageProducer

Example of the psana configuration file:

Code Block
[ImgAlgos.CameraImageProducerpsana] 
source             = DetInfo(:Opal1000)
key_in          = 
key_out         = img
subtract_offset = true
print_bits      = 1

[ImgAlgos.ImgCalib] 
source          = DetInfo(:Opal1000)
key_in       
files           = img
key_out/reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/<file-name-1>.xtc
modules         = calibrated
fname_peds ImgAlgos.CameraImageProducer ImgAlgos.ImgSaveInFile
events      = <pedestal-file-name>
fname_mask      = <mask-file-name>
fname_bkgd   5

[ImgAlgos.CameraImageProducer] 
source   = 
fname_gain      = DetInfo(:Opal1000)
printkey_bitsin      = 31

Example of the mask file and resulting image:
Image RemovedImage Removed

Example for Module ImgAlgos::ImgRadialCorrection

See Module ImgAlgos::ImgRadialCorrection

Code Block

[psana]
files = /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
    = 
key_out         /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-2>.xtc \
    = img
subtract_offset = true
print_bits    ...
        /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc
skip-events = 500
events= 15

[ImgAlgos.ImgSaveInFile]
source        = 10
modules = cspad_mod.CsPadCalib \
#DetInfo(:Opal1000)
key           = ImgAlgos.CSPadBkgdSubtract \img
fname          CSPadPixCoords.CSPadImageProducer \
= img-from-my-experiment
saveAll          ImgAlgos.ImgRadialCorrection \
= true
#eventSave          ImgAlgos.ImgSaveInFile:1
 

[cspad_mod.CsPadCalib]
inputKey= 1

This script saves text files with images like img-from-my-experiment-<run-date-time.nsec>.txt, which can be drawn by the python script

./ImgAlgos/data/PlotCameraImageFromFile.py <filename>.txt <Amin> <Amax>

Example for Module ImgAlgos::ImgAverage

See Module ImgAlgos::ImgAverage

Code Block

[psana]      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadBkgdSubtract]
source           = DetInfo(CxiDs1.0:Cspad.0)
inputKey      = calibrated
outputKey     = bkgd_subtracted_arr
bkgd_fname    = <the-file-name-with-background-array>
norm_sector   = 0
print_bits    = 0

[CSPadPixCoords.CSPadImageProducer]
calibDir   
files   = /reg/d/psdm/<instrument>/<experiment>/calib
typeGroupName = CsPad::CalibV1
source/xtc/<file-name>.xtc
modules = ImgAlgos.CameraImageProducer \
         = CxiDs1.0:Cspad.0
keyImgAlgos.ImgAverage
events  = 1000

[ImgAlgos.CameraImageProducer] 
source          = DetInfo(:Opal1000)
key_in           = calibrated
imgkeykey_out         = current_img
#tiltIsAppliedsubtract_offset = true
print_bits      = 1

[ImgAlgos.ImgRadialCorrectionImgAverage] 
source          = DetInfo(CxiDs1.0:Cspad.0:Opal1000)
inkeykey             = current_img
outkeyavefile         = r_cor_img
xcenterimg-ave.dat
rmsfile         = 866
ycenterimg-rms.dat
print_bits       = 857
rmin  31
evts_stage1        = 100
rmax     evts_stage2     = 810100
n_phi_bins    = 60
eventgate_width1         = 0200
printgate_bitswidth2     = 3
20

Example for Module ImgAlgos::ImgCalib

See Module ImgAlgos::ImgCalib

Code Block
[ImgAlgos.ImgSaveInFile:1CameraImageProducer] 
source          = CxiDs1.0:Cspad.0
keyDetInfo(:Opal1000)
key_in          = 
key_out         = img
subtract_offset = r_cor_img
fnametrue
print_bits      = 1

[ImgAlgos.ImgCalib] 
source          = <file-name-for-image-array>
#saveAllDetInfo(:Opal1000)
key_in          = true
eventSaveimg
key_out         = 8

Note: the option of the background subtraction (ImgAlgos.CSPadBkgdSubtract) is commented out in this configuration file . In order to evoke this option, the comment sign (#) should be removed from the list of modules and the key=bkgd_subtracted_arr should be used in CSPadPixCoords.CSPadImageProducer.

Calibrated image and spectrum:
Image RemovedImage Removed

Calibrated and radial-corrected image, spectrum, and subtracted r-phi65 distribution for n_phi_bins=65:
Image RemovedImage RemovedImage Removed

calibrated
fname_peds      = <pedestal-file-name>
fname_mask      = <mask-file-name>
fname_bkgd      = 
fname_gain      = 
print_bits      = 31

Example of the mask file and resulting image:
Image AddedImage AddedCalibrated and radial-corrected image, spectrum, and subtracted r-phi12 distribution for n_phi_bins=12:
Image RemovedImage RemovedImage Removed

Example for Module ImgAlgos::

...

ImgRadialCorrection

See Module ImgAlgos::ImgPeakFinderConfiguration file example:ImgRadialCorrection

Code Block
[psana]
files                                                               
files         = /reg/= /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
                /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-2>.xtc \
                ...
                /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc

modules skip-events = 500
events      = ImgAlgos.ImgPeakFinder PrintSeparator

#skip-events  = 500
events10
modules = cspad_mod.CsPadCalib \
#        = 120

[ImgAlgos.ImgPeakFinder]CSPadBkgdSubtract \
source          = DetInfo(:Opal1000)
keyCSPadPixCoords.CSPadImageProducer \
          ImgAlgos.ImgRadialCorrection \
 = 
peaksKey       =  peaks
threshold_low  =  20
threshold_high =  50
sigmaImgAlgos.ImgSaveInFile:1
 

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals    =  1.5
smear_radius  yes
doPixelStatus =  2
peak_radius  no
doCommonMode  =  3
xmin   no

[ImgAlgos.CSPadBkgdSubtract]
source        =  200
xmax    DetInfo(CxiDs1.0:Cspad.0)
inputKey       =  800calibrated
yminoutputKey       = bkgd_subtracted_arr
bkgd_fname    =  100
ymax<the-file-name-with-background-array>
norm_sector       = 0
print_bits    =  900
testEvent0

[CSPadPixCoords.CSPadImageProducer]
calibDir      =  115
print_bits/reg/d/psdm/<instrument>/<experiment>/calib
typeGroupName = CsPad::CalibV1
source        =  0
finderIsOn CxiDs1.0:Cspad.0
key           = calibrated
imgkey        = current_img
#tiltIsApplied = true
  • This algorithm consumes ~15 ms/event on psana0101 for full Opal1000 (1024x1024) camera image.
  • Smearing algorithm use a "safety margin" which is currently set to 10 pixels (offset from each boarder of the full image size).

Image on different stages of this algorithm:
Image RemovedImage RemovedImage RemovedImage RemovedImage Removed

  1. raw image,
  2. image in the window with amplitudes above the threshold_low
    • few peaks at the edges were discarded by the window limits,
    • image still contains many 1-photon pixels, which need to be eliminated,
  3. smeared image,
  4. raw image with found peaks (marked by the red circles)
  5. zoom of the previous plot.

Example for Module ImgAlgos::ImgPeakFilter

Code Block

[ImgAlgos.ImgPeakFilter] 
source  
[ImgAlgos.ImgRadialCorrection]
source        = DetInfo(CxiDs1.0:Cspad.0)
inkey         = current_img
outkey        = r_cor_img
xcenter       = DetInfo(:Opal1000)
key866
ycenter       = 857
rmin        = peaks
threshold_peak = 5
threshold_total= 0100
rmax          = 810
n_peaks_minphi_bins    = 60
event         = 100
print_bits    = 3

[ImgAlgos.ImgSaveInFile:1]
source        = 11
fnameCxiDs1.0:Cspad.0
key           = img
selection_mode = SELECTION_ON

Example for Module ImgPeakFinder and ImgPeakFilter for CSPad

Module ImgAlgos::ImgPeakFinder works on image. In order to apply this algorithm to CSPad the image should be produced. In next example the image is produced using consequtive modules cspad_mod.CsPadCalib, ImgAlgos.CSPadMaskApply, and CSPadPixCoords.CSPadImageProducer:

Code Block

[psana]
files   = /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
   r_cor_img
fname         = <file-name-for-image-array>
#saveAll       = true
eventSave     = 8

Note: the option of the background subtraction (ImgAlgos.CSPadBkgdSubtract) is commented out in this configuration file . In order to evoke this option, the comment sign (#) should be removed from the list of modules and the key=bkgd_subtracted_arr should be used in CSPadPixCoords.CSPadImageProducer.

Calibrated image and spectrum:
Image AddedImage Added

Calibrated and radial-corrected image, spectrum, and subtracted r-phi65 distribution for n_phi_bins=65:
Image AddedImage AddedImage Added

Calibrated and radial-corrected image, spectrum, and subtracted r-phi12 distribution for n_phi_bins=12:
Image AddedImage AddedImage Added

Example for Module ImgAlgos::ImgPeakFinder

See Module ImgAlgos::ImgPeakFinder

Configuration file example:

Code Block

[psana]               ...
          /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc

events      =  1000
experiment  = cxii0212
calib-dir   = ana-cxii0212/calib

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadMaskApply \
          CSPadPixCoords.CSPadImageProducer \
          ImgAlgos.ImgPeakFinder \
          ImgAlgos.ImgPeakFilter \
       
files   ImgAlgos.ImgSaveInFile:1

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey/reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
      = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = yes

[ImgAlgos.CSPadMaskApply]
source /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-2>.xtc \
               = DetInfo(CxiDs1.0:Cspad.0)
inkey...
               = calibrated
outkey /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc

modules       =   ImgAlgos.ImgPeakFinder PrintSeparator

#skip-events  = masked_arr
mask_fname500
events        = <your-local-directory>/<mask-file-name>.dat
masked_amp120

[ImgAlgos.ImgPeakFinder] 
source         =  0
print_bitsDetInfo(:Opal1000)
key         =  5
mask_control_bits = 15

[CSPadPixCoords.CSPadImageProducer]
calibDir
peaksKey       = /reg/d/psdm/<instrument>/<experiment>/calib
typeGroupName  peaks
threshold_low  =  CsPad::CalibV1
source   20
threshold_high =  50
sigma      = CxiDs1.0:Cspad.0
key   =  1.5
smear_radius   =  2
peak_radius    = masked_arr
imgkey 3
xmin           =  img
print_bits200
xmax       =  0
#tiltIsApplied  =  true

[ImgAlgos.ImgPeakFinder] 
source800
ymin           = DetInfo(CxiDs1.0:Cspad.0)
key 100
ymax           = = img900
peaksKeytestEvent       = peaks
threshold_low  = 2
threshold_high = 5
sigma 115
print_bits     =  0
finderIsOn     = 1.5
smear_radius   = 5
peak_radius    = 7
xmin  true
  • This algorithm consumes ~15 ms/event on psana0101 for full Opal1000 (1024x1024) camera image.
  • Smearing algorithm use a "safety margin" which is currently set to 10 pixels (offset from each boarder of the full image size).

Image on different stages of this algorithm:
Image AddedImage AddedImage AddedImage AddedImage Added

  1. raw image,
  2. image in the window with amplitudes above the threshold_low
    • few peaks at the edges were discarded by the window limits,
    • image still contains many 1-photon pixels, which need to be eliminated,
  3. smeared image,
  4. raw image with found peaks (marked by the red circles)
  5. zoom of the previous plot.

Example for Module ImgAlgos::ImgPeakFilter

Code Block

[ImgAlgos.ImgPeakFilter] 
source         = 20
xmaxDetInfo(:Opal1000)
key            = 1700peaks
yminthreshold_peak = 5
threshold_total= 0
n_peaks_min    = 10
print_bits   = 20
ymax           = 170011
#testEvent  fname    = 5
print_bits     = 3
#finderIsOn    img
selection_mode = SELECTION_ON

Example for Module ImgPeakFinder and ImgPeakFilter for CSPad

Module ImgAlgos::ImgPeakFinder works on image. In order to apply this algorithm to CSPad the image should be produced. In next example the image is produced using consequtive modules cspad_mod.CsPadCalib, ImgAlgos.CSPadMaskApply, and CSPadPixCoords.CSPadImageProducer:

Code Block

[psana]
files   = /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
          ...
true

[ImgAlgos.ImgPeakFilter] 
source         = DetInfo(CxiDs1.0:Cspad.0)
key            = peaks
threshold_peak = 5
threshold_total= 0
n_peaks_min    = 10
print_bits     = 11
fname          = cspad-img
selection_mode = SELECTION_ON

[ImgAlgos.ImgSaveInFile:1]
source         = CxiDs1.0:Cspad.0
key/reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc

events      =  1000
experiment  = cxii0212
calib-dir   = ana-cxii0212/calib

modules = cspad_mod.CsPadCalib \
            = img
fnameImgAlgos.CSPadMaskApply \
          = cspad-img
#eventSaveCSPadPixCoords.CSPadImageProducer \
      = 1
saveAll    ImgAlgos.ImgPeakFinder \
         = true

Example for Module ImgAlgos::ImgSaveInFile

Code Block

modules = ...ImgAlgos.ImgPeakFilter \
          ImgAlgos.ImgSaveInFile:1 ...

[ImgAlgos.ImgSaveInFile:1]
sourcecspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = DetInfo(:Opal1000)   # or CxiDs1.0:Cspad.0
keyyes
doPixelStatus = no
doCommonMode  = yes

[ImgAlgos.CSPadMaskApply]
source            = img
fnameDetInfo(CxiDs1.0:Cspad.0)
inkey             = my-img
#eventSavecalibrated
outkey      = 5
saveAll     = masked_arr
mask_fname  = true

See Module ImgAlgos::ImgSaveInFile

Example for Module ImgAlgos::CSPadArrAverage

See Module ImgAlgos::CSPadArrAverage

Configuration file example for evaluation of pedestals:

Code Block

[psana]
modules = ImgAlgos.CSPadArrAverage
files   = <path<your-tolocal-thedirectory>/<mask-darkfile-run-file>name>.xtc

[ImgAlgos.CSPadArrAverage]
sourcedat
masked_amp  = DetInfo(CxiDs1.0:Cspad.0)
key     =
avefile = cspad-pedestals-ave.dat
rmsfile = cspad-pedestals-rms.dat 0
print_bits  = 15
evts_stage1 = 100
evts_stage2 = 100
gate_width1 = 100
gate_width2 5
mask_control_bits =  10

Configuration file example for evaluation of background:

Code Block

[psana]
files15

[CSPadPixCoords.CSPadImageProducer]
calibDir       = <path-to-the-background-run-file>.xtc
modules/reg/d/psdm/<instrument>/<experiment>/calib
typeGroupName  = CsPad::CalibV1
source         = cspad_mod.CsPadCalib ImgAlgos.CSPadArrAverage
skip-events = 500
eventsCxiDs1.0:Cspad.0
key            = 1000000

[cspad_mod.CsPadCalib]
inputKey  masked_arr
imgkey    = 
outputKey     = calibratedimg
doPedestalsprint_bits   = yes
doPixelStatus =  no0
doCommonMode#tiltIsApplied  = notrue

[ImgAlgos.CSPadArrAverageImgPeakFinder] 
source         = DetInfo(CxiDs1.0:Cspad.0)
key            = calibrated
avefile = cspad-background-ave.dat
rmsfile = cspad-background-rms.dat
print_bits  = 15

Images of the CSPad arrays for averaged and rms values, respectively, in one of the CXI runs:
Image RemovedImage Removed

Example for Module ImgAlgos::CSPadBkgdSubtract

See Module ImgAlgos::CSPadBkgdSubtract

Code Block

[psana]
files = /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
 img
peaksKey       = peaks
threshold_low  = 2
threshold_high = 5
sigma          = 1.5
smear_radius   = 5
peak_radius    = 7
xmin           /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-2>.xtc \
= 20
xmax           = ...1700
ymin        /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc
skip-events   = 50020
events ymax     = 10
modules     = cspad_mod.CsPadCalib ImgAlgos.CSPadBkgdSubtract

[cspad_mod.CsPadCalib]
inputKey1700
#testEvent      = 5
outputKeyprint_bits     = calibrated3
doPedestals#finderIsOn   = yes
doPixelStatus = no
doCommonMode  = no

true

[ImgAlgos.CSPadBkgdSubtractImgPeakFilter] 
source         = DetInfo(CxiDs1.0:Cspad.0)
inputKeykey      = calibrated
outputKey     = bkgd_subtractedpeaks
bkgdthreshold_fname peak   = <the-file-name-with-background-array>
norm_sector= 5
threshold_total= 0
n_peaks_min    = 010
print_bits     = 3

The file with the background array, bkgd_fname, was obtained by averaging 1000 events using module CSPadArrAverage. Subtraction is done with normalization for norm_sector=0.
Event image and pixel amplitude spectrum before and after the background subtraction are shown in plots:
Image RemovedImage Removed
Image RemovedImage Removed

Other event with better subtracted background:
Image RemovedImage Removed

Example for Module ImgAlgos::CSPadMaskApply

See Module ImgAlgos::CSPadMaskApply

The array for mask contains zeros and ones for masked and passed pixels, respectively, and has a shape of full-size CSPad array 4*8*185388.
For example, it can be generated by the command

Code Block

./MakePixelMask.py <input-background-cspad-arr-file-name> <threshold> <output-file-name>

for the averaged background amplitude array <input-background-cspad-arr-file-name> abtained as a result of ImgAlgos::CSPadArrAverage module.

Plots show the averaged background, and the mask arrays generated from this background for three thresholds 10, 20, and 30 EDU:
Image RemovedImage RemovedImage RemovedImage Removed

The best results in filtering can be achieved in combination of modiles:

11
fname          = cspad-img
selection_mode = SELECTION_ON

[ImgAlgos.ImgSaveInFile:1]
source         = CxiDs1.0:Cspad.0
key            = img
fname          = cspad-img
#eventSave     = 1
saveAll        = true

Example for Module ImgAlgos::ImgSaveInFile

Code Block

modules = ... ImgAlgos.ImgSaveInFile:1 ...

[ImgAlgos.ImgSaveInFile:1]
source         = DetInfo(:Opal1000)   # or CxiDs1.0:Cspad.0
key            = img
fname          = my-img
#eventSave     = 5
saveAll        = true

See Module ImgAlgos::ImgSaveInFile

Example for Module ImgAlgos::CSPadArrAverage

See Module ImgAlgos::CSPadArrAverage

Configuration file example for evaluation of pedestals:

Code Block

[psana]
modules = ImgAlgos.CSPadArrAverage
files   = <path-to-the-dark-run-file>.xtc

[ImgAlgos.CSPadArrAverage]
source  = DetInfo(CxiDs1.0:Cspad.0)
key     =
avefile = cspad-pedestals-ave.dat
rmsfile = cspad-pedestals-rms.dat
print_bits  = 15
evts_stage1 = 100
evts_stage2 = 100
gate_width1 = 100
gate_width2 =  10

Configuration file example for evaluation of background:

Code Block

[psana]
files       = <path-to-the-background-run-file>.xtc
modules     = cspad_mod.CsPadCalib ImgAlgos.CSPadArrAverage
skip-events = 500
events      = 1000000

[cspad_mod.CsPadCalib]
Code Block

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadBkgdSubtract \
          ImgAlgos.CSPadMaskApply \
          ...
[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated_arr
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadBkgdSubtract]
source        = DetInfo(CxiDs1.0:Cspad.0)
inputKey      = calibrated_arr
outputKey     = bkgd_subtracted_arr
bkgd_fname calibrated
doPedestals   = ana-cxi49012/cspad-cxi49012-r0025-background-ave.dat
norm_sector   = 0
print_bits  yes
doPixelStatus = no
doCommonMode  = 0no

[ImgAlgos.CSPadMaskApplyCSPadArrAverage]
source            = DetInfo(CxiDs1.0:Cspad.0)
inkeykey     = calibrated
avefile = cspad-background-ave.dat
rmsfile   = cspad-background-rms.dat
print_bits  = bkgd_subtracted_arr
outkey       15

Images of the CSPad arrays for averaged and rms values, respectively, in one of the CXI runs:
Image AddedImage Added

Example for Module ImgAlgos::CSPadBkgdSubtract

See Module ImgAlgos::CSPadBkgdSubtract

Code Block

[psana]
files = /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
     = masked_arr
mask_fname     /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-2>.xtc \
     = ana-cxi49012/cspad-cxi49012-r0025-mask-40.dat
masked_amp   ...
        /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc
skip-events =  0
print_bits500
events      = 10
modules  =  3
mask_control_bits =  1

where

cspad_mod.CsPadCalib

...

 ImgAlgos.CSPadBkgdSubtract

...



[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no


[ImgAlgos.CSPadBkgdSubtract]
source        = DetInfo(CxiDs1.0:Cspad.0)
inputKey      = calibrated
outputKey     = bkgd_subtracted
bkgd_fname    = <the-file-name-with-background-array>
norm_sector   = 0
print_bits    = 3

The file with the background array, bkgd_fname, was obtained by averaging 1000 events using module CSPadArrAverage. Subtraction is done with normalization for norm_sector=0.
Event image and pixel amplitude spectrum before and after the background subtraction are shown in plots:
Image AddedImage Added
Image AddedImage Added

Other event with better subtracted background:
Image AddedImage Added

In the test with images for background represented by the water and solvent rings this filter provides the background suppression factor about 100.
The background images that still pass this filter have significantly larger intensity with respect to averaged background:
Image RemovedImage RemovedImage Removed

Input parameter mask_control_bits allows to control masking regions of 2x1. For example, if all edges need to be masked, then use mask_control_bits = 15, which gives image array like:
Image Removed
where red regions/lines of pixels of amplitude=8 are masked.

Example for Module ImgAlgos::

...

CSPadMaskApply

See Module ImgAlgos::CSPadArrNoise

Code Block

[psana]
modules = cspad_mod.CsPadCalib ImgAlgos.CSPadArrNoise

files   = /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s00-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s01-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s02-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s03-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s04-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s05-c00.xtc

#skip-events = 1000
events      = 10

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadArrNoise]
source          = DetInfo(CxiDs1.0:Cspad.0)
key             = calibrated
statusfile      = ana-cxi49012/cspad-cxi49012-r0020-noise-status.dat
maskfile        = ana-cxi49012/cspad-cxi49012-r0200-noise-mask.dat
print_bits      = 255
rmin            = 3
dr              = 1
SoNThr          = 3
frac_noisy_imgs = 0.15

Index map in median algorithm for rmin=3, dr=1:

Code Block

CSPadArrNoise::printMatrixOfIndexesForMedian():
 0 0 0 0 1 0 0 0 0
 0 0 1 1 1 1 1 0 0
 0 1 0 0 0 0 0 1 0
 0 1 0 0 0 0 0 1 0
 1 1 0 0 + 0 0 1 1
 0 1 0 0 0 0 0 1 0
 0 1 0 0 0 0 0 1 0
 0 0 1 1 1 1 1 0 0
 0 0 0 0 1 0 0 0 0

Pixel status (fraction of events where S/N > SoNThr):
Image Removed Image Removed Image Removed
For cspad-cxi49012-r0020 with parameters from confguration file (frac_noisy_imgs=0.15) we get, depending on number of events:

Nnoisy, Ntotal, Nnoisy/Ntotal pixels =94585 2296960 0.041 for 10 events
Nnoisy, Ntotal, Nnoisy/Ntotal pixels =2112 2296960 0.00092 for 100 events

Pixel mask for noisy pixels with |S/N| > SoNThr:
Image Removed Image Removed

Example for Module ImgAlgos::CSPadArrPeakFinder

See Module ImgAlgos::CSPadArrPeakFinder

CSPadMaskApply

The array for mask contains zeros and ones for masked and passed pixels, respectively, and has a shape of full-size CSPad array 4*8*185388.
For example, it can be generated by the command

Code Block

./MakePixelMask.py <input-background-cspad-arr-file-name> <threshold> <output-file-name>

for the averaged background amplitude array <input-background-cspad-arr-file-name> abtained as a result of ImgAlgos::CSPadArrAverage module.

Plots show the averaged background, and the mask arrays generated from this background for three thresholds 10, 20, and 30 EDU:
Image AddedImage AddedImage AddedImage Added

The best results in filtering can be achieved in combination of modiles:

Code Block

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadBkgdSubtract \
          ImgAlgos.CSPadMaskApply \
          ...
[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated_arr
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadBkgdSubtract]
source        = DetInfo(CxiDs1.0:Cspad.0)
inputKey      = calibrated_arr
outputKey     = bkgd_subtracted_arr
bkgd_fname    = ana-cxi49012/cspad-cxi49012-r0025-background-ave.dat
norm_sector   = 0
print_bits    = 0

[ImgAlgos.CSPadMaskApply]
source            = DetInfo(CxiDs1.0:Cspad.0)
inkey             = bkgd_subtracted_arr
outkey            = masked_arr
mask_fname        = ana-cxi49012/cspad-cxi49012-r0025-mask-40.dat
masked_amp        =  0
print_bits        =  3
mask_control_bits =  1

where

  • cspad_mod.CsPadCalib - subtracts the pedestals from raw CSPad data,
  • ImgAlgos.CSPadBkgdSubtract - subtracts the background,
  • ImgAlgos.CSPadMaskApply - apply the mask.

In the test with images for background represented by the water and solvent rings this filter provides the background suppression factor about 100.
The background images that still pass this filter have significantly larger intensity with respect to averaged background:
Image AddedImage AddedImage Added

Input parameter mask_control_bits allows to control masking regions of 2x1. For example, if all edges need to be masked, then use mask_control_bits = 15, which gives image array like:
Image Added
where red regions/lines of pixels of amplitude=8 are masked.

Example for Module ImgAlgos::CSPadArrNoise

See Module ImgAlgos::CSPadArrNoise

Code Block

[psana]
modules = cspad_mod.CsPadCalib ImgAlgos.CSPadArrNoise

files   = /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s00-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s01-c00.xtc \
Code Block

[psana]
files   = \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s00-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s01-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s02-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s03-c00.xtc \
#   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s04-c00.xtc \ suddenly it became unavailable...
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s05-c00.xtc

#skip-events = 1000
#events      = 200

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadMaskApply \
          ImgAlgos.CSPadArrPeakFinder

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadMaskApply]
source            = DetInfo(CxiDs1.0:Cspad.0)
inkey             = calibrated
outkey/reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s02-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s03-c00.xtc \
 = masked_arr
mask_fname        = ana-cxi49012/cspad-cxi49012-r0150-mask-badregs.dat
#mask_fname/reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s04-c00.xtc \
         = ana-cxi49012/cspad-cxi49012-r0150-mask-bkgd.dat
#mask_fname   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s05-c00.xtc

#skip-events = 1000
events      = ana-cxi49012/cspad-cxi49012-r0150-mask-rects.dat
masked_amp10

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey =  8
print_bits  = calibrated
doPedestals   = yes
doPixelStatus =  1
mask_control_bitsno
doCommonMode  = 15no

[ImgAlgos.CSPadArrPeakFinderCSPadArrNoise]
source            = DetInfo(CxiDs1.0:Cspad.0)
key             = calibrated
statusfile = masked_arr
key_peaks_out     = ana-cxi49012/cspad-cxi49012-r0020-noise-status.dat
maskfile      = peaks

hot_pix_mask_inp_file = ana-cxi49012/cspad-cxi49012-r0150r0200-noise-mask.dat
hot_pix_mask_out_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask-out.dat
frac_noisy_evts_file  = ana-cxi49012/cspad-cxi49012-r0150-noise-frac.dat

evt_file_out      = tmp/cspad-ev-

rmin              =     3
dr                =     1
SoNThr            =     3
frac_noisy_imgs   =   0.1

peak_npix_min     =     4
peak_npix_max     =    25
peak_amp_tot_thr  =   100.

event_npeak_min   =    10
event_amp_tot_thr =  1000.

nevents_mask_update = 100
nevents_mask_accum  =  50

selection_mode    = SELECTION_ON
out_file_bits     =    15
print_bits        =    512

Results:

Code Block

[info:TimeInterval::startTime] Start time: 2012-06-12 15:32:02
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 1000  N selected = 55  Fraction of selected = 0.055
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 2000  N selected = 62  Fraction of selected = 0.031
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 3000  N selected = 81  Fraction of selected = 0.027
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 4000  N selected = 95  Fraction of selected = 0.02375
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 5000  N selected = 150  Fraction of selected = 0.03
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 6000  N selected = 265  Fraction of selected = 0.0441667
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 7000  N selected = 404  Fraction of selected = 0.0577143
[info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY =====
[info:TimeInterval::stopTime] Time to process 7945 events is 3747.48 sec, or 0.471678 sec/event

Selected events

ev-007713:
Image RemovedImage RemovedImage Removed

ev-008944:
Image RemovedImage RemovedImage Removed

Example for TimeStampFilter and XtcOutputModule

This example demonstrates how to run psana with the time stamp filter and event writer in xtc file.
Both modules are available in psana library and they need only to be described in the configuration file. For example, the configuration file tstamp-filter-and-event-writer.cfg may looks like:

print_bits      = 255
rmin            = 3
dr              = 1
SoNThr          = 3
frac_noisy_imgs = 0.15

Index map in median algorithm for rmin=3, dr=1:

Code Block

CSPadArrNoise::printMatrixOfIndexesForMedian():
 0 0 0 0 1 0 0 0 0
 0 0 1 1 1 1 1 0 0
 0 1 0 0 0 0 0 1 0
 0 1 0 0 0 0 0 1 0
 1 1 0 0 + 0 0 1 1
 0 1 0 0 0 0 0 1 0
 0 1 0 0 0 0 0 1 0
 0 0 1 1 1 1 1 0 0
 0 0 0 0 1 0 0 0 0

Pixel status (fraction of events where S/N > SoNThr):
Image Added Image Added Image Added
For cspad-cxi49012-r0020 with parameters from confguration file (frac_noisy_imgs=0.15) we get, depending on number of events:

Nnoisy, Ntotal, Nnoisy/Ntotal pixels =94585 2296960 0.041 for 10 events
Nnoisy, Ntotal, Nnoisy/Ntotal pixels =2112 2296960 0.00092 for 100 events

Pixel mask for noisy pixels with |S/N| > SoNThr:
Image Added Image Added

Example for Module ImgAlgos::CSPadArrPeakFinder

See Module ImgAlgos::CSPadArrPeakFinder

Code Block

[psana]
files   = \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s00-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s01-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s02-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s03-c00.xtc \
#   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s04-c00.xtc \ suddenly it became unavailable...
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s05-c00.xtc

#skip-events = 1000
#events      = 200

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadMaskApply \
          ImgAlgos.CSPadArrPeakFinder

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadMaskApply]
source            = DetInfo(CxiDs1.0:Cspad.0)
inkey             = calibrated
outkey            = masked_arr
mask_fname        = ana-cxi49012/cspad-cxi49012-r0150-mask-badregs.dat
#mask_fname        = ana-cxi49012/cspad-cxi49012-r0150-mask-bkgd.dat
#mask_fname        = ana-cxi49012/cspad-cxi49012-r0150-mask-rects.dat
masked_amp        =  8
print_bits        =  1
mask_control_bits = 15

[ImgAlgos.CSPadArrPeakFinder]
source            = DetInfo(CxiDs1.0:Cspad.0)
key               = masked_arr
key_peaks_out     = peaks

hot_pix_mask_inp_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask.dat
hot_pix_mask_out_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask-out.dat
frac_noisy_evts_file  = ana-cxi49012/cspad-cxi49012-r0150-noise-frac.dat

evt_file_out      = tmp/cspad-ev-

rmin
Code Block

[psana]                                   =     3
dr                =       1
filesSoNThr         = /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s00-c00.xtc \
 =     3
frac_noisy_imgs   =   0.1

peak_npix_min    /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s01-c00.xtc \
 =       4
peak_npix_max     =    /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s02-c00.xtc \
                /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s03-c00.xtc \
                /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s04-c00.xtc \
                /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s05-c00.xtc

skip-events   = 10
events        = 100
modules       = ImgAlgos.TimeStampFilter  PSXtcOutput.XtcOutputModule 

[PSXtcOutput.XtcOutputModule]
dirName       = ./test_out

[ImgAlgos.TimeStampFilter]
tsinterval    = 2012-02-02 18:17:00.409143728 / 2012-02-02 18:17:00.525853474
filterIsOn    = true
print_bits    = 11

Command to run psana is:
psana -c ./tstamp-filter-and-event-writer.cfg
For this configuration file psana will skip 10 events and loop over the next 100 events from the /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s0*-c00.xtc files and run consecutively modules = ImgAlgos.TimeStampFilter PSXtcOutput.XtcOutputModule. Parameters of these modules are described in the bottom part of the configuration file. Module TimeStampFilter passes events from the specified time interval and prints some useful information. Module XtcOutputModule will write passed events in the file with auto-generated name ./test_out/e158-r0021.xtcf.

See also: Module ImgAlgos::TimeStampFilter and PSXtcOutput::PSXtcOutput

Example for Module ImgAlgos::CSPadArrPeakAnalysis

See Module ImgAlgos::CSPadArrPeakAnalysis

Example of the psana configuration file:

Code Block

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadMaskApply \
          ImgAlgos.CSPadArrPeakFinder \
          ImgAlgos.CSPadArrPeakAnalysis

# ...configuration parameters of other modules...

[ImgAlgos.CSPadArrPeakAnalysis]
source             = DetInfo(CxiDs1.0:Cspad.0)
key                = peaks
print_bits         = 7
fname_root         = file.root

After execution in psana the file.root containing histogram(s) and ntuple(s) will be produced. Then, auxiliary script in root, running by the commend
root -q -f proc.C
produces the plots with histograms:

Image RemovedImage Removed

Example for Module ImgAlgos::PnccdImageProducer

See Module ImgAlgos::PnccdImageProducer

25
peak_amp_tot_thr  =   100.

event_npeak_min   =    10
event_amp_tot_thr =  1000.

nevents_mask_update = 100
nevents_mask_accum  =  50

selection_mode    = SELECTION_ON
out_file_bits     =    15
print_bits        =    512

Results:

Code Block

[info:TimeInterval::startTime] Start time: 2012-06-12 15:32:02
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 1000  N selected = 55  Fraction of selected = 0.055
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 2000  N selected = 62  Fraction of selected = 0.031
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 3000  N selected = 81  Fraction of selected = 0.027
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 4000  N selected = 95  Fraction of selected = 0.02375
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 5000  N selected = 150  Fraction of selected = 0.03
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 6000  N selected = 265  Fraction of selected = 0.0441667
[info:ImgAlgos.CSPadArrPeakFinder]   N processed events = 7000  N selected = 404  Fraction of selected = 0.0577143
[info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY =====
[info:TimeInterval::stopTime] Time to process 7945 events is 3747.48 sec, or 0.471678 sec/event

Selected events

ev-007713:
Image AddedImage AddedImage Added

ev-008944:
Image AddedImage AddedImage Added

Example for TimeStampFilter and XtcOutputModule

This example demonstrates how to run psana with the time stamp filter and event writer in xtc file.
Both modules are available in psana library and they need only to be described in the configuration file. For example, the configuration file tstamp-filter-and-event-writer.cfg may looks likeExample of the psana configuration file:

Code Block
[psana]                                                               
files         = /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/<file-name-1>.xtc\
                /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/<file-name-2>.xtc
#skip-events = 100
events       =  5
modules = ImgAlgos.PnccdImageProducer ImgAlgos.ImgSaveInFile

[ImgAlgos.PnccdImageProducer]
source
files         = DetInfo(:pnCCD)
inkey/reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s00-c00.xtc \
         =
outimgkey     = imgpnccd
print_bits    = 1

[ImgAlgos.ImgSaveInFile]
source /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s01-c00.xtc \
          = DetInfo(:pnCCD)
key     /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s02-c00.xtc \
     = imgpnccd
fname         = pnccd-img-ev
saveAll /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s03-c00.xtc \
       = true
#eventSave     = 82
print_bits    = 1

This script saves text files with images like pnccd-img-ev-<run-date-time.nsec>.txt, which can be presented as:
Image Removed

Example for Module ImgAlgos::CameraImageProducer

See Module ImgAlgos::CameraImageProducer

Example of the psana configuration file:

Code Block

[psana]  /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s04-c00.xtc \
                /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s05-c00.xtc

skip-events   = 10
events        = 100
modules       = ImgAlgos.TimeStampFilter  PSXtcOutput.XtcOutputModule 

[PSXtcOutput.XtcOutputModule]
dirName       = ./test_out

[ImgAlgos.TimeStampFilter]
tsinterval    = 2012-02-02 18:17:00.409143728 / 2012-02-02 18:17:00.525853474
filterIsOn    = true
print_bits    = 11

Command to run psana is:
psana -c ./tstamp-filter-and-event-writer.cfg
For this configuration file psana will skip 10 events and loop over the next 100 events from the /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/e158-r0021-s0*-c00.xtc files and run consecutively modules = ImgAlgos.TimeStampFilter PSXtcOutput.XtcOutputModule. Parameters of these modules are described in the bottom part of the configuration file. Module TimeStampFilter passes events from the specified time interval and prints some useful information. Module XtcOutputModule will write passed events in the file with auto-generated name ./test_out/e158-r0021.xtcf.

See also: Module ImgAlgos::TimeStampFilter and PSXtcOutput::PSXtcOutput

Example for Module ImgAlgos::CSPadArrPeakAnalysis

See Module ImgAlgos::CSPadArrPeakAnalysis

Example of the psana configuration file:

Code Block

modules = cspad_mod.CsPadCalib \
  
files           = /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc/<file-name-1>.xtc
modules         = ImgAlgos.CameraImageProducer ImgAlgos.ImgSaveInFile
events          = 5

[ImgAlgos.CameraImageProducer] 
source          = DetInfo(:Opal1000)
key_inImgAlgos.CSPadMaskApply \
          =ImgAlgos.CSPadArrPeakFinder \
key_out         = img
subtract_offset = true
print_bits      = 15ImgAlgos.CSPadArrPeakAnalysis

# ...configuration parameters of other modules...

[ImgAlgos.ImgSaveInFileCSPadArrPeakAnalysis]
source             = DetInfo(:Opal1000CxiDs1.0:Cspad.0)
key                = imgpeaks
fnameprint_bits         = img-from-my-experiment
saveAll  7
fname_root     = true
#eventSave     = 1

This script saves text files with images like img-from-my-experiment-<run-date-time.nsec>.txt, which can be drawn by the python script

...

file.root

After execution in psana the file.root containing histogram(s) and ntuple(s) will be produced. Then, auxiliary script in root, running by the commend
root -q -f proc.C
produces the plots with histograms:

Image AddedImage Added