Versions Compared

Old Version 14

changes.mady.by.user Mikhail Dubrovin

Saved on

compared with

New Version Current

changes.mady.by.user Amedeo Perazzo

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Include Page
PSDM:Psana Module Examples
PSDM:Psana Module Examples

Table of Contents

This page provides examples for selected modules from Psana Module Catalog.

Auxiliary scripts

A few python scripts in ImgPixSpectra/data/ show how to process/plot the spectral array stored in the file.

  • PlotSpectralArrayFromFile.py - allows to plot content of the spectral array as a 2-d plot.
  • SpectralArray.py - provides access to the spectral array stored in the file. The class SpectralArray defined in this script is used in the PlotSpectralArrayFromFile.py.
  • MergeArrays.py - sums the arrays from different files defined in the list and saves resulting array in a single file with the same shape. In this script the list of files is hardwired in the get_list_of_input_file_names() method. The output file name, out_fname, is also hardwired in the call to spectra_merging(out_fname).

A few auxiliary scripts for example are located in the directory ImgAlgos/data:

  • psana.cfg - set non-default parameters to run psana for ImgAlgos::ImgPeakFinder and ImgAlgos::ImgPeakFilter
    The psana running this script saves images and peaks for event 115 in text files.
  • PlotCameraImageFromFile.py - Plots image and spectrum for image saved in file.
  • PlotCameraImageAndPeaks.py - Plots image with found peaks and spectrum.

Example for Package ImgPixSpectra

See Package ImgPixSpectra

Modules:

  • ImgPixSpectra::CSPadPixSpectra
  • ImgPixSpectra::MiniCSPadPixSpectra
  • ImgPixSpectra::CameraPixSpectra

Configuration file example for MiniCSPadPixSpectra:

Code Block

[psana]
files         = /reg/d/psdm/<instrument>/<experiment>/<file-name>.xtc
modules       = ImgPixSpectra.MiniCSPadPixSpectra

[ImgPixSpectra.MiniCSPadPixSpectra]
source        = CxiSc1.0:Cspad2x2.0
amin          =   500.
amax          =  1000.
nbins         =   100
arr_fname     = mini-cspad-pix-spectra.txt

To get images from saved file one may execute the auxiliary script:

Code Block

ImgPixSpectra/data/PlotSpectralArrayFromFile.py mini-cspad-pix-spectra.txt

generates image for limited range of pixels for CSPad, MiniCSPad, or Camera, respectively:
Image RemovedImage Removed Image Removed

Examples for Package ImgAlgos

See Package ImgAlgos

Example for Module ImgAlgos::ImgPeakFinder

See Module ImgAlgos::ImgPeakFinder

Configuration file example:

Code Block

[psana]                                                               
files         = /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
                /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-2>.xtc \
                ...
                /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc

modules       = ImgAlgos.ImgPeakFinder PrintSeparator

#skip-events  = 500
events        = 120

[ImgAlgos.ImgPeakFinder] 
source         = DetInfo(:Opal1000)
key            = 
peaksKey       =  peaks
threshold_low  =  20
threshold_high =  50
sigma          =  1.5
smear_radius   =  2
peak_radius    =  3
xmin           =  200
xmax           =  800
ymin           =  100
ymax           =  900
testEvent      =  115
print_bits     =  0
finderIsOn     =  true
  • This algorithm consumes ~15 ms/event on psana0101 for full Opal1000 (1024x1024) camera image.
  • Smearing algorithm use a "safety margin" which is currently set to 10 pixels (offset from each boarder of the full image size).

Image on different stages of this algorithm:
Image RemovedImage RemovedImage RemovedImage RemovedImage Removed

  1. raw image,
  2. image in the window with amplitudes above the threshold_low
    • few peaks at the edges were discarded by the window limits,
    • image still contains many 1-photon pixels, which need to be eliminated,
  3. smeared image,
  4. raw image with found peaks (marked by the red circles)
  5. zoom of the previous plot.

Example for Module ImgAlgos::CSPadArrAverage

See Module ImgAlgos::CSPadArrAverage

Configuration file example for evaluation of pedestals:

Code Block

[psana]
modules = ImgAlgos.CSPadArrAverage
files   = <path-to-the-dark-run-file>.xtc

[ImgAlgos.CSPadArrAverage]
source  = DetInfo(CxiDs1.0:Cspad.0)
key     =
avefile = cspad-pedestals-ave.dat
rmsfile = cspad-pedestals-rms.dat
print_bits  = 15
evts_stage1 = 100
evts_stage2 = 100
gate_width1 = 100
gate_width2 =  10

Configuration file example for evaluation of background:

Code Block

[psana]
files       = <path-to-the-background-run-file>.xtc
modules     = cspad_mod.CsPadCalib ImgAlgos.CSPadArrAverage
skip-events = 500
events      = 1000000

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadArrAverage]
source  = DetInfo(CxiDs1.0:Cspad.0)
key     = calibrated
avefile = cspad-background-ave.dat
rmsfile = cspad-background-rms.dat
print_bits  = 15

Images of the CSPad arrays for averaged and rms values, respectively, in one of the CXI runs:
Image RemovedImage Removed

Example for Module ImgAlgos::CSPadBkgdSubtract

See Module ImgAlgos::CSPadBkgdSubtract

Code Block

[psana]
files = /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
        /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-2>.xtc \
        ...
        /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc
skip-events = 500
events      = 10
modules     = cspad_mod.CsPadCalib ImgAlgos.CSPadBkgdSubtract

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no


[ImgAlgos.CSPadBkgdSubtract]
source        = DetInfo(CxiDs1.0:Cspad.0)
inputKey      = calibrated
outputKey     = bkgd_subtracted
bkgd_fname    = <the-file-name-with-background-array>
norm_sector   = 0
print_bits    = 3

The file with the background array, bkgd_fname, was obtained by averaging 1000 events using module CSPadArrAverage. Subtraction is done with normalization for norm_sector=0.
Event image and pixel amplitude spectrum before and after the background subtraction are shown in plots:
Image RemovedImage Removed
Image RemovedImage Removed

Other event with better subtracted background:
Image RemovedImage Removed

Example for Module ImgAlgos::CSPadMaskApply

See Module ImgAlgos::CSPadMaskApply

The array for mask contains zeros and ones for masked and passed pixels, respectively, and has a shape of full-size CSPad array 4*8*185388.
For example, it can be generated by the command

Code Block

./MakePixelMask.py <input-background-cspad-arr-file-name> <threshold> <output-file-name>

for the averaged background amplitude array <input-background-cspad-arr-file-name> abtained as a result of ImgAlgos::CSPadArrAverage module.

Plots show the averaged background, and the mask arrays generated from this background for three thresholds 10, 20, and 30 EDU:
Image RemovedImage RemovedImage RemovedImage Removed

The best results in filtering can be achieved in combination of modiles:

Code Block

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadBkgdSubtract \
          ImgAlgos.CSPadMaskApply \
          CSPadPixCoords.CSPadImageProducer \
          ImgAlgos.ImgPixAmpFilter 

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated_arr
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadBkgdSubtract]
source        = DetInfo(CxiDs1.0:Cspad.0)
inputKey      = calibrated_arr
outputKey     = bkgd_subtracted_arr
bkgd_fname    = ana-cxi49012/cspad-cxi49012-r0025-background-ave.dat
norm_sector   = 0
print_bits    = 0

[ImgAlgos.CSPadMaskApply]
source        = DetInfo(CxiDs1.0:Cspad.0)
inkey         = bkgd_subtracted_arr
outkey        = masked_arr
mask_fname    = ana-cxi49012/cspad-cxi49012-r0025-mask-20.dat
masked_amp    = -5
print_bits    = 0

[CSPadPixCoords.CSPadImageProducer]
calibDir      = /reg/d/psdm/CXI/cxi49012/calib
typeGroupName = CsPad::CalibV1
source        = DetInfo(:Cspad.0)
#key           = calibrated_arr
key           = masked_arr
imgkey        = masked_img
runNumber     = 25
#filter        = true
#tiltIsApplied = true

[ImgAlgos.ImgPixAmpFilter]
source        = DetInfo(:Cspad.0)
key           = masked_img
threshold     =  200
numPixMin     =  200
xmin          =  630
xmax          = 1070
ymin          =  630
ymax          = 1070
filterIsOn    = true
print_bits    = 11

where

  • cspad_mod.CsPadCalib - subtracts the pedestals from raw CSPad data,
  • ImgAlgos.CSPadBkgdSubtract - subtracts the background,
  • ImgAlgos.CSPadMaskApply - apply the mask,
  • CSPadPixCoords.CSPadImageProducer - produces the CSPad image,
  • ImgAlgos.ImgPixAmpFilter - apply the image-based amplitude threshold filter.

In the test with images for background represented by the water and solvent rings this filter provides the background suppression factor about 100.
The background images that still pass this filter have significantly larger intensity with respect to averaged background:
Image RemovedImage RemovedImage Removed

Example for Module ImgAlgos::ImgRadialCorrection

See Module ImgAlgos::ImgRadialCorrection

Code Block

[psana]
files = /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-1>.xtc \
        /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-2>.xtc \
        ...
        /reg/d/psdm/<instrument>/<experiment>/xtc/<file-name-N>.xtc
skip-events = 500
events      = 10
modules = cspad_mod.CsPadCalib \
#         ImgAlgos.CSPadBkgdSubtract \
          CSPadPixCoords.CSPadImageProducer \
          ImgAlgos.ImgRadialCorrection \
          CSPadPixCoords.CSPadImageGetTest:1
 

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadBkgdSubtract]
source        = DetInfo(CxiDs1.0:Cspad.0)
inputKey      = calibrated
outputKey     = bkgd_subtracted_arr
bkgd_fname    = <the-file-name-with-background-array>
norm_sector   = 0
print_bits    = 0

[CSPadPixCoords.CSPadImageProducer]
calibDir      = /reg/d/psdm/<instrument>/<experiment>/calib
typeGroupName = CsPad::CalibV1
source        = CxiDs1.0:Cspad.0
key           = calibrated
#key           = bkgd_subtracted_arr
imgkey        = current_img
runNumber     = 22
#tiltIsApplied = true

[ImgAlgos.ImgRadialCorrection]
source        = DetInfo(CxiDs1.0:Cspad.0)
inkey         = current_img
outkey        = r_cor_img
xcenter       = 866
ycenter       = 857
rmin          = 100
rmax          = 810
n_phi_bins    = 60
event         = 0
print_bits    = 3

[CSPadPixCoords.CSPadImageGetTest:1]
source        = CxiDs1.0:Cspad.0
key           = r_cor_img
fname         = <file-name-for-image-array>
#saveAll       = true
eventSave     = 8

Note: the option of the background subtraction (ImgAlgos.CSPadBkgdSubtract) is commented out in this configuration file . In order to evoke this option, the comment sign (#) should be removed from the list of modules and the key=bkgd_subtracted_arr should be used in CSPadPixCoords.CSPadImageProducer.

Calibrated image and spectrum:
Image RemovedImage Removed

Calibrated and radial-corrected image, spectrum, and subtracted r-phi65 distribution for n_phi_bins=65:
Image RemovedImage RemovedImage Removed

Calibrated and radial-corrected image, spectrum, and subtracted r-phi12 distribution for n_phi_bins=12:
Image RemovedImage RemovedImage Removed

Example for Module ImgAlgos::CSPadArrNoise

See Module ImgAlgos::CSPadArrNoise

Code Block

[psana]
modules = cspad_mod.CsPadCalib ImgAlgos.CSPadArrNoise

files   = /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s00-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s01-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s02-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s03-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s04-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0020-s05-c00.xtc

#skip-events = 1000
events      = 10

[cspad_mod.CsPadCalib]
inputKey      = 
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadArrNoise]
source          = DetInfo(CxiDs1.0:Cspad.0)
key             = calibrated
statusfile      = ana-cxi49012/cspad-cxi49012-r0020-noise-status.dat
maskfile        = ana-cxi49012/cspad-cxi49012-r0200-noise-mask.dat
print_bits      = 255
rmin            = 3
dr              = 1
SoNThr          = 3
frac_noisy_imgs = 0.15

Index map in median algorithm for rmin=3, dr=1:

Code Block

CSPadArrNoise::printMatrixOfIndexesForMedian():
 0 0 0 0 1 0 0 0 0
 0 0 1 1 1 1 1 0 0
 0 1 0 0 0 0 0 1 0
 0 1 0 0 0 0 0 1 0
 1 1 0 0 + 0 0 1 1
 0 1 0 0 0 0 0 1 0
 0 1 0 0 0 0 0 1 0
 0 0 1 1 1 1 1 0 0
 0 0 0 0 1 0 0 0 0

Pixel status (fraction of events where S/N > SoNThr):
Image Removed Image Removed Image Removed
For cspad-cxi49012-r0020 with parameters from confguration file (frac_noisy_imgs=0.15) we get, depending on number of events:

Nnoisy, Ntotal, Nnoisy/Ntotal pixels =94585 2296960 0.041 for 10 events
Nnoisy, Ntotal, Nnoisy/Ntotal pixels =2112 2296960 0.00092 for 100 events

Pixel mask for noisy pixels with |S/N| > SoNThr:
Image Removed Image Removed

Example for Module ImgAlgos::TimeStampFilter and PSXtcOutput.XtcOutputModule

This example demonstrates how to run psana with the time stamp filter and event writer in xtc file.
Both modules are available in psana library and they need only to be described in the configuration file. For example, the configuration file ./ana-cxi49012/psana-cxi49012-r0021-tstamp-filter-and-event-writer.cfg may looks like:

Code Block

[psana]                                                               
files         = /reg/d/psdm/cxi/cxi49012/xtc/e158-r0021-s00-c00.xtc \
                /reg/d/psdm/cxi/cxi49012/xtc/e158-r0021-s01-c00.xtc \
                /reg/d/psdm/cxi/cxi49012/xtc/e158-r0021-s02-c00.xtc \
                /reg/d/psdm/cxi/cxi49012/xtc/e158-r0021-s03-c00.xtc \
                /reg/d/psdm/cxi/cxi49012/xtc/e158-r0021-s04-c00.xtc \
                /reg/d/psdm/cxi/cxi49012/xtc/e158-r0021-s05-c00.xtc

skip-events   = 10
events        = 100
modules       = ImgAlgos.TimeStampFilter  PSXtcOutput.XtcOutputModule 

[PSXtcOutput.XtcOutputModule]
dirName       = ./test_out

[ImgAlgos.TimeStampFilter]
tsinterval    = 2012-02-02 18:17:00.409143728 / 2012-02-02 18:17:00.525853474
filterIsOn    = true
print_bits    = 11

Command to run psana is:
psana -c ./ana-cxi49012/psana-cxi49012-r0021-tstamp-filter-and-event-writer.cfg
For this configuration file psana will skip 10 events and loop over the next 100 events from the /reg/d/psdm/cxi/cxi49012/xtc/e158-r0021-s0*-c00.xtc files and run consecutively modules = ImgAlgos.TimeStampFilter PSXtcOutput.XtcOutputModule. Parameters of these modules are described in the bottom part of the configuration file. Module TimeStampFilter passes events from the specified time interval and prints some useful information. Module XtcOutputModule will write passed events in the file with auto-generated name ./test_out/e158-r0021-s00-c00.xtcf.

See also: Module ImgAlgos::TimeStampFilter and PSXtcOutput::PSXtcOutput