Introduction

Test the filter for runs where hits were seen.
Use cxi49012 from 2012-02-06 runs 115-153.

Analysis with Cheetah

Files from

/reg/d/psdm/cxi/cxi49012/scratch/hdf5/r0150-a/
(also available r0150-HG0, r0150-kb)

HDF5 structure:

Input file name  : /reg/d/psdm/cxi/cxi49012/scratch/hdf5/r0150-a/LCLS_2012_Feb06_r0150_194829_1585d_cspad.h5
<HDF5 file "LCLS_2012_Feb06_r0150_194829_1585d_cspad.h5" (mode r, 6.6M)> (File) /
     data (Group) /data
                  /data/data                  len = (1480, 1552)
                  /data/rawdata               len = (1480, 1552)
                  /data/radialAverageCounter  len = (1178,)
                  /data/radialAverage         len = (1178,)
     processing (Group) /processing
                        /processing/pixelmasks                   len = (1480, 1552)
                        /processing/hitfinder
                        /processing/hitfinder/peakinfo-raw       len = (7, 4)
                        /processing/hitfinder/peakinfo-assembled len = (7, 4)
                        /processing/hitfinder/peakinfo           len = (7, 4)
     LCLS (Group) /LCLS
                  /LCLS/ebeamLTUPosY           len = (1,)
                  /LCLS/ebeamLTUPosX           len = (1,)
                  /LCLS/f_21_ENRC              len = (1,)
                  /LCLS/eventTime              len = ()
                  /LCLS/phaseCavityCharge2     len = (1,)
                  /LCLS/evr41                  len = (1,)
                  /LCLS/detectorPosition       len = (1,)
                  /LCLS/f_11_ENRC              len = (1,)
                  /LCLS/photon_wavelength_A    len = (1,)
                  /LCLS/ebeamLTUAngY           len = (1,)
                  /LCLS/ebeamLTUAngX           len = (1,)
                  /LCLS/fiducial               len = (1,)
                  /LCLS/photon_energy_eV       len = (1,)
                  /LCLS/ebeamPkCurrBC2         len = (1,)
                  /LCLS/ebeamL3Energy          len = (1,)
                  /LCLS/machineTime            len = (1,)
                  /LCLS/cspadQuadTemperature   len = (4,)
                  /LCLS/detectorEncoderValue   len = (1,)
                  /LCLS/f_22_ENRC              len = (1,)
                  /LCLS/phaseCavityCharge1     len = (1,)
                  /LCLS/phaseCavityTime1       len = (1,)
                  /LCLS/phaseCavityTime2       len = (1,)
                  /LCLS/ebeamCharge            len = (1,)
                  /LCLS/f_12_ENRC              len = (1,)
                  /LCLS/eventTimeString        len = ()
The End

Time is available as:

/LCLS/eventTime: Mon Feb 6 19:48:29 2012
/LCLS/eventTimeString: Mon Feb 6 19:48:29 2012
/LCLS/machineTime: 1328586509
/LCLS/phaseCavityTime1: 7.37246259e-316
/LCLS/phaseCavityTime2: 0.
/LCLS/fiducial: 88157 that is 0x1585d as in the name of the file

Time in nanosecond is missing in the A.Barty's event hdf5 format.
Fiducial is missing in xtc->hdf5 translation.
There is no way to make sure that we compare the same events...

Log file:

/reg/d/psdm/cxi/cxi49012/scratch/hdf5/r0150-a/log.txt

>-------- Start of ini params --------<
defaultPhotonEnergyeV=5978.421875
defaultCameraLengthMm=-999.000000
detectorType=cspad
detectorName=CxiDs1
startAtFrame=0
stopAtFrame=0
nThreads=16
useHelperThreads=0
ioSpeedTest=0
threadPurge=10000
geometry=/reg/neh/home/hgliu/cxi/cxi49012/scratch/hgliu/pixmap/feb12_geo.h5
darkcal=/reg/d/ana12/cxi/cxi49012/scratch/hgliu/darcal/r0040-darkcal.h5
gaincal=/reg/d/psdm/cxi/cxi49012/scratch/cheetah/calib/gain_map.h5
peakmask=
badPixelMap=/reg/neh/home/knass/2012jan/scratch/knass/scripts/badpix-combo3.h5
subtractcmModule=0
cmModule=0
subtractUnbondedPixels=0
wiremaskFile=No_file_specified
subtractBehindWires=0
useGaincal=0
invertGain=0
generateDarkcal=0
generateGaincal=0
useBadPixelMap=1
useDarkcalSubtraction=1
hitfinder=1
saveHits=1
savePeakInfo=1
saveRaw=1
saveAssembled=0
saveDetectorCorrectedOnly=1
saveDetectorRaw=0
hdf5dump=0
saveInterval=1000
useAutoHotPixel=0
maskSaturatedPixels=1
pixelSaturationADC=15565
maskSaturatedPixels=1
pixelSaturationADC=15565
useSubtractPersistentBackground=0
useBackgroundBufferMutex=1
useLocalBackgroundSubtraction=0
localBackgroundRadius=3
tofName=CxiSc1
tofChannel=0
hitfinderUseTOF=0
hitfinderTOFMinSample=0
hitfinderTOFMaxSample=19999
hitfinderTOFThresh=0.100000
saveRadialStacks=0
radialStackSize=10000
cmFloor=0.100000
pixelSize=0.000110
debugLevel=2
hotpixFreq=0.900000
hotpixADC=1000
hotpixMemory=50
powderThresh=-20000
powderSumHits=1
powderSumBlanks=1
hitfinderADC=400
hitfinderNAT=2
hitfinderTIT=1000.000000
hitfinderCheckGradient=0
hitfinderMinGradient=0.000000
hitfinderCluster=0
hitfinderNPeaks=5
hitfinderNPeaksMax=1000
hitfinderAlgorithm=6
hitfinderMinPixCount=3
hitfinderMaxPixCount=20
hitfinderCheckPeakSeparation=0
hitfinderMaxPeakSeparation=5.000000
hitfinderSubtractLocalBG=0
hitfinderLocalBGRadius=4
hitfinderLocalBGThickness=1
hitfinderLimitRes=0
hitfinderMinRes=20.000000
hitfinderMaxRes=2.000000
hitfinderUsePeakMask=0
hitfinderMinSNR=15.000000
selfdarkMemory=50
bgMemory=50
bgRecalc=50
bgMedian=0.500000
bgIncludeHits=0
bgNoBeamReset=0
bgFiducialGlitchReset=0
scaleBackground=0
startFrames=0
>-------- End of ini params --------<

>-------- Start of job --------<
nFrames: 1000,  nHits: 20 (2.00%), recentHits: 20 (2.00%), wallTime: 0hr 0min 51sec (19.6 fps)
nFrames: 2000,  nHits: 20 (1.00%), recentHits: 0 (0.00%), wallTime: 0hr 2min 4sec (16.1 fps)
nFrames: 3000,  nHits: 43 (1.43%), recentHits: 23 (2.30%), wallTime: 0hr 3min 2sec (16.5 fps)
nFrames: 4000,  nHits: 43 (1.08%), recentHits: 0 (0.00%), wallTime: 0hr 4min 1sec (16.6 fps)
nFrames: 5000,  nHits: 56 (1.12%), recentHits: 13 (1.30%), wallTime: 0hr 4min 56sec (16.9 fps)
nFrames: 6000,  nHits: 109 (1.82%), recentHits: 53 (5.30%), wallTime: 0hr 5min 43sec (17.5 fps)
nFrames: 7000,  nHits: 221 (3.16%), recentHits: 112 (11.20%), wallTime: 0hr 6min 53sec (16.9 fps)
nFrames: 8000,  nHits: 254 (3.18%), recentHits: 33 (3.30%), wallTime: 0hr 7min 47sec (17.1 fps)
nFrames: 9000,  nHits: 528 (5.87%), recentHits: 274 (27.40%), wallTime: 0hr 9min 4sec (16.5 fps)
>-------- End of job --------<
End time: Mon Feb  6 20:28:14 2012
Elapsed time: 0hr 9min 43sec
Frames processed: 9534
nFrames in powder patterns:
        class0: 9006
        class1: 528
Number of hits: 528
Average hit rate: 5.54 %
Average frame rate: 16.35 fps
Average data rate: 71.65 MB/sec
Average photon energy: 5989.52  eV
Photon energy sigma:  6.60 eV

Selected events:

LCLS_2012_Feb06_r0150_194813_14270_cspad:

LCLS_2012_Feb06_r0150_194829_1585d_cspad

LCLS_2012_Feb06_r0150_194833_15f20_cspad

Analysis of cxi49012-r0150 in PSANA

General info about cxi49012-r0150

operator: aquila (Andrew L. Aquila)
analysis: hgliu (Haiguang Liu)
start / end time: 19:47:19 / 19:48:39
duration: 1 min 20 sec
total 9534 events
sample: RC Batch: Q1 Buffer: 100 ?l protein , 100 ?l lipidic sponge ...

Pedestals

Configuration file psana-cxi49012-r0133-pedestals.cfg for pedestals evaluation:

[psana]
modules = ImgAlgos.CSPadArrAverage
files   = /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s00-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s01-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s02-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s03-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s04-c00.xtc \
          /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s05-c00.xtc

#skip-events = 0
#events      = 100

[ImgAlgos.CSPadArrAverage]
source  = DetInfo(CxiDs1.0:Cspad.0)
key     =
avefile = cspad-cxi49012-r0133-pedestals-ave.dat
rmsfile = cspad-cxi49012-r0133-pedestals-rms.dat
print_bits  =  15
evts_stage1 = 100
evts_stage2 = 100
gate_width1 = 200
gate_width2 =  20

Runs with dark images: 115, 116, 121 127, 129, 133.
Pedestals from run 133:

Background

Portion of the configuration file psana-cxi49012-r0150-background-average.cfg for background averaging:

[ImgAlgos.CSPadArrAverage]
source        = DetInfo(CxiDs1.0:Cspad.0)
key           = calibrated
avefile       = ana-cxi49012/cspad-cxi49012-r0150-background-ave.dat
rmsfile       = ana-cxi49012/cspad-cxi49012-r0150-background-rms.dat
print_bits    = 15

Files with background as cspad array averaged for different event ranges:
cspad-cxi49012-r0150-background-ave-0-end.dat
cspad-cxi49012-r0150-background-ave-0-1000.dat
cspad-cxi49012-r0150-background-ave-1000-2000.dat
cspad-cxi49012-r0150-background-ave-2000-3000.dat
cspad-cxi49012-r0150-background-ave-4000-5000.dat
cspad-cxi49012-r0150-background-ave-7000-8000.dat

Images of Run 150 background averaged for event ranges: 0-end (9534), 0-1000, 2000-3000, 7000-8000:

Because of

the shape of background is differen during the run, and
the CSPadArrPeakFinder will subtract background level by definition,
we will not subtract the background by the CSPadBkgdSubtract.

Mask

Command to generate mask:
./MakePixelMask.py ana-cxi49012/cspad-cxi49012-r0150-background-ave.dat 0

Three type of mask have been generated:

cspad-cxi49012-r0150-mask-badregs.dat - masks bad regions of 2x1 only
cspad-cxi49012-r0150-mask-bkgd.dat - masks the high background central region
cspad-cxi49012-r0150-mask-rects.dat - rectangular mask of the high background central region

Initial mask for noisy pixels

Use the first 50 events to define the noisy pixel:

[ImgAlgos.CSPadArrNoise]
source          = DetInfo(CxiDs1.0:Cspad.0)
key             = calibrated
fracfile        = ana-cxi49012/cspad-cxi49012-r0150-noise-frac.dat
maskfile        = ana-cxi49012/cspad-cxi49012-r0150-noise-mask.dat
print_bits      = 255
rmin            = 3
dr              = 1
SoNThr          = 3
frac_noisy_imgs = 0.15

Array, image, and spectrum of fraction of noisy frames:

$LCLS Data Analysis - Internal > Filter for cxi49012 runs grater than 133 > cspad-cxi49012-r0150-noise-frac-arr.png$ $LCLS Data Analysis - Internal > Filter for cxi49012 runs grater than 133 > cspad-cxi49012-r0150-noise-frac-img.png$ $LCLS Data Analysis - Internal > Filter for cxi49012 runs grater than 133 > cspad-cxi49012-r0150-noise-frac-spe.png$

Array and image of the mask for noisy pixels:

Run peak finder `CSPadArrPeakFinder`

[psana]
files   = \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s00-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s01-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s02-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s03-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s04-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s05-c00.xtc

#skip-events = 1000
#events      = 23

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadMaskApply \
          ImgAlgos.CSPadArrPeakFinder
#          ImgAlgos.CSPadArrSaveInFile


[cspad_mod.CsPadCalib]
inputKey      =
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no


[ImgAlgos.CSPadMaskApply]
source            = DetInfo(CxiDs1.0:Cspad.0)
inkey             = calibrated
outkey            = masked_arr
mask_fname        = ana-cxi49012/cspad-cxi49012-r0150-mask-badregs.dat
#mask_fname        = ana-cxi49012/cspad-cxi49012-r0150-mask-bkgd.dat
#mask_fname        = ana-cxi49012/cspad-cxi49012-r0150-mask-rects.dat
masked_amp        =  8
print_bits        =  1
mask_control_bits = 15

[ImgAlgos.CSPadArrPeakFinder]
source            = DetInfo(CxiDs1.0:Cspad.0)
key               = masked_arr
key_peaks_out     = peaks
hot_pix_mask_inp_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask.dat
hot_pix_mask_out_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask-out.dat
frac_noisy_evts_file  = ana-cxi49012/cspad-cxi49012-r0150-noise-frac.dat
evt_file_out      = tmp/cspad-ev-
rmin              =     3
dr                =     1
SoNThr            =     3
frac_noisy_imgs   =   0.1
peak_npix_min     =     4
peak_npix_max     =    25
peak_amp_tot_thr  =   100.
event_npeak_min   =    10
event_amp_tot_thr =  1000.
nevents_mask_update = 100
nevents_mask_accum  =  50

selection_mode    = SELECTION_ON
out_file_bits     =    15
print_bits        =    512

[info:TimeInterval::startTime] Start time: 2012-06-13 10:01:09 and 837008000 nsec
[info:ImgAlgos.CSPadArrPeakFinder]
   NFrames:    1000  NHits:      71 (  7.10%)  Time:    473.930 sec ( 2.110 fps)
   NFrames:    2000  NHits:      71 (  3.55%)  Time:    918.985 sec ( 2.176 fps)
   NFrames:    3000  NHits:     102 (  3.40%)  Time:   1376.008 sec ( 2.180 fps)
   NFrames:    4000  NHits:     102 (  2.55%)  Time:   1822.940 sec ( 2.194 fps)
   NFrames:    5000  NHits:     120 (  2.40%)  Time:   2290.219 sec ( 2.183 fps)
   NFrames:    6000  NHits:     182 (  3.03%)  Time:   2757.514 sec ( 2.176 fps)
   NFrames:    7000  NHits:     311 (  4.44%)  Time:   3248.353 sec ( 2.155 fps)
   NFrames:    8000  NHits:     357 (  4.46%)  Time:   3806.922 sec ( 2.101 fps)
   NFrames:    9000  NHits:     686 (  7.62%)  Time:   4366.899 sec ( 2.061 fps)
 ===== JOB SUMMARY =====
   NFrames:    9534  NHits:     686 (  7.20%)  Time:   4607.013 sec ( 2.069 fps)
[info:TimeInterval::stopTime] Time to process 9534 events is 4607.01 sec, or 0.483 sec/event

Needs to be compared with Cheetah selection-a log file:

/reg/d/psdm/cxi/cxi49012/scratch/hdf5/r0150-a/log.txt

>-------- Start of job --------<
nFrames: 1000,  nHits: 20 (2.00%), recentHits: 20 (2.00%), wallTime: 0hr 0min 51sec (19.6 fps)
nFrames: 2000,  nHits: 20 (1.00%), recentHits: 0 (0.00%), wallTime: 0hr 2min 4sec (16.1 fps)
nFrames: 3000,  nHits: 43 (1.43%), recentHits: 23 (2.30%), wallTime: 0hr 3min 2sec (16.5 fps)
nFrames: 4000,  nHits: 43 (1.08%), recentHits: 0 (0.00%), wallTime: 0hr 4min 1sec (16.6 fps)
nFrames: 5000,  nHits: 56 (1.12%), recentHits: 13 (1.30%), wallTime: 0hr 4min 56sec (16.9 fps)
nFrames: 6000,  nHits: 109 (1.82%), recentHits: 53 (5.30%), wallTime: 0hr 5min 43sec (17.5 fps)
nFrames: 7000,  nHits: 221 (3.16%), recentHits: 112 (11.20%), wallTime: 0hr 6min 53sec (16.9 fps)
nFrames: 8000,  nHits: 254 (3.18%), recentHits: 33 (3.30%), wallTime: 0hr 7min 47sec (17.1 fps)
nFrames: 9000,  nHits: 528 (5.87%), recentHits: 274 (27.40%), wallTime: 0hr 9min 4sec (16.5 fps)
>-------- End of job --------<
End time: Mon Feb  6 20:28:14 2012
Elapsed time: 0hr 9min 43sec
Frames processed: 9534
Number of hits: 528
Average hit rate: 5.54 %
Average frame rate: 16.35 fps
Average data rate: 71.65 MB/sec
Average photon energy: 5989.52  eV
Photon energy sigma:  6.60 eV

2012-06-14 Comparison of selected events in PSANA vs Cheetah selection-a

psana file name: cspad-ev-000443-r0150-2012-02-06-194723.342595531-peaks.txt - fiducials are missing in psana data.
Cheetah file name: LCLS_2012_Feb06_r0150_194740_11456_cspad.h5 - nanoseconds are missing in Cheetah-based analysis.
We need external info to check matching of selected events. Command pyxtcreader <list-of-xtc-files> | grep L1Accept > file-time-to-fiducial generates a list of frame headers, which contain both the timestamp and fiducial, like time=1328586439.927456987, fiducials=63221,....

Matching algorithm use 3 lists:

list of records generated by the pyxtcreader command,
list of files with selected frames in Cheetah,
list of files with selected frames in psana.

Python script loops over all frames of particular run (pyxtcreader records) and check if the Cheetah and/or PSANA have selected particular frame. For Cheetah it checks

date: YYYYMMDD,
time: HHMMSS,
fiducials.

For PSANA it checks

date: YYYYMMDD,
time: HHMMSS,

nanoseconds.
The portion of the matching algorithm log file:

Frame: 8928   1328586514 sec   25313196 nsec   fiducials: 89906    HHMMSS: 194834    date: 20120206   isInCheetah: False   isInPSANA: False
Frame: 8929   1328586514 sec   33643630 nsec   fiducials: 89909    HHMMSS: 194834    date: 20120206   isInCheetah: False   isInPSANA: True
Frame: 8930   1328586514 sec   41973933 nsec   fiducials: 89912    HHMMSS: 194834    date: 20120206   isInCheetah: True    isInPSANA: True
Frame: 8931   1328586514 sec   50302898 nsec   fiducials: 89915    HHMMSS: 194834    date: 20120206   isInCheetah: False   isInPSANA: False
Frame: 8932   1328586514 sec   58634037 nsec   fiducials: 89918    HHMMSS: 194834    date: 20120206   isInCheetah: True    isInPSANA: True
Frame: 8933   1328586514 sec   66961663 nsec   fiducials: 89921    HHMMSS: 194834    date: 20120206   isInCheetah: False   isInPSANA: False
Frame: 8934   1328586514 sec   75294182 nsec   fiducials: 89924    HHMMSS: 194834    date: 20120206   isInCheetah: True    isInPSANA: True
Frame: 8935   1328586514 sec   83622351 nsec   fiducials: 89927    HHMMSS: 194834    date: 20120206   isInCheetah: False   isInPSANA: False
Frame: 8936   1328586514 sec   91952812 nsec   fiducials: 89930    HHMMSS: 194834    date: 20120206   isInCheetah: False   isInPSANA: False
Frame: 8937   1328586514 sec  100280993 nsec   fiducials: 89933    HHMMSS: 194834    date: 20120206   isInCheetah: True    isInPSANA: True

Finally, we count for statistics of different possibilities using status of boolean variables isInCheetah and isInPSANA.

Result for all counters in r0150:

 Total number of frames in file  : 9534
 Total number of hits in Cheetah : 528
 Total number of hits in PSANA   : 686
 Non-found hits in both          : 8831
 Found in both                   : 511
 Found in PSANA only             : 175
 Found in Cheetah only           : 17

Summary

For current set of parameters

ImgAlgos.CSPadArrPeakFinder selects 7.20% of hits
Cheetah saves 5.87% of hits
PSANA missed 17 and selected extra 175 events versus Cheetah for more than 500 selected frames.

Selected events

ev-007713:

ev-008944:

Events selected by Cheetah only

Most of 17 events, selected by Cheetah only, were not selected by the CSPadArrPeakFinder because 9 peaks were found only in stead of required 10.
Example of events:

2012-06-18 Change selection from 10 peaks to 9

[info:TimeInterval::startTime] Start time: 2012-06-18 16:41:01 and 651642000 nsec
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    1000  NHits:      84 (  8.40%)  Time:    478.098 sec ( 2.092 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    2000  NHits:      84 (  4.20%)  Time:    926.035 sec ( 2.160 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    3000  NHits:     116 (  3.87%)  Time:   1383.505 sec ( 2.168 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    4000  NHits:     116 (  2.90%)  Time:   1832.200 sec ( 2.183 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    5000  NHits:     135 (  2.70%)  Time:   2285.611 sec ( 2.188 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    6000  NHits:     203 (  3.38%)  Time:   2754.092 sec ( 2.179 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    7000  NHits:     340 (  4.86%)  Time:   3247.141 sec ( 2.156 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    8000  NHits:     390 (  4.88%)  Time:   3712.526 sec ( 2.155 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    9000  NHits:     736 (  8.18%)  Time:   4278.251 sec ( 2.104 fps)
[info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY =====
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    9534  NHits:     736 (  7.72%)  Time:   4516.692 sec ( 2.111 fps)
[info:TimeInterval::stopTime] Time to process 9534 events is 4516.69 sec, or 0.473746 sec/event

Comparison of selected events in PSANA vs Cheetah selection-a

Result for all counters:

 Total number of frames          : 9534
 Total number of hits in Cheetah : 528
 Total number of hits in PSANA   : 736
 Non-found hits in both          : 8788
 Found in both                   : 518
 Found in PSANA only             : 218
 Found in Cheetah only           : 10

10 Events selected by Cheetah only

Frame: 6084   1328586490 sec   334312132 nsec   fiducials: 81374    HHMMSS: 194810    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 6504   1328586493 sec   832694890 nsec   fiducials: 82634    HHMMSS: 194813    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 6553   1328586494 sec   240845567 nsec   fiducials: 82781    HHMMSS: 194814    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 6851   1328586496 sec   723213642 nsec   fiducials: 83675    HHMMSS: 194816    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 6974   1328586497 sec   747879423 nsec   fiducials: 84044    HHMMSS: 194817    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 6994   1328586497 sec   914490230 nsec   fiducials: 84104    HHMMSS: 194817    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 7911   1328586505 sec   553606864 nsec   fiducials: 86855    HHMMSS: 194825    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 8440   1328586509 sec   960190204 nsec   fiducials: 88442    HHMMSS: 194829    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 8449   1328586510 sec   35161608 nsec   fiducials: 88469    HHMMSS: 194830    date: 20120206   isInCheetah: True   isInPSANA: False
Frame: 8452   1328586510 sec   60152788 nsec   fiducials: 88478    HHMMSS: 194830    date: 20120206   isInCheetah: True   isInPSANA: False

2012-06-21 Speed-up processing using multithreading

Log file shows that speed of this algorithm is about 2fps, that is slow. This speed is defined by the median evaluation for each pixel of the image over about 50 remote neighbor pixels. PSANA framework supplies data for each quad. For each section of the quad the median algorithm does independent processing, that can be done in parallel in multithreading mode. We use the OpenMP procedure
#pragma omp parallel for
just before the for loop over all available CSPAD sections of the quad. For the thread-safe processing the code inside the for loop is modified in order to isolate all variables and output data in different threads. Comparative difference between speed of the single-thread and multi-treading modes is shown below.

In multithreading mode:

[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    1000  NHits:     163 ( 16.30%)  Time:    160.467 sec ( 6.232 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    2000  NHits:     163 (  8.15%)  Time:    279.290 sec ( 7.161 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    3000  NHits:     203 (  6.77%)  Time:    400.101 sec ( 7.498 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    4000  NHits:     203 (  5.07%)  Time:    522.545 sec ( 7.655 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    5000  NHits:     236 (  4.72%)  Time:    648.397 sec ( 7.711 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    6000  NHits:     359 (  5.98%)  Time:    797.908 sec ( 7.520 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    7000  NHits:     593 (  8.47%)  Time:    970.897 sec ( 7.210 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    8000  NHits:     689 (  8.61%)  Time:   1116.850 sec ( 7.163 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    9000  NHits:    1165 ( 12.94%)  Time:   1372.103 sec ( 6.559 fps)
[info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY =====
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    9534  NHits:    1165 ( 12.22%)  Time:   1451.644 sec ( 6.568 fps)
[info:TimeInterval::stopTime] Time to process 9534 events is 1451.64 sec, or 0.15226 sec/event
dubrovin@psana0204

Snapshots from top:

PID   USER      PR  NI  VIRT  RES  SHR S %CPU  %MEM    TIME+ COMMAND
31250 dubrovin  16   0 1126m 250m  16m R 426.1  3.1  83:20.34 psana
31250 dubrovin  16   0  998m 212m  16m R 515.2  2.7  93:06.69 psana
31250 dubrovin  16   0 1062m 239m  16m R 278.5  3.0  94:49.83 psana
31250 dubrovin  16   0 1126m 259m  16m R 440.5  3.2 100:26.43 psana

Single-thread mode

[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    1000  NHits:     163 ( 16.30%)  Time:    503.288 sec ( 1.987 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    2000  NHits:     163 (  8.15%)  Time:    949.256 sec ( 2.107 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    3000  NHits:     203 (  6.77%)  Time:   1408.933 sec ( 2.129 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    4000  NHits:     203 (  5.07%)  Time:   1855.885 sec ( 2.155 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    5000  NHits:     236 (  4.72%)  Time:   2312.426 sec ( 2.162 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    6000  NHits:     359 (  5.98%)  Time:   2798.862 sec ( 2.144 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    7000  NHits:     593 (  8.47%)  Time:   3321.919 sec ( 2.107 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    8000  NHits:     689 (  8.61%)  Time:   3798.451 sec ( 2.106 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    9000  NHits:    1165 ( 12.94%)  Time:   4406.237 sec ( 2.043 fps)
[info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY =====
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    9534  NHits:    1165 ( 12.22%)  Time:   4644.862 sec ( 2.053 fps)
[info:TimeInterval::stopTime] Time to process 9534 events is 4644.86 sec, or 0.487189 sec/event
dubrovin@psana0204

Snapshots from top:

PID   USER      PR  NI  VIRT  RES  SHR S %CPU  %MEM    TIME+ COMMAND
31736 dubrovin  18   0 1088m 269m  16m R 102.2  3.4  26:18.80 psana
31736 dubrovin  18   0 1024m 249m  16m R 104.0  3.1  28:00.43 psana
31736 dubrovin  17   0 1024m 250m  16m R  99.0  3.1  44:54.80 psana
31736 dubrovin  19   0 1088m 269m  16m R 103.5  3.4  48:24.64 psana

Comparison of selected events in PSANA vs Cheetah selection-a

Result for all counters:

 Total number of frames          : 9534
 Total number of hits in Cheetah : 528
 Total number of hits in PSANA   : 1165
 Non-found hits in both          : 8368
 Found in both                   : 527
 Found in PSANA only             : 638
 Found in Cheetah only           : 1

In this version PSANA missed 1 event only, that is 0.01%.
Parameters of the CSPadArrPeakFinder have not been changed.
Code has been changed for thread-safe operations.
It may happen due to persistence in member-data, which was eliminated in new code.

Summary for multithreading

Great success is acheived in isolation of data and parameters between threads.
- All crashes disappeared
- Single- and multi-threading modes select exactly the same set of events
Multithreading allows to speed-up the CSPadArrPeakFinder in 6.568fps / 2.053fps about 3 times.

2012-06-27 Tuning of parameters

After meeting with Anton the peak finder parameters are changed;

SoNThr=3 is changed to SoNThr_noise=3
Add SoNThr_signal=8
frac_noisy_imgs=0.9 (it was 0.1)
remove amplitude thresholds in ADU peak_amp_tot_thr=0. and event_amp_tot_thr=0.
Add event_npeak_max=10000
Changed nevents_mask_update=0 (it was 100) so the mask of noisy pixels is updated every 50 events without any gap.
Add peak_SoN_thr - threshold on S/N counted over peak connected pixels.

Selection V5:

[ImgAlgos.CSPadArrPeakFinder]
source            = DetInfo(CxiDs1.0:Cspad.0)
key               = masked_arr
key_peaks_out     = peaks
#key_signal_out    = signal-arr
#hot_pix_mask_inp_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask.dat
hot_pix_mask_out_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask-out.dat
frac_noisy_evts_file  = ana-cxi49012/cspad-cxi49012-r0150-noise-frac.dat
evt_file_out        = tmp/cspad-ev-
rmin                =     3
dr                  =     1
SoNThr_noise        =     3
SoNThr_signal       =     7
frac_noisy_imgs     =   0.9
peak_npix_min       =     3
peak_npix_max       =    20
peak_amp_tot_thr    =     0.
peak_SoN_thr        =     7.
event_npeak_min     =     5
event_npeak_max     = 10000
event_amp_tot_thr   =     0.
nevents_mask_update =     0
nevents_mask_accum  =    50
selection_mode      = SELECTION_ON
out_file_bits       =     12
print_bits          =    577

[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    1000  NHits:      22 (  2.20%)  Time:    137.074 sec ( 7.295 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    2000  NHits:      22 (  1.10%)  Time:    292.825 sec ( 6.830 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    3000  NHits:      50 (  1.67%)  Time:    453.272 sec ( 6.619 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    4000  NHits:      50 (  1.25%)  Time:    605.666 sec ( 6.604 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    5000  NHits:      70 (  1.40%)  Time:    758.880 sec ( 6.589 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    6000  NHits:     153 (  2.55%)  Time:    919.956 sec ( 6.522 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    7000  NHits:     329 (  4.70%)  Time:   1099.423 sec ( 6.367 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    8000  NHits:     390 (  4.88%)  Time:   1258.966 sec ( 6.354 fps)
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    9000  NHits:     744 (  8.27%)  Time:   1489.109 sec ( 6.044 fps)
[info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY =====
[info:ImgAlgos.CSPadArrPeakFinder]   NFrames:    9534  NHits:     744 (  7.80%)  Time:   1569.498 sec ( 6.075 fps)
[info:TimeInterval::stopTime] Time to process 9534 events is 1569.5 sec, or 0.164621 sec/event


Mask of noisy:
[info:ImgAlgos.CSPadArrPeakFinder]   Event:9499  Collected for mask update:50  Statistics: Nnoisy:147  Ntotal:2296960  Nnoisy/Ntotal pixels:6.39976e-05

Comparison with Cheetah

 Cheetah selection version       : a       : HG0     : kb
 Total number of frames          : 9534    : 9534    : 9534
 Total number of hits in Cheetah : 528 	   : 187     : 317
 Total number of hits in PSANA   : 744 	   : 744     : 744
 Non-selected frames in both     : 8783    : 8790    : 8785
 Found in both                   : 521 	   : 187     : 312
 Found in PSANA only             : 223 	   : 557     : 432
 Found in Cheetah only           : 7 	   : 0       : 5

ev-006785-r0150-2012-02-06-194816.173410278:

cxi49012-r0150-selected-arr-194833.467189379:

Summary overall

Peak finder / filter is ready
All features are implemented
It is demonstrated that it finds peaks correctly, depending on selection conditions.
It is demonstrated that for similar selection parameters with Cheetah the mismatch in number of selected events does not exceed 1%.
OpenMP multithreading for parallel processing of sectors in quads 3-times speed-up calculations.

F2. Peak finding algorithm features:

ImgAlgos.CSPadMaskApply - is used to apply constant mask on 2x1 edges and bad regions.
Noisy mask is generated dynamically over each 50 events. Initial noisy-pixel mask can be loaded from file, but is not loaded, like in Cheetah...
ADC thresholds can be applied on pixel and total peak amplitudes, but are not applied, like in Cheetah...

Median algorithm use ring-shaped neighbor pixels to calculate the level of background and noise fluctuations.
Cheetah uses square region of the same size. Actually, the difference is in 4 pixels:

Index vector size: 24
 0 0 0 0 1 0 0 0 0
 0 0 1 1 1 1 1 0 0
 0 1 0 0 0 0 0 1 0
 0 1 0 0 0 0 0 1 0
 1 1 0 0 + 0 0 1 1
 0 1 0 0 0 0 0 1 0
 0 1 0 0 0 0 0 1 0
 0 0 1 1 1 1 1 0 0
 0 0 0 0 1 0 0 0 0

S/N thresholds can be applied on individual pixels or entire peak.

2012-07-09 Module ImgAlgos::CSPadArrPeakAnalysis

See module description in Module ImgAlgos::CSPadArrPeakAnalysis

Example of the psana configuration file:

[psana]
files   = \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s00-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s01-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s02-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s03-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s04-c00.xtc \
   /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s05-c00.xtc

modules = cspad_mod.CsPadCalib \
          ImgAlgos.CSPadMaskApply \
          ImgAlgos.CSPadArrPeakFinder \
          ImgAlgos.CSPadArrPeakAnalysis

[cspad_mod.CsPadCalib]
inputKey      =
outputKey     = calibrated
doPedestals   = yes
doPixelStatus = no
doCommonMode  = no

[ImgAlgos.CSPadMaskApply]
source            = DetInfo(CxiDs1.0:Cspad.0)
inkey             = calibrated
outkey            = masked_arr
mask_fname        = ana-cxi49012/cspad-cxi49012-r0150-mask-badregs.dat
masked_amp        = -1
print_bits        =  5
mask_control_bits = 15

[ImgAlgos.CSPadArrPeakFinder]
source              = DetInfo(CxiDs1.0:Cspad.0)
key                 = masked_arr
key_peaks_out       = peaks
evt_file_out        = tmp/cspad-ev-
rmin                =     3
dr                  =     1
SoNThr_noise        =     3
SoNThr_signal       =     7
frac_noisy_imgs     =   0.9
peak_npix_min       =     3
peak_npix_max       =    20
peak_amp_tot_thr    =     0.
peak_SoN_thr        =     7.
event_npeak_min     =     5
event_npeak_max     = 10000
event_amp_tot_thr   =     0.
nevents_mask_update =     0
nevents_mask_accum  =    50
selection_mode      = SELECTION_OFF
out_file_bits       =     0
print_bits          =   513

[ImgAlgos.CSPadArrPeakAnalysis]
source              = DetInfo(CxiDs1.0:Cspad.0)
key                 = peaks
print_bits          = 7
fname_root          = file.root

In order to collect statistics about all peaks (not only for selected events), the selection_mode parameter is set to SELECTION_OFF.

After execution of this script in psana the file.root containing histogram(s) and ntuple(s) will be produced. Then, auxiliary script in root, running by the command root -q -f proc.C produces the plots with histograms:

These distributions were obtained w/o selection and represent background events mainly:

Number of peaks (in event) - distribution shows that large fraction of events has ?5 peaks;
Number of pixels in peak is ?9 in most cases;
atot - total signal amplitude in the peak region (S);
btot - total background in the peak region;
noise - total noise in the peak region (N);
son - S/N ratio;
Quad - population of peaks over quads;
Sect - population of peaks over sectors;
col - population of peaks over columns;
row - population of peaks over rows;
sigma_c - rms size of the peak region in columns;
sigma_r - rms size of the peak region in rows.

In case if the selection_mode parameter is set to SELECTION_ON the distributions represent peaks for selected signal events:

2012-07-12 Test of selected and re-selected events

There may be a potential problem with a list of selected events; when the same filter use the xtc-file of re-written events, filter may find a different number of selected events... This may happen due to changing conditions.

There are at least two reasons for changing conditions:

hot pixel mask is generated dynamically under control of the nevents_mask_update and nevents_mask_accum parameters. The mask itself depends on a set of events and is different for original and selected set of events. I have tested for this effect and did not find any difference in selected number of events for cxi49012-r0150 (9534 events total and 740-selected). In order to get rid of this effect, the dynamic mask update can be switched off by setting nevents_mask_update to the large value. Constant hot pixel mask still can be loaded from the hot_pix_mask_inp_file.
if the input xtc file name has a wrong structure (for example: e158-r0150-s00-c00.xtc is right, but e158-r0150.xtc is wrong) the name of experiment can not be defined correctly and the standard calib directory can not be found as well. In this case the calibration parameters will be different that gives significant difference in conditions of the peak finding algorithm. I have observed the 25% difference in the number of selected events.

Introduction

Analysis with Cheetah

Files from

HDF5 structure:

Time is available as:

Log file:

Selected events:

Analysis of cxi49012-r0150 in PSANA

General info about cxi49012-r0150

Pedestals

Background

Mask

Initial mask for noisy pixels

Run peak finder CSPadArrPeakFinder

Needs to be compared with Cheetah selection-a log file:

2012-06-14 Comparison of selected events in PSANA vs Cheetah selection-a

Summary

Selected events

Events selected by Cheetah only

2012-06-18 Change selection from 10 peaks to 9

Comparison of selected events in PSANA vs Cheetah selection-a

2012-06-21 Speed-up processing using multithreading

In multithreading mode:

Single-thread mode

Comparison of selected events in PSANA vs Cheetah selection-a

Summary for multithreading

2012-06-27 Tuning of parameters

Comparison with Cheetah

Summary overall

2012-07-09 Module ImgAlgos::CSPadArrPeakAnalysis

2012-07-12 Test of selected and re-selected events

Run peak finder `CSPadArrPeakFinder`