Introduction
Test the filter for runs where hits were seen.
Use cxi49012 from 2012-02-06 runs 115-153.
Analysis with Cheetah
Files from
/reg/d/psdm/cxi/cxi49012/scratch/hdf5/r0150-a/
(also available r0150-HG0
, r0150-kb
)
HDF5 structure:
Input file name : /reg/d/psdm/cxi/cxi49012/scratch/hdf5/r0150-a/LCLS_2012_Feb06_r0150_194829_1585d_cspad.h5 <HDF5 file "LCLS_2012_Feb06_r0150_194829_1585d_cspad.h5" (mode r, 6.6M)> (File) / data (Group) /data /data/data len = (1480, 1552) /data/rawdata len = (1480, 1552) /data/radialAverageCounter len = (1178,) /data/radialAverage len = (1178,) processing (Group) /processing /processing/pixelmasks len = (1480, 1552) /processing/hitfinder /processing/hitfinder/peakinfo-raw len = (7, 4) /processing/hitfinder/peakinfo-assembled len = (7, 4) /processing/hitfinder/peakinfo len = (7, 4) LCLS (Group) /LCLS /LCLS/ebeamLTUPosY len = (1,) /LCLS/ebeamLTUPosX len = (1,) /LCLS/f_21_ENRC len = (1,) /LCLS/eventTime len = () /LCLS/phaseCavityCharge2 len = (1,) /LCLS/evr41 len = (1,) /LCLS/detectorPosition len = (1,) /LCLS/f_11_ENRC len = (1,) /LCLS/photon_wavelength_A len = (1,) /LCLS/ebeamLTUAngY len = (1,) /LCLS/ebeamLTUAngX len = (1,) /LCLS/fiducial len = (1,) /LCLS/photon_energy_eV len = (1,) /LCLS/ebeamPkCurrBC2 len = (1,) /LCLS/ebeamL3Energy len = (1,) /LCLS/machineTime len = (1,) /LCLS/cspadQuadTemperature len = (4,) /LCLS/detectorEncoderValue len = (1,) /LCLS/f_22_ENRC len = (1,) /LCLS/phaseCavityCharge1 len = (1,) /LCLS/phaseCavityTime1 len = (1,) /LCLS/phaseCavityTime2 len = (1,) /LCLS/ebeamCharge len = (1,) /LCLS/f_12_ENRC len = (1,) /LCLS/eventTimeString len = () The End
Time is available as:
/LCLS/eventTime: Mon Feb 6 19:48:29 2012
/LCLS/eventTimeString: Mon Feb 6 19:48:29 2012
/LCLS/machineTime: 1328586509
/LCLS/phaseCavityTime1: 7.37246259e-316
/LCLS/phaseCavityTime2: 0.
/LCLS/fiducial: 88157
that is 0x1585d as in the name of the file
- Time in nanosecond is missing in the A.Barty's event hdf5 format.
- Fiducial is missing in xtc->hdf5 translation.
- There is no way to make sure that we compare the same events...
Log file:
/reg/d/psdm/cxi/cxi49012/scratch/hdf5/r0150-a/log.txt
>-------- Start of ini params --------< defaultPhotonEnergyeV=5978.421875 defaultCameraLengthMm=-999.000000 detectorType=cspad detectorName=CxiDs1 startAtFrame=0 stopAtFrame=0 nThreads=16 useHelperThreads=0 ioSpeedTest=0 threadPurge=10000 geometry=/reg/neh/home/hgliu/cxi/cxi49012/scratch/hgliu/pixmap/feb12_geo.h5 darkcal=/reg/d/ana12/cxi/cxi49012/scratch/hgliu/darcal/r0040-darkcal.h5 gaincal=/reg/d/psdm/cxi/cxi49012/scratch/cheetah/calib/gain_map.h5 peakmask= badPixelMap=/reg/neh/home/knass/2012jan/scratch/knass/scripts/badpix-combo3.h5 subtractcmModule=0 cmModule=0 subtractUnbondedPixels=0 wiremaskFile=No_file_specified subtractBehindWires=0 useGaincal=0 invertGain=0 generateDarkcal=0 generateGaincal=0 useBadPixelMap=1 useDarkcalSubtraction=1 hitfinder=1 saveHits=1 savePeakInfo=1 saveRaw=1 saveAssembled=0 saveDetectorCorrectedOnly=1 saveDetectorRaw=0 hdf5dump=0 saveInterval=1000 useAutoHotPixel=0 maskSaturatedPixels=1 pixelSaturationADC=15565 maskSaturatedPixels=1 pixelSaturationADC=15565 useSubtractPersistentBackground=0 useBackgroundBufferMutex=1 useLocalBackgroundSubtraction=0 localBackgroundRadius=3 tofName=CxiSc1 tofChannel=0 hitfinderUseTOF=0 hitfinderTOFMinSample=0 hitfinderTOFMaxSample=19999 hitfinderTOFThresh=0.100000 saveRadialStacks=0 radialStackSize=10000 cmFloor=0.100000 pixelSize=0.000110 debugLevel=2 hotpixFreq=0.900000 hotpixADC=1000 hotpixMemory=50 powderThresh=-20000 powderSumHits=1 powderSumBlanks=1 hitfinderADC=400 hitfinderNAT=2 hitfinderTIT=1000.000000 hitfinderCheckGradient=0 hitfinderMinGradient=0.000000 hitfinderCluster=0 hitfinderNPeaks=5 hitfinderNPeaksMax=1000 hitfinderAlgorithm=6 hitfinderMinPixCount=3 hitfinderMaxPixCount=20 hitfinderCheckPeakSeparation=0 hitfinderMaxPeakSeparation=5.000000 hitfinderSubtractLocalBG=0 hitfinderLocalBGRadius=4 hitfinderLocalBGThickness=1 hitfinderLimitRes=0 hitfinderMinRes=20.000000 hitfinderMaxRes=2.000000 hitfinderUsePeakMask=0 hitfinderMinSNR=15.000000 selfdarkMemory=50 bgMemory=50 bgRecalc=50 bgMedian=0.500000 bgIncludeHits=0 bgNoBeamReset=0 bgFiducialGlitchReset=0 scaleBackground=0 startFrames=0 >-------- End of ini params --------< >-------- Start of job --------< nFrames: 1000, nHits: 20 (2.00%), recentHits: 20 (2.00%), wallTime: 0hr 0min 51sec (19.6 fps) nFrames: 2000, nHits: 20 (1.00%), recentHits: 0 (0.00%), wallTime: 0hr 2min 4sec (16.1 fps) nFrames: 3000, nHits: 43 (1.43%), recentHits: 23 (2.30%), wallTime: 0hr 3min 2sec (16.5 fps) nFrames: 4000, nHits: 43 (1.08%), recentHits: 0 (0.00%), wallTime: 0hr 4min 1sec (16.6 fps) nFrames: 5000, nHits: 56 (1.12%), recentHits: 13 (1.30%), wallTime: 0hr 4min 56sec (16.9 fps) nFrames: 6000, nHits: 109 (1.82%), recentHits: 53 (5.30%), wallTime: 0hr 5min 43sec (17.5 fps) nFrames: 7000, nHits: 221 (3.16%), recentHits: 112 (11.20%), wallTime: 0hr 6min 53sec (16.9 fps) nFrames: 8000, nHits: 254 (3.18%), recentHits: 33 (3.30%), wallTime: 0hr 7min 47sec (17.1 fps) nFrames: 9000, nHits: 528 (5.87%), recentHits: 274 (27.40%), wallTime: 0hr 9min 4sec (16.5 fps) >-------- End of job --------< End time: Mon Feb 6 20:28:14 2012 Elapsed time: 0hr 9min 43sec Frames processed: 9534 nFrames in powder patterns: class0: 9006 class1: 528 Number of hits: 528 Average hit rate: 5.54 % Average frame rate: 16.35 fps Average data rate: 71.65 MB/sec Average photon energy: 5989.52 eV Photon energy sigma: 6.60 eV
Selected events:
LCLS_2012_Feb06_r0150_194813_14270_cspad:
LCLS_2012_Feb06_r0150_194829_1585d_cspad
LCLS_2012_Feb06_r0150_194833_15f20_cspad
Analysis of cxi49012-r0150 in PSANA
General info about cxi49012-r0150
- operator: aquila (Andrew L. Aquila)
- analysis: hgliu (Haiguang Liu)
- start / end time: 19:47:19 / 19:48:39
- duration: 1 min 20 sec
- total 9534 events
- sample: RC Batch: Q1 Buffer: 100 ?l protein , 100 ?l lipidic sponge ...
Pedestals
Configuration file psana-cxi49012-r0133-pedestals.cfg
for pedestals evaluation:
[psana] modules = ImgAlgos.CSPadArrAverage files = /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s00-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s01-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s02-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s03-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s04-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0133-s05-c00.xtc #skip-events = 0 #events = 100 [ImgAlgos.CSPadArrAverage] source = DetInfo(CxiDs1.0:Cspad.0) key = avefile = cspad-cxi49012-r0133-pedestals-ave.dat rmsfile = cspad-cxi49012-r0133-pedestals-rms.dat print_bits = 15 evts_stage1 = 100 evts_stage2 = 100 gate_width1 = 200 gate_width2 = 20
Runs with dark images: 115, 116, 121 127, 129, 133.
Pedestals from run 133:
Background
Portion of the configuration file psana-cxi49012-r0150-background-average.cfg
for background averaging:
[ImgAlgos.CSPadArrAverage] source = DetInfo(CxiDs1.0:Cspad.0) key = calibrated avefile = ana-cxi49012/cspad-cxi49012-r0150-background-ave.dat rmsfile = ana-cxi49012/cspad-cxi49012-r0150-background-rms.dat print_bits = 15
Files with background as cspad array averaged for different event ranges:
cspad-cxi49012-r0150-background-ave-0-end.dat
cspad-cxi49012-r0150-background-ave-0-1000.dat
cspad-cxi49012-r0150-background-ave-1000-2000.dat
cspad-cxi49012-r0150-background-ave-2000-3000.dat
cspad-cxi49012-r0150-background-ave-4000-5000.dat
cspad-cxi49012-r0150-background-ave-7000-8000.dat
Images of Run 150 background averaged for event ranges: 0-end (9534), 0-1000, 2000-3000, 7000-8000:
Because of
- the shape of background is differen during the run, and
- the
CSPadArrPeakFinder
will subtract background level by definition,
we will not subtract the background by theCSPadBkgdSubtract
.
Mask
Command to generate mask:
./MakePixelMask.py ana-cxi49012/cspad-cxi49012-r0150-background-ave.dat 0
Three type of mask have been generated:
- cspad-cxi49012-r0150-mask-badregs.dat - masks bad regions of 2x1 only
- cspad-cxi49012-r0150-mask-bkgd.dat - masks the high background central region
- cspad-cxi49012-r0150-mask-rects.dat - rectangular mask of the high background central region
Initial mask for noisy pixels
Use the first 50 events to define the noisy pixel:
[ImgAlgos.CSPadArrNoise] source = DetInfo(CxiDs1.0:Cspad.0) key = calibrated fracfile = ana-cxi49012/cspad-cxi49012-r0150-noise-frac.dat maskfile = ana-cxi49012/cspad-cxi49012-r0150-noise-mask.dat print_bits = 255 rmin = 3 dr = 1 SoNThr = 3 frac_noisy_imgs = 0.15
Array, image, and spectrum of fraction of noisy frames:
Array and image of the mask for noisy pixels:
Run peak finder CSPadArrPeakFinder
[psana] files = \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s00-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s01-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s02-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s03-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s04-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s05-c00.xtc #skip-events = 1000 #events = 23 modules = cspad_mod.CsPadCalib \ ImgAlgos.CSPadMaskApply \ ImgAlgos.CSPadArrPeakFinder # ImgAlgos.CSPadArrSaveInFile [cspad_mod.CsPadCalib] inputKey = outputKey = calibrated doPedestals = yes doPixelStatus = no doCommonMode = no [ImgAlgos.CSPadMaskApply] source = DetInfo(CxiDs1.0:Cspad.0) inkey = calibrated outkey = masked_arr mask_fname = ana-cxi49012/cspad-cxi49012-r0150-mask-badregs.dat #mask_fname = ana-cxi49012/cspad-cxi49012-r0150-mask-bkgd.dat #mask_fname = ana-cxi49012/cspad-cxi49012-r0150-mask-rects.dat masked_amp = 8 print_bits = 1 mask_control_bits = 15 [ImgAlgos.CSPadArrPeakFinder] source = DetInfo(CxiDs1.0:Cspad.0) key = masked_arr key_peaks_out = peaks hot_pix_mask_inp_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask.dat hot_pix_mask_out_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask-out.dat frac_noisy_evts_file = ana-cxi49012/cspad-cxi49012-r0150-noise-frac.dat evt_file_out = tmp/cspad-ev- rmin = 3 dr = 1 SoNThr = 3 frac_noisy_imgs = 0.1 peak_npix_min = 4 peak_npix_max = 25 peak_amp_tot_thr = 100. event_npeak_min = 10 event_amp_tot_thr = 1000. nevents_mask_update = 100 nevents_mask_accum = 50 selection_mode = SELECTION_ON out_file_bits = 15 print_bits = 512
[info:TimeInterval::startTime] Start time: 2012-06-13 10:01:09 and 837008000 nsec [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 1000 NHits: 71 ( 7.10%) Time: 473.930 sec ( 2.110 fps) NFrames: 2000 NHits: 71 ( 3.55%) Time: 918.985 sec ( 2.176 fps) NFrames: 3000 NHits: 102 ( 3.40%) Time: 1376.008 sec ( 2.180 fps) NFrames: 4000 NHits: 102 ( 2.55%) Time: 1822.940 sec ( 2.194 fps) NFrames: 5000 NHits: 120 ( 2.40%) Time: 2290.219 sec ( 2.183 fps) NFrames: 6000 NHits: 182 ( 3.03%) Time: 2757.514 sec ( 2.176 fps) NFrames: 7000 NHits: 311 ( 4.44%) Time: 3248.353 sec ( 2.155 fps) NFrames: 8000 NHits: 357 ( 4.46%) Time: 3806.922 sec ( 2.101 fps) NFrames: 9000 NHits: 686 ( 7.62%) Time: 4366.899 sec ( 2.061 fps) ===== JOB SUMMARY ===== NFrames: 9534 NHits: 686 ( 7.20%) Time: 4607.013 sec ( 2.069 fps) [info:TimeInterval::stopTime] Time to process 9534 events is 4607.01 sec, or 0.483 sec/event
Needs to be compared with Cheetah selection-a log file:
/reg/d/psdm/cxi/cxi49012/scratch/hdf5/r0150-a/log.txt
>-------- Start of job --------< nFrames: 1000, nHits: 20 (2.00%), recentHits: 20 (2.00%), wallTime: 0hr 0min 51sec (19.6 fps) nFrames: 2000, nHits: 20 (1.00%), recentHits: 0 (0.00%), wallTime: 0hr 2min 4sec (16.1 fps) nFrames: 3000, nHits: 43 (1.43%), recentHits: 23 (2.30%), wallTime: 0hr 3min 2sec (16.5 fps) nFrames: 4000, nHits: 43 (1.08%), recentHits: 0 (0.00%), wallTime: 0hr 4min 1sec (16.6 fps) nFrames: 5000, nHits: 56 (1.12%), recentHits: 13 (1.30%), wallTime: 0hr 4min 56sec (16.9 fps) nFrames: 6000, nHits: 109 (1.82%), recentHits: 53 (5.30%), wallTime: 0hr 5min 43sec (17.5 fps) nFrames: 7000, nHits: 221 (3.16%), recentHits: 112 (11.20%), wallTime: 0hr 6min 53sec (16.9 fps) nFrames: 8000, nHits: 254 (3.18%), recentHits: 33 (3.30%), wallTime: 0hr 7min 47sec (17.1 fps) nFrames: 9000, nHits: 528 (5.87%), recentHits: 274 (27.40%), wallTime: 0hr 9min 4sec (16.5 fps) >-------- End of job --------< End time: Mon Feb 6 20:28:14 2012 Elapsed time: 0hr 9min 43sec Frames processed: 9534 Number of hits: 528 Average hit rate: 5.54 % Average frame rate: 16.35 fps Average data rate: 71.65 MB/sec Average photon energy: 5989.52 eV Photon energy sigma: 6.60 eV
2012-06-14 Comparison of selected events in PSANA vs Cheetah selection-a
- psana file name:
cspad-ev-000443-r0150-2012-02-06-194723.342595531-peaks.txt
- fiducials are missing in psana data. - Cheetah file name:
LCLS_2012_Feb06_r0150_194740_11456_cspad.h5
- nanoseconds are missing in Cheetah-based analysis. - We need external info to check matching of selected events. Command
pyxtcreader <list-of-xtc-files> | grep L1Accept > file-time-to-fiducial
generates a list of frame headers, which contain both the timestamp and fiducial, liketime=1328586439.927456987, fiducials=63221,...
.
Matching algorithm use 3 lists:
- list of records generated by the
pyxtcreader
command, - list of files with selected frames in Cheetah,
- list of files with selected frames in psana.
Python script loops over all frames of particular run (pyxtcreader
records) and check if the Cheetah and/or PSANA have selected particular frame. For Cheetah it checks
- date: YYYYMMDD,
- time: HHMMSS,
- fiducials.
For PSANA it checks
- date: YYYYMMDD,
- time: HHMMSS,
- nanoseconds.
The portion of the matching algorithm log file:Frame: 8928 1328586514 sec 25313196 nsec fiducials: 89906 HHMMSS: 194834 date: 20120206 isInCheetah: False isInPSANA: False Frame: 8929 1328586514 sec 33643630 nsec fiducials: 89909 HHMMSS: 194834 date: 20120206 isInCheetah: False isInPSANA: True Frame: 8930 1328586514 sec 41973933 nsec fiducials: 89912 HHMMSS: 194834 date: 20120206 isInCheetah: True isInPSANA: True Frame: 8931 1328586514 sec 50302898 nsec fiducials: 89915 HHMMSS: 194834 date: 20120206 isInCheetah: False isInPSANA: False Frame: 8932 1328586514 sec 58634037 nsec fiducials: 89918 HHMMSS: 194834 date: 20120206 isInCheetah: True isInPSANA: True Frame: 8933 1328586514 sec 66961663 nsec fiducials: 89921 HHMMSS: 194834 date: 20120206 isInCheetah: False isInPSANA: False Frame: 8934 1328586514 sec 75294182 nsec fiducials: 89924 HHMMSS: 194834 date: 20120206 isInCheetah: True isInPSANA: True Frame: 8935 1328586514 sec 83622351 nsec fiducials: 89927 HHMMSS: 194834 date: 20120206 isInCheetah: False isInPSANA: False Frame: 8936 1328586514 sec 91952812 nsec fiducials: 89930 HHMMSS: 194834 date: 20120206 isInCheetah: False isInPSANA: False Frame: 8937 1328586514 sec 100280993 nsec fiducials: 89933 HHMMSS: 194834 date: 20120206 isInCheetah: True isInPSANA: True
Finally, we count for statistics of different possibilities using status of boolean variables isInCheetah
and isInPSANA
.
Result for all counters in r0150:
Total number of frames in file : 9534 Total number of hits in Cheetah : 528 Total number of hits in PSANA : 686 Non-found hits in both : 8831 Found in both : 511 Found in PSANA only : 175 Found in Cheetah only : 17
Summary
For current set of parameters
- ImgAlgos.CSPadArrPeakFinder selects 7.20% of hits
- Cheetah saves 5.87% of hits
- PSANA missed 17 and selected extra 175 events versus Cheetah for more than 500 selected frames.
Selected events
ev-007713:
ev-008944:
Events selected by Cheetah only
Most of 17 events, selected by Cheetah only, were not selected by the CSPadArrPeakFinder
because 9 peaks were found only in stead of required 10.
Example of events:
2012-06-18 Change selection from 10 peaks to 9
[info:TimeInterval::startTime] Start time: 2012-06-18 16:41:01 and 651642000 nsec [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 1000 NHits: 84 ( 8.40%) Time: 478.098 sec ( 2.092 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 2000 NHits: 84 ( 4.20%) Time: 926.035 sec ( 2.160 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 3000 NHits: 116 ( 3.87%) Time: 1383.505 sec ( 2.168 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 4000 NHits: 116 ( 2.90%) Time: 1832.200 sec ( 2.183 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 5000 NHits: 135 ( 2.70%) Time: 2285.611 sec ( 2.188 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 6000 NHits: 203 ( 3.38%) Time: 2754.092 sec ( 2.179 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 7000 NHits: 340 ( 4.86%) Time: 3247.141 sec ( 2.156 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 8000 NHits: 390 ( 4.88%) Time: 3712.526 sec ( 2.155 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 9000 NHits: 736 ( 8.18%) Time: 4278.251 sec ( 2.104 fps) [info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY ===== [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 9534 NHits: 736 ( 7.72%) Time: 4516.692 sec ( 2.111 fps) [info:TimeInterval::stopTime] Time to process 9534 events is 4516.69 sec, or 0.473746 sec/event
Comparison of selected events in PSANA vs Cheetah selection-a
Result for all counters:
Total number of frames : 9534 Total number of hits in Cheetah : 528 Total number of hits in PSANA : 736 Non-found hits in both : 8788 Found in both : 518 Found in PSANA only : 218 Found in Cheetah only : 10
10 Events selected by Cheetah only
Frame: 6084 1328586490 sec 334312132 nsec fiducials: 81374 HHMMSS: 194810 date: 20120206 isInCheetah: True isInPSANA: False Frame: 6504 1328586493 sec 832694890 nsec fiducials: 82634 HHMMSS: 194813 date: 20120206 isInCheetah: True isInPSANA: False Frame: 6553 1328586494 sec 240845567 nsec fiducials: 82781 HHMMSS: 194814 date: 20120206 isInCheetah: True isInPSANA: False Frame: 6851 1328586496 sec 723213642 nsec fiducials: 83675 HHMMSS: 194816 date: 20120206 isInCheetah: True isInPSANA: False Frame: 6974 1328586497 sec 747879423 nsec fiducials: 84044 HHMMSS: 194817 date: 20120206 isInCheetah: True isInPSANA: False Frame: 6994 1328586497 sec 914490230 nsec fiducials: 84104 HHMMSS: 194817 date: 20120206 isInCheetah: True isInPSANA: False Frame: 7911 1328586505 sec 553606864 nsec fiducials: 86855 HHMMSS: 194825 date: 20120206 isInCheetah: True isInPSANA: False Frame: 8440 1328586509 sec 960190204 nsec fiducials: 88442 HHMMSS: 194829 date: 20120206 isInCheetah: True isInPSANA: False Frame: 8449 1328586510 sec 35161608 nsec fiducials: 88469 HHMMSS: 194830 date: 20120206 isInCheetah: True isInPSANA: False Frame: 8452 1328586510 sec 60152788 nsec fiducials: 88478 HHMMSS: 194830 date: 20120206 isInCheetah: True isInPSANA: False
2012-06-21 Speed-up processing using multithreading
Log file shows that speed of this algorithm is about 2fps, that is slow. This speed is defined by the median evaluation for each pixel of the image over about 50 remote neighbor pixels. PSANA framework supplies data for each quad. For each section of the quad the median algorithm does independent processing, that can be done in parallel in multithreading mode. We use the OpenMP procedure
#pragma omp parallel for
just before the for
loop over all available CSPAD sections of the quad. For the thread-safe processing the code inside the for
loop is modified in order to isolate all variables and output data in different threads. Comparative difference between speed of the single-thread and multi-treading modes is shown below.
In multithreading mode:
[info:ImgAlgos.CSPadArrPeakFinder] NFrames: 1000 NHits: 163 ( 16.30%) Time: 160.467 sec ( 6.232 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 2000 NHits: 163 ( 8.15%) Time: 279.290 sec ( 7.161 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 3000 NHits: 203 ( 6.77%) Time: 400.101 sec ( 7.498 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 4000 NHits: 203 ( 5.07%) Time: 522.545 sec ( 7.655 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 5000 NHits: 236 ( 4.72%) Time: 648.397 sec ( 7.711 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 6000 NHits: 359 ( 5.98%) Time: 797.908 sec ( 7.520 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 7000 NHits: 593 ( 8.47%) Time: 970.897 sec ( 7.210 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 8000 NHits: 689 ( 8.61%) Time: 1116.850 sec ( 7.163 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 9000 NHits: 1165 ( 12.94%) Time: 1372.103 sec ( 6.559 fps) [info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY ===== [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 9534 NHits: 1165 ( 12.22%) Time: 1451.644 sec ( 6.568 fps) [info:TimeInterval::stopTime] Time to process 9534 events is 1451.64 sec, or 0.15226 sec/event dubrovin@psana0204
Snapshots from top
:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 31250 dubrovin 16 0 1126m 250m 16m R 426.1 3.1 83:20.34 psana 31250 dubrovin 16 0 998m 212m 16m R 515.2 2.7 93:06.69 psana 31250 dubrovin 16 0 1062m 239m 16m R 278.5 3.0 94:49.83 psana 31250 dubrovin 16 0 1126m 259m 16m R 440.5 3.2 100:26.43 psana
Single-thread mode
[info:ImgAlgos.CSPadArrPeakFinder] NFrames: 1000 NHits: 163 ( 16.30%) Time: 503.288 sec ( 1.987 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 2000 NHits: 163 ( 8.15%) Time: 949.256 sec ( 2.107 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 3000 NHits: 203 ( 6.77%) Time: 1408.933 sec ( 2.129 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 4000 NHits: 203 ( 5.07%) Time: 1855.885 sec ( 2.155 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 5000 NHits: 236 ( 4.72%) Time: 2312.426 sec ( 2.162 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 6000 NHits: 359 ( 5.98%) Time: 2798.862 sec ( 2.144 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 7000 NHits: 593 ( 8.47%) Time: 3321.919 sec ( 2.107 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 8000 NHits: 689 ( 8.61%) Time: 3798.451 sec ( 2.106 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 9000 NHits: 1165 ( 12.94%) Time: 4406.237 sec ( 2.043 fps) [info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY ===== [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 9534 NHits: 1165 ( 12.22%) Time: 4644.862 sec ( 2.053 fps) [info:TimeInterval::stopTime] Time to process 9534 events is 4644.86 sec, or 0.487189 sec/event dubrovin@psana0204
Snapshots from top
:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 31736 dubrovin 18 0 1088m 269m 16m R 102.2 3.4 26:18.80 psana 31736 dubrovin 18 0 1024m 249m 16m R 104.0 3.1 28:00.43 psana 31736 dubrovin 17 0 1024m 250m 16m R 99.0 3.1 44:54.80 psana 31736 dubrovin 19 0 1088m 269m 16m R 103.5 3.4 48:24.64 psana
Comparison of selected events in PSANA vs Cheetah selection-a
Result for all counters:
Total number of frames : 9534 Total number of hits in Cheetah : 528 Total number of hits in PSANA : 1165 Non-found hits in both : 8368 Found in both : 527 Found in PSANA only : 638 Found in Cheetah only : 1
- In this version PSANA missed 1 event only, that is 0.01%.
- Parameters of the
CSPadArrPeakFinder
have not been changed. - Code has been changed for thread-safe operations.
- It may happen due to persistence in member-data, which was eliminated in new code.
Summary for multithreading
- Great success is acheived in isolation of data and parameters between threads.
- All crashes disappeared
- Single- and multi-threading modes select exactly the same set of events
- Multithreading allows to speed-up the
CSPadArrPeakFinder
in 6.568fps / 2.053fps about 3 times.
2012-06-27 Tuning of parameters
After meeting with Anton the peak finder parameters are changed;
SoNThr=3
is changed toSoNThr_noise=3
- Add
SoNThr_signal=8
frac_noisy_imgs=0.9
(it was 0.1)- remove amplitude thresholds in ADU
peak_amp_tot_thr=0.
andevent_amp_tot_thr=0.
- Add
event_npeak_max=10000
- Changed
nevents_mask_update=0
(it was 100) so the mask of noisy pixels is updated every 50 events without any gap. - Add
peak_SoN_thr
- threshold on S/N counted over peak connected pixels.
Selection V5:
[ImgAlgos.CSPadArrPeakFinder] source = DetInfo(CxiDs1.0:Cspad.0) key = masked_arr key_peaks_out = peaks #key_signal_out = signal-arr #hot_pix_mask_inp_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask.dat hot_pix_mask_out_file = ana-cxi49012/cspad-cxi49012-r0150-noise-mask-out.dat frac_noisy_evts_file = ana-cxi49012/cspad-cxi49012-r0150-noise-frac.dat evt_file_out = tmp/cspad-ev- rmin = 3 dr = 1 SoNThr_noise = 3 SoNThr_signal = 7 frac_noisy_imgs = 0.9 peak_npix_min = 3 peak_npix_max = 20 peak_amp_tot_thr = 0. peak_SoN_thr = 7. event_npeak_min = 5 event_npeak_max = 10000 event_amp_tot_thr = 0. nevents_mask_update = 0 nevents_mask_accum = 50 selection_mode = SELECTION_ON out_file_bits = 12 print_bits = 577
[info:ImgAlgos.CSPadArrPeakFinder] NFrames: 1000 NHits: 22 ( 2.20%) Time: 137.074 sec ( 7.295 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 2000 NHits: 22 ( 1.10%) Time: 292.825 sec ( 6.830 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 3000 NHits: 50 ( 1.67%) Time: 453.272 sec ( 6.619 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 4000 NHits: 50 ( 1.25%) Time: 605.666 sec ( 6.604 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 5000 NHits: 70 ( 1.40%) Time: 758.880 sec ( 6.589 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 6000 NHits: 153 ( 2.55%) Time: 919.956 sec ( 6.522 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 7000 NHits: 329 ( 4.70%) Time: 1099.423 sec ( 6.367 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 8000 NHits: 390 ( 4.88%) Time: 1258.966 sec ( 6.354 fps) [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 9000 NHits: 744 ( 8.27%) Time: 1489.109 sec ( 6.044 fps) [info:ImgAlgos.CSPadArrPeakFinder] ===== JOB SUMMARY ===== [info:ImgAlgos.CSPadArrPeakFinder] NFrames: 9534 NHits: 744 ( 7.80%) Time: 1569.498 sec ( 6.075 fps) [info:TimeInterval::stopTime] Time to process 9534 events is 1569.5 sec, or 0.164621 sec/event Mask of noisy: [info:ImgAlgos.CSPadArrPeakFinder] Event:9499 Collected for mask update:50 Statistics: Nnoisy:147 Ntotal:2296960 Nnoisy/Ntotal pixels:6.39976e-05
Comparison with Cheetah
Cheetah selection version : a : HG0 : kb Total number of frames : 9534 : 9534 : 9534 Total number of hits in Cheetah : 528 : 187 : 317 Total number of hits in PSANA : 744 : 744 : 744 Non-selected frames in both : 8783 : 8790 : 8785 Found in both : 521 : 187 : 312 Found in PSANA only : 223 : 557 : 432 Found in Cheetah only : 7 : 0 : 5
ev-006785-r0150-2012-02-06-194816.173410278:
cxi49012-r0150-selected-arr-194833.467189379:
Summary overall
- Peak finder / filter is ready
- All features are implemented
- It is demonstrated that it finds peaks correctly, depending on selection conditions.
- It is demonstrated that for similar selection parameters with Cheetah the mismatch in number of selected events does not exceed 1%.
- OpenMP multithreading for parallel processing of sectors in quads 3-times speed-up calculations.
F2. Peak finding algorithm features:
- ImgAlgos.CSPadMaskApply - is used to apply constant mask on 2x1 edges and bad regions.
- Noisy mask is generated dynamically over each 50 events. Initial noisy-pixel mask can be loaded from file, but is not loaded, like in Cheetah...
- ADC thresholds can be applied on pixel and total peak amplitudes, but are not applied, like in Cheetah...
- Median algorithm use ring-shaped neighbor pixels to calculate the level of background and noise fluctuations.
Cheetah uses square region of the same size. Actually, the difference is in 4 pixels:Index vector size: 24 0 0 0 0 1 0 0 0 0 0 0 1 1 1 1 1 0 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 1 1 0 0 + 0 0 1 1 0 1 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0
- S/N thresholds can be applied on individual pixels or entire peak.
2012-07-09 Module ImgAlgos::CSPadArrPeakAnalysis
See module description in Module ImgAlgos::CSPadArrPeakAnalysis
Example of the psana configuration file:
[psana] files = \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s00-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s01-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s02-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s03-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s04-c00.xtc \ /reg/d/psdm/cxi/cxi49012/xtc/e158-r0150-s05-c00.xtc modules = cspad_mod.CsPadCalib \ ImgAlgos.CSPadMaskApply \ ImgAlgos.CSPadArrPeakFinder \ ImgAlgos.CSPadArrPeakAnalysis [cspad_mod.CsPadCalib] inputKey = outputKey = calibrated doPedestals = yes doPixelStatus = no doCommonMode = no [ImgAlgos.CSPadMaskApply] source = DetInfo(CxiDs1.0:Cspad.0) inkey = calibrated outkey = masked_arr mask_fname = ana-cxi49012/cspad-cxi49012-r0150-mask-badregs.dat masked_amp = -1 print_bits = 5 mask_control_bits = 15 [ImgAlgos.CSPadArrPeakFinder] source = DetInfo(CxiDs1.0:Cspad.0) key = masked_arr key_peaks_out = peaks evt_file_out = tmp/cspad-ev- rmin = 3 dr = 1 SoNThr_noise = 3 SoNThr_signal = 7 frac_noisy_imgs = 0.9 peak_npix_min = 3 peak_npix_max = 20 peak_amp_tot_thr = 0. peak_SoN_thr = 7. event_npeak_min = 5 event_npeak_max = 10000 event_amp_tot_thr = 0. nevents_mask_update = 0 nevents_mask_accum = 50 selection_mode = SELECTION_OFF out_file_bits = 0 print_bits = 513 [ImgAlgos.CSPadArrPeakAnalysis] source = DetInfo(CxiDs1.0:Cspad.0) key = peaks print_bits = 7 fname_root = file.root
In order to collect statistics about all peaks (not only for selected events), the selection_mode
parameter is set to SELECTION_OFF
.
After execution of this script in psana the file.root
containing histogram(s) and ntuple(s) will be produced. Then, auxiliary script in root, running by the command root -q -f proc.C
produces the plots with histograms:
These distributions were obtained w/o selection and represent background events mainly:
Number of peaks
(in event) - distribution shows that large fraction of events has ?5 peaks;Number of pixels in peak
is ?9 in most cases;atot
- total signal amplitude in the peak region (S);btot
- total background in the peak region;noise
- total noise in the peak region (N);son
- S/N ratio;Quad
- population of peaks over quads;Sect
- population of peaks over sectors;col
- population of peaks over columns;row
- population of peaks over rows;sigma_c
- rms size of the peak region in columns;sigma_r
- rms size of the peak region in rows.
In case if the selection_mode
parameter is set to SELECTION_ON
the distributions represent peaks for selected signal events:
2012-07-12 Test of selected and re-selected events
There may be a potential problem with a list of selected events; when the same filter use the xtc-file of re-written events, filter may find a different number of selected events... This may happen due to changing conditions.
There are at least two reasons for changing conditions:
- hot pixel mask is generated dynamically under control of the
nevents_mask_update
andnevents_mask_accum
parameters. The mask itself depends on a set of events and is different for original and selected set of events. I have tested for this effect and did not find any difference in selected number of events for cxi49012-r0150 (9534 events total and 740-selected). In order to get rid of this effect, the dynamic mask update can be switched off by settingnevents_mask_update
to the large value. Constant hot pixel mask still can be loaded from thehot_pix_mask_inp_file
. - if the input xtc file name has a wrong structure (for example: e158-r0150-s00-c00.xtc is right, but e158-r0150.xtc is wrong) the name of experiment can not be defined correctly and the standard
calib
directory can not be found as well. In this case the calibration parameters will be different that gives significant difference in conditions of the peak finding algorithm. I have observed the 25% difference in the number of selected events.