Contents

2013-11-09 Arianna Gleason, Eric Christophe Galtier, meca6013

Q: Need in dark calibration files:

Shift 1: For dark run=29; range=30-102
Shift 2: For dark run=103; range=105-182
Shift 3: For dark run=184, range=185-249
Shift 4: For dark run=252, range=254-326
Shift 5: For dark run=327, range=328-397

A: New files are deployed using Calibration Management Tool

2013-11-08 Cindy Bolme, Eric Christophe Galtier, meca1113

Q: Need in dark runs for correct ranges

> For dark run=44; range=1-43
> For dark run=45; range=46-78
> For dark run=79, range=80-221
> For dark run=222, range=223-372
> For dark run=373, range=374-527
> For dark run=528, range=529-659

A: Calibration directory is modified:

  • Old files from /reg/d/psdm/MEC/meca1113/calib/*/*/pedestals/* were removed.
    Their copy is preserved in ~/LCLS/CSPadAlignment-v01/calib-meca1113-2014-01-08-copy

  • New files were generated using Calibration Management Tool
    • run 44 does not have cspad data and events
    • dark run is included in run range 
    • 1-end.data is also added, generated from run 45.

 

2013-12-03 Eric Christophe Galtier, meca1113

Q: experiment: meca1113, run: 528 - can't write in calib, need in pedestals

A:

  • command % getfacl /reg/d/psdm/mec/meca1113/calib
    gave me the mask
    mask::r-x
    which blocked writing permission for all groups.
  •  Using calibman I have generated pedestals in
    /reg/d/psdm/mec/meca1113/calib/CsPad2x2::CalibV1/MecTargetChamber.0:Cspad2x2.*/pedestals/528-end.data
    /reg/d/psdm/mec/meca1113/calib/CsPad::CalibV1/MecTargetChamber.0:Cspad.0/pedestals/528-end.data

 

2013-11-08 Cindy Bolme, Eric Christophe Galtier, meca1113

Q: We are converting the cspad data into tif images from the xtcexplorer using the pyana_image module.  The tiff images are saving as 8-bit files. Could you please change it to save 16-bit tiffs?

A: Examples for meca1113

Comment from Eric:

Thanks a lot! Now indeed I manage to extract data from princeton camera devices saved in previous experiment. This is all great :) Thank you very much. I'm also really surprised how easy it is to setup the psana package to have access to all the features. I gave quick instructions to the users and it was enough for them to be able to extract all the information during the experiment. They still need to have the environment imported when they log in but this is so little ^^ All this is very cool :) I will try to make a script that will be generic for data extraction at MEC handling Princeton, Opal and CSPAD cameras so that it will be ready for the users. I will try that on old runs that was looking like that.

 

2013-10-22 Henrik Till Lemke

Q: I got some coordinates using:

X, Y = coord.get_cspad_pix_coordinate_arrays_um  (config=None), but those are now in the (4, 8, 185, 388) instead of the (32, 185, 388) like in the data. Is it just the first dimension flattened?

A:  If you are sure that all 2x1 in data are there, then, just change shape: arr.shape = (32, 185, 388) If you need in arrays shaped as real data you need to use method from PyCSPadImage.CSPADPixCoords.py

x_arr_as_data, y_arr_as_data = pixccords.get_cspad_pix_arrays_shaped_by_config (x_arr, y_arr, config)

which uses config object. Or, use conversion method directly from PyCSPadImage.CSPadConfigPars.py

arr_as_data = config.getCSPadPixArrayShapedAsData(arr_in)

See for details interface description in  CSPAD pixel coordinates and image producer in Python

2013-10-22 Brendan Murphy, amo74213

Use for examples amo01509-r0125

Q: Need in a set of modules for Acqiris in psana.
A few generic modules, which would be very useful to many users, would do the following:

1) Export the Acqiris configuration data from a run and save it as a file. This seems to be done by psana_examples.DumpAcqiris,
but I don't know how to pipe the output into a data file using the psana configuration file.

2) Average multiple Acqiris traces from a run into one string of doubles, which is saved as a file. Again, psana_examples.DumpAcqTdc
appears to be doing part of this, but I'm not sure how to save or manipulate the output.

3) Add or subtract a string of doubles (for example, the results of (2) for traces with no "hits") from an Acqiris trace.

4) Perform basic constant fraction discrimination on a string of doubles (for example, an Acqiris trace.) The ringing-subtraction can be skipped.
Save the CFD "hit" times, which may be many per shot, along with event data (maybe FEE data, or other info) in a data file.

Previously we had all these functions together in one, large, clumsy Myana.cc file. Because all the variables were accessible
in the file, it was relatively easy to add or change the routine. With psana, I don't see how to perform the same actions easily,
but I agree with your idea for multiple generic modules. The result should be much more flexible.

Please let me know if you have more questions, I think that the results of this effort can be quite valuable to any LCLS users who use
the Acqiris.

A: Psana Module Catalog: Modules AcqirisArrProducer, AcqirisAverage, and AcqirisCalib.

See modules ImgAlgos::AcqirisArrProducer, AcqirisAverage, and AcqirisCalib in

 

2012-10-17 Marcin Sikorski, xcs

Q: For 49 to 51 in xcs72913
Could you please help me with reading XPP Cspad's (both small and cspad 140) images in pyana? How to extract 2D image for every event() ? I tried to follow confluence page but I failed.
A: See description and examples of the package pyimgalgos:

Until new release (> ana-0.10.2) is generated,
you will need to update this package in your release directory:

addpkg py_img_algos HEAD

2013-10-17 Andreas Schropp,

Q1: since quite some time I had a look on the data extraction from CSPads
again and was wondering about the parameters that are written into the
configuration file 0-end.data, e.g. for CSPad 1:

  198.05 198.42
   95.10 307.54
    0.09   0.03

Could you shortly explain what these numbers are about?

A1,2: See for details

  • CSPAD2x2 Alignment
  • CSPAD Geometry and Alignment
    Q2: If you have a considerable amount of tilting involved. How do you interpolate the numbers in psana? Just linear?
    A2: We use regular 2d rotation matrix. Z coordinate is constant (by current precision).

Q2: Could you shortly explain what these numbers are about? Thanks,
Andreas

ps. If you have a considerable amount of tilting involved. How do you
interpolate the numbers in psana? Just linear?

2013-10-09 Maxence GAUTHIER, meca6113

Q: Thanks for the example. I am trying to make it work, but seems I do not have the permission (neither Luke Fletcher who is the PI of our experiment) to put a calibration data file in the calib file. Is it possible to have it ? (or only Luke). I also need to modify the alignment and tilt files since our cspad are not cspad 1 2 3 as recorded in CSPAD Geometry and Alignment in local space but 1 5 and 2 corresponding, respectively to cspad 1 2 and 3 in our raw data.

A: (Igor) Need to change Acess Control List (ACL):

% sudo -u psdatmgr chmod -R 0770 /reg/d/.../calib

In order to set writing permission, all pars needs to be set as for res directory, generated autamatically by Wilko:

% getfacl /reg/d/psdm/mec/meca6113/res
getfacl: Removing leading '/' from absolute path names
# file: reg/data/ana14/mec/meca6113/res
# owner: psdatmgr
# group: ps-data
# flags: -s-
user::rwx
group::r-x
group:ps-data:r-x
group:ps-mec:rwx
group:meca6113:rwx
mask::rwx
other::---
default:user::rwx
default:group::r-x
default:group:ps-data:r-x
default:group:ps-mec:rwx
default:group:meca6113:rwx
default:mask::rwx
default:other::---

To set each parameter use command:

% sudo -u psdatmgr setfacl -R -m 'group::rwx' /reg/d/.../calib

Igor set:

getfacl /reg/d/psdm/mec/meca6113/calib
getfacl: Removing leading '/' from absolute path names
# file: reg/data/ana14/mec/meca6113/calib
# owner: psdatmgr
# group: ps-data
user::rwx
group::rwx
group:ps-mec:rwx
group:meca6113:rwx
mask::rwx
other::---
default:user::rwx
default:group::r-x
default:group:ps-mec:rwx
default:group:meca6113:rwx
default:mask::rwx
default:other::---

2013-10-09 Maxence GAUTHIER, Yin, Jing, mmo@ualberta.ca, zchen2@ualberta.ca, meca6113

Q: How to get images in tiff format
A: In Examples for meca6113.

2013-10-09 Cornelius Gati, MEC

Q: How to speedup readout from hdf5
A: Increase internal h5py cash to 128MB

 

2013-10-01 Brendan Murphy, amo74213

Q1: Need in psana modules:
1) Threshold peak finding algorithm: if pixel intensity exceeds threshold - set pixel to 1
2) Blob finding algorithm : find blobs and set central pixel to 1 or N (number of pixels in the blob)
A: Add psana module psana - Module Catalog#Module ImgAlgos::ImgHitFinder

2013-09-24 Cornelius Gati, MEC

Q1: How to start Matlab
A1: See: Computing (including Analysis)#Running the Analysis
For example:

ssh psananeh;
/reg/common/package/scripts/matlic;

it prints:

dubrovin@psana0105 /reg/common/package/scripts/matlic
Available MATLAB licenses: psana0101 psana0102 psana0103 psana0104 psana0105 psana0106 psana0206

Q2: How to fit data in python or matlab
A2: See references and example in Fit to data.

2013-09-23 Brendan Murphy, amo74213

Is it possible to get help with a psana code which will run hit/blob-finders directly on the XTC files? Maybe I can be a "guinea pig" tester for the new shared library release. Hopefully I can code slightly better than an actual guinea pig, but the difference may be marginal (smile).

A: There is a couple of peak finding algorithms in psana:
Psana Module Catalog - Old#Module ImgAlgos::ImgPeakFinder
Psana Module Catalog - Old#Module ImgAlgos::ImgPeakFinderAB
Which one should be used depends on images in your experiment.
In order to advice anything else I need to know more about your analysis.

2013-09-19 Dieter Ries, works with ultracold neutrons.

Q: HDF5 Explorer - Old This looks like a very useful tool to me, but unfortunately I couldn't find any information on if hdf5explorer is available outside of SLAC Is it released to the public somewhere? If yes, where can I find it and under which license is it released?
A: to get entire LCLS software release use instructions in
PSDM software distribution

If you prefer to use HDF5 and directly access data, then look at some prescriptions in
How to access HDF5 data from Python

The right way to perform LCLS analysis is to use PSANA:
Psana User Manual - Old
or its interactive version:
Interactive PSANA Study
Tutorial
which currently allows to get data both in C++ and Python code.

2013-09-18 Cornelius Gati, MEC

Q: How to get time synchronization between two arrays? Example in
How to access HDF5 data from Python#Example 4: Time-based syncronization of two datasets always prints 'Missing of syncronization'.
A: This example assumes that both arrays should have the same times, but some of measurements may be missing.

2013-07-31 Philipp Sperling

Q: I have forgotten my password. Could you help me how I can renew my password to my account.
A: Send e-mail to account-services@slac.stanford.edu

2013-07-30 Shaughnessy Brown

Q1: My module creates and writes to a .txt file. By default, this file is
located in the release directory; is it possible to specify a different location?
A1: use complete path, for example: /reg/neh/home/sbbrown/<further-path-to-your-subdirectory>/file.txt
Q2: How do I save a file to a local location.
A2: Use scp. Other options are described in Data exportation.

2013-07-30 Philip Hart

Q: How to calibrate pedestals of two (or more) cspad2x2 in pswana?
A: Psana Module Examples - Old#Example for module cspad_mod::CsPad2x2Pedestals

2013-07-16 Simon Lieu

Q: Could you please explain what EPICS and PV are?
EPICS - Experimental Physics and Industrial Control System
PV - Process Variables

These variables contain "meta-data" - data about experimental setup,
for example, stepping motor positions, beam stopper status,
detector positions, attenuator status, units of measurement, etc.

Access to EPICS variables in psana is described in
https://confluence.slac.stanford.edu/display/PCDS/Psana+User+Manual#PsanaUserManual-AccessingEPICSData

2013-06-28 Matthieu Chollet, xppi0113

Q: Another thing we noticed, is that over time the grabber take a lot of CPU memory. in the process we see Xorg close to 100% and the fan of the computer are fully on. They shutdown as soon as we close the grabber window.
A: In def paintEvent(self, e): the graphical window was permanently updated. Now it is updated during zoom operation only, when dynamical image of the zoom-rect needs to be shown.

2013-06-28 Matthieu Chollet, xppi0113

Q: when grabbing images the color scheme is not correct...
on xpp-control using xppeloggrabber
A: One more place where jpeg needs to be used in stead of ppm format;
in grabImage change ppm to jpeg:

   def grabImage(self):
         #fname = tempfile.NamedTemporaryFile(mode='r+b',suffix='.ppm')
        fname = tempfile.NamedTemporaryFile(mode='r+b',suffix='.jpeg')
        if( 0 == os.system('import -trim -frame -border %s' % (fname.name))) :

2013-06-22 Henrik Till Lemke Re:xpp65013

Q: Can't start
LogBookGrabber_qt.py using command:

 /reg/g/pcds/pds/grabber/bin/LogBookGrabber_qt.py -i AMO:0 -e xpp65013 -u xppopr -w https://pswww.slac.stanford.edu/ws-auth [pswww.slac.stanford.edu] -f config-pars.txt -c "ls -l"

A of Igor: use correct command

/reg/g/pcds/pds/grabber/bin/LogBookGrabber_qt.py \
-i XPP \
-e current \
-u xppopr \
-w https://pswww.slac.stanford.edu/ws-auth \
-f config-pars.txt \
-c "ls -l"

Q: Need to load any file.
A: Option "All files (*)" is added

Q: Problem with colors of grabbed images in ELog.
A: image was submitted in e-log as a png file. Now it is changed to jpg.

2013-06-13 Henrik Till Lemke Re:xpp66213

Q: CSPAD alignment does not work in evaluateCSPadPixCoordinate for exp: xpp66213, image data:run 150, dark: 149, using calib-xpp-2013-01-29.
A: Package PyCSPadImage is in stage of modification. Use earlier version V00-00-09 of this package. Because you do not use offline software release rules, it is necessary to do non-standard operations:
1. Get PyCSPadImage package content:

         newrel ana-current test-ana-0.9.11
         cd test-ana-0.9.11
         sit_setup
         addpkg PyCSPadImage V00-00-09

2. Copy files form  PyCSPadImage/src/*.py
to: ~lemke/mypy/ixppy/cspad/alignment/
and try to run your application again.

2013-05-14 Mariano Trigo, CXI

Q: How to generate CSPAD image from array, how to save image(s) in file etc.
Mariano works on specific analysis module which averages CSPAD array in bins, depending on external parameters. He wants to have arrays like
Nbins1750x1750 for data, background etc. The bin is defined by external variables.
A: Useful examples of code in Psana Module Catalog - Old: cspad_mod.CsPadCalib, ImgAlgos::ImgCalib, ImgAlgos::ImgSaveInFile, ImgAlgos::CSPadArrAverage, ImgAlgos::CSPadArrSaveInFile, ImgAlgos::CSPadMaskApply, ...

2013-05-13 Silke Nelson, xppi0412

Q: How to save something in root-style ntuples?
A: ROOT as a histogramming package for psana
Psana User Manual - Old#Histogramming Service

2013-05-05 Henrik Till Lemke, xpp

Q: LogBookGrabber , line 1388 class LocalParameter () : SyntaxError: invalid syntax
A: Fixed class LocalParameter, wrong version of python, missing PyQt4. Use:

setenv PYTHONPATH ${PYTHONPATH}:/reg/g/pcds/pds/grabber/lib/python2.7/site-packages

Q: am I really supposed to enter my password in real text as argument?
A: from 2013-05-06 the password option can be omitted.

2013-05-02 TJ Lane, cxi64813

Discussion about CSPAD geometry alignment parameters;

  • quality check of input parameters
  • conversion between psana and vector-style parameters

2013-04-18 Mariano Trigo,

Q: How to compile psana with openMP, like it is done with cspad peak finders?
A: In file SConscript in the package directory, for example: $SIT_RELDIR/ana-current/ImgAlgos/SConscript
The line:

standardSConscript(UTESTS=[])

should be replaced by:

standardSConscript( LIBS="gomp rt pthread", UTESTS=[], CCFLAGS="-fopenmp")

2013-03-01 Andrew Higginbotham, mec73313

Q: 1) pedestals are not subtracted in psana...
A: Follow to correct name conventions for calib directory:

calib/CsPad2x2::CalibV1/MecTargetChamber.0:Cspad2x2.1/pedestals/0-end.data

It was mistake in Mec...Chamber and 0-end.data name and extension.
For local calib directory use in psana cfg file:

[psana]
calib-dir = ./calib
OR:
calib-dir = /reg/neh/home3/hggnbthm/test-mec/calib

Q: 2) Run number is 0 if file has 1 event... -> Andy?

2013-02-28 Andrew Higginbotham, mec73313

Q: How to get TIFF images of CSPad2x2 ?
A: Use example to produce text image. Then use script ConvertTextImageToTiff.py to convert TXT to TIFF format.
If you need to subtract pedestals use Psana Module Catalog - Old#Module cspad_mod.CsPadCalib.

2013-02-20 Marc Messerschmidt, cxi

Q: could you check and maybe update the CSpad geometry files using powder
diffraction from cxi80410/run1458?
The pattern should look like the following:
A: Use constants from 2013-01-31 CSPad CxiDs1 alignment

2013-02-19 Marc Messerschmidt, cxi

Q: I see a lot of cicles in the average - I remember seeing these from a long time
ago, but forgot the reason. Can I get an average without these?

A: You use

[ImgAlgos.CSPadArrAverage]
key = peakfinder

the signal image after peakfinder. So, you observe artifacts of this algorithm... When the ring of surrounded pixels for any given central pixel goes through the real peak, average intensity for central pixel is biased in positive side, while the difference I(pix) - I(averaged-over-ring) is negative. Probably algorithm can be improved somehow.

2013-02-13 James Budarz, xpp56012

Q: I'm having trouble viewing our experiment's hdf5 files with hdfview. It
looks like the hdf5 libraries have something wrong, as the option to use
such files is greyed out. I get an error stating that my experiment's
files are not a compatible version.
The version of hdfview I'm using is located at
/reg/neh/operator/xppopr/hdfview/hdfview and an example file
is /reg/d/psdm/xpp/xpp56012/hdf5/xpp56012-r0186.h5.

A: Try HDF5 Explorer - Old

2013-02-13 Mtg with cxi guys.

Q: PSANA User suggestion - a few requests/suggestions about psana/CSPad/svn etc.

A:
In your paper "Suggestions for computing software development at LCLS/CXI", which we discussed last Wednesday, you have raised so many questions which can not be
answered in a single e-mail.
Anyway, I will try to clarify a few of discussed issues and send you
appropriate references to documentation.

Portability:
There is a document explaining of how you can import/install the LCLS offline packages on your own computer: PSDM software distribution

Extensibility:
This option existed for a long time, though it was not documented anywhere. So, I have composed a new confluence page devoted to this issue: Users' Software Repository
You are encouraged to work on shareable code. All necessary infrastructure exists, though it was not actively used so far...

Documentation and Transparency:
We are working hard on documentation in a mean time. If I will have a chance I will try to improve the cspad coordinate package docs, and
simplify code as well.

CSPad alignment central page is: CSPAD Alignment
I agree, it is not well-composed and self-explaining document and needs in improvement. All other referred pages can by used to clarify some details.

Moving to a Modern Scientific Software Model: SVN or GIT?
I am not going to argue too much on this issue.
Our team is happy with version of svn which we use.
It allows to support shared module library without manual intervention
and re-compilation of all packages any time when you changed something, like you may have in other systems, for example for myana or chittah.
I would only say that it would be nice to have so consistent system
when we had been working on HEP analysis...

Software visibility is a good idea for well collaborating team.
This is not a case for LCLS user groups; due to competition most of groups do not want to share their tools. So, central framework code is completely
visible. Users' code is your own business. But, as I said, infrastructure is available for a long time. It's almost the first time when someone express some
interest to this infrastructure... In particular, for CSPad we have developed a bunch of general purpose modules: Psana Module Catalog - Old
If you have more ideas or need help, please let us know.
In particular, I will be willing to (help) write modules for polarization, solid angle correction, and more detailed gain correction.

Automated quad shifting:
If I have a chance, I will try to implement automated quad shifting.

Parallel execution:
there is an example of parallel processing in module
ImgAlgos::CSPadArrPeakFinder, where all 2x1 are processed in parallel. Memory management in general is a nightmare in case of parallel calculations. It is
unlikely that it can be done automatically for any given module of psana or any other package. One has to work on specific parallel algorithm.
On the other hand, psana allows parallel processing for separate events.
Just submit several butch jobs.

Non-linear gain correction:
I discussed this issue with Philip Hart. It exists as a good idea, but calibration, maintenance, and parametrization issues are not well elaborated.
In your case I would suggest to use the flat field calibration with intensity equivalent to your rings and apply psana module cspad_mod.CsPadCalib: Psana Module Catalog - Old#Module cspadmod.CsPadCalib

2013-02-10 Chun Hong Yoon, cxi43312

Q: I am trying to write a pyana module that read a CsPad from xtc.
I would like to
1) read CsPad data from xtc
2) subtract dark
3) apply common mode correction
2) display the image
Do you have an example that I could run?
I couldn't find anything relevant on confluence.

A:Example which you need is in Psana Module Examples - Old#Examplefor Package CSPadPixCoords.
See description of psana modules in Psana Module Catalog - Old.

2013-01-31 Henrik Till Lemke, xpp

Q: Is there a way to receive x,y pixel coordinates for each pixel in a 32,185,388 matrix from just the calibration input, so independent of the calibration saved in a particular experiment folder and defining a particular run?
A: Add module CSPadPixCoordsWODB.py in PyCSPadImage package:
CSPad image producer in Python (DEPRICATED)

2013-01-31 Derek Anthony Mendez Jr, cxi64813

Q: Need in help in alignment of cspad ds1 for /reg/d/psdm/cxi/cxi64813/xtc/e243-r0058-s00-c00.xtc
A: Done:

2013-01-24 Henrik Till Lemke, xpp

Q: The XPP cspad is being rearranged today and we'll have a new geometry. Would you be able to produce an input for your software until end tomorrow? I guess the metrology measurement will become available late afternoon today.

A: Done:

2013-01-22 Derek Anthony Mendez Jr, cxi64813

Q: I was told to speak to you about the built-in CSPAD calibrations (centering, quad shifting, etc) implemented in psana. I would like to speak with you for maybe ten minutes
about how they work.
A: BEside our conversation, look at

2012-12-11 Ferguson, Kenneth Ramon, cxii0312

Q: Since beginning of December the ImgAlgos.CSPAddArrPeakFinder module get a 0% of hits, although it was ok earlier...
A: On November 30 in ana-0.7.7 Andy made changes in the code for all CSPad modules (string->Source, Module -> CSPadBaseModule, etc).
On Dec.11 Andy has fixed code. Use ana-current = ana-0.7.8

2012-11-15 Marc Messerschmidt, cxi

Q: do you have a CSPad2x2 gain correction built into the hdf5explorer?
A: hdf5explorer has a gain correction for entire image (not for CSPad array).
Use averaging for flat field run, then go to What and how to display to tab Gain Corr to Make button and check the box Apply gain correction.

2012-11-09 Stefan P. Moeller, sxrcom11

Q: hdf5explorer does not work with correlations for new hdf5 files.
A: Tree problems were revealed in code and fixed:

  • names of datasets with spaces: these names were saved correctly in hdf5expconfig file, but only the part of the name by the 1st space was retreaved for the next session;
  • names of datasets were saved in hdf5expconfig file, but were not available for another hdf5 file. Apply try-except protection.
  • in GUIWhatToDisplay the self.indTabOpen was not necessarily initialized.
    These bugs are fixed since release ana-0.7.2.
    How to get updated package is explained in
    HDF5 Explorer - Old#HDF5Explorer How to run this program

2012-11-08 Marc Messerschmidt, cxi

Q: CSPad2x2 pedestals are not calculated with CSPadArrAverage module.
A: CSPad2x2 is not implemented in CSPadArrAverage module.
Use module CsPad2x2Pedestals: Psana Module Catalog - Old#Module cspadmod.CsPad2x2Pedestals

2012-10-17 David McGonegle, MEC

Q: Pyana gives:

OSError: [Errno 2] No such file or directory:
'/reg/d/psdm/MEC/mec62812/calib/CsPad::CalibV1/MecTargetChamber.0:Cspad2x2.3/pedestals'

How we can extract the images from a dark shot run in order to create the pedestal file.

A: Use commands:

cd <your-favorite-directory>
sit_setup
psana -m cspad_mod.CsPadPedestals <dark-image-files.xtc>

Please see for details:
Psana Module Catalog - Old#Module cspad_mod.CsPadPedestals

2012-09-15 Marc Messerschmidt, cxi

Q: Why the ImgAlgos.CSPadArrPeakFinder generates odd messages like

[[infoinfo::ImgAlgos.CSPadArrPeakFinder] ImgAlgos.CSPadArrPeakFinder] Peak candidate:q/s/c/r=2/5/105.1/369.1  npix/min:max=31/9:99  Btot=29.0 noise=5.9  ampmax=401.0  amptot/thr=2443.0/100.0  SoN/thr=414.8/5.0 Peak candidate:q/s/c/r=2/5/145.4/385.0  npix/min:max=9/9:99  Btot=8.0 noise=4.6  ampmax=330.0  amptot/thr=687.0/100.0  SoN/thr=148.9/5.0

A: This message was added for test purpose only and is generated inside the loop committed for parallel processing of sections. All sections are processed on isolated memory. The MsgLog(...) class is not designed for parallel processing. Some messages may be missing due to overlapping in time requests to MsgLog(...).

2012-09-13 Josef Frisch, Ryan Coffee, Nick Hartmann

Mtg. at 14:00: Joseph Barrera gave a tutorial of how to use python modules with pyana and psana...
Q: Ryan Coffee described what do they want to do with data:

  • Get Opal image with two 1024 bin spectra for signal and reference,
  • Calibrate (each 5th event) both spectra
  • Evaluate the difference between them 2*(f(signal)-f(reference)),
  • Find the peak location.

A: In order to help access data, two psana modules were implemented:

2012-09-12 Sebastian Carron (at Wed. grp mtg.)

Q: General complain about inconvenience of access to data...
We have to discuss, what exactly does he need.

2012-08-30 Marcin Sikorski, xcs

Q: Need in real-time algorithm for image self-correlations in time.

  • Run duration 500-10^6 images, ~one hour, processing time ~<1 hour,
  • g2(tau) = <I(t)*I(t+tau)> / (<I(t)> * <I(t+tau)>),
  • Princeton camera 1300x1340,
  • Image frequency up to 120 Hz,
  • Tau - 50 points log in time delay,
  • For tests: xcsi0112-r0015.

A: Work on ImgAlgos/test/corana application is in progress. Combination of psana module with stand-alone batch multiprocessor application.

2012-07-27 Shibom Basu, amo

Q: I have a script cspad_geometry.pro to generate pixelmap in hdf5 format for cspad detector geometry where the coordinates of each pixel will be generated. As far I know any file with .pro extension will be compilable in IDL program...Is there any other way to compile .pro extension files?

A: There are a few other options to get CSPad pixel coordinates;

2012-07-23 Wenkai Zhang, xpp50312

Q: How to get corrected (pedestals, common mode, gain, peakfinder) image for CSPad2x2?
A: Use psana, see example in Example of psana analysis for CSPad2x2.

2012-07-06 Chun Hong Yoon, cxi43312

Q: How to get pnccd image data in psana?
A: New type of data have been added for pnccd by Andy:

  • Psana::PNCCD::FramesV1
  • Psana::PNCCD::FullFrameV1
    Example in $SIT_RELDIR/ana-0.5.8/psana_examples/src/DumpPnccd.cpp is also updated.
    See also Module ImgAlgos::PnccdImageProducer

2012-07-02 Johan Hattne, CXI

Q: Can not reproduce metrology in https://confluence.slac.stanford.edu/display/PCDS/2011-08-10+CSPad+alignment+parameters
A: Drop outliers, which may be caused by typos at optical measurement recording.

2012-06-27 Oleg Krupin, xppi0212

Q: How to get CSPad2x2 data in psana module and save it as an array
A: Implemented new module: CSPadPixCoords::CSPad2x2ImageProducer. See description and example:
Module CSPadPixCoords::CSPad2x2ImageProducer,
Example for Module CSPadPixCoords::CSPad2x2ImageProducer

2012-06-29 Daniel Flath, XCS

Q: How to integrate the matplotlib image with pyqt GUI?
A: See example in /reg/neh/home/dubrovin/LCLS/test_GUI_with_graphics/

2012-06-25 Sebastian Carron, RED Controls & Data Systems

Q: Why in cxii0212-r0105 the pyana with simple user module can not get ebeam data, while they were available for earlier run?
A: Command: pyxtcreader /reg/d/psdm/CXI/cxii0212/xtc/e165-r0105-s00-c00.xtc
shows that the BLD is not available in this file. It has nothing to do with pyana. Problem of DAQ?

2012-06-25 Chun Hong Yoon, cxi43312

Q: Why psana reads events so slow? The cxi43312-r0411 event readout rate is 7Hz only for psana_examples.DumpCsPad
A: In my test with configuration file

[psana]
files   = \
  /reg/d/psdm/cxi/cxi43312/xtc/e142-r0411-s00-c00.xtc\
  /reg/d/psdm/cxi/cxi43312/xtc/e142-r0411-s01-c00.xtc\
  /reg/d/psdm/cxi/cxi43312/xtc/e142-r0411-s02-c00.xtc\
  /reg/d/psdm/cxi/cxi43312/xtc/e142-r0411-s03-c00.xtc\
  /reg/d/psdm/cxi/cxi43312/xtc/e142-r0411-s04-c00.xtc\
  /reg/d/psdm/cxi/cxi43312/xtc/e142-r0411-s05-c00.xtc

events  = 100
modules = ImgAlgos.Tahometer
          psana_examples.DumpCsPad

[ImgAlgos.Tahometer]
print_bits    = 7
dn            = 10

[psana_examples.DumpCsPad]
source        = DetInfo(CxiDs1.0:Cspad.0)

it gives ?20Hz:
Time to process 100 events is 4.61388 sec, or 0.0461388 sec/event

Command:
time psana -c psana-cxi43312-r0411-....cfg
gives:
0.130u 1.508s 0:05.69 28.6% 0+0k 0+0io 0pf+0w
that indicates that CPU loading time is about 28.6% from astro. This is an indication that the file system is not quite effective and CPU is waiting for data.

2012-05-21 Marc Messerschmidt

Q: hdf5explorer does not plot cspad2x2 for cxii0212-r0105.h5 and cxi49812-r007.h5
A: cspad2x2 was not recognized due to renaming of CsPad::MiniElement in CsPad2x2.
Naming is fixed in GlobalMethods.py for tag V00-00-14.

2012-05-09 Ingrid run tests with psaana on, cxi22010/xtc/e60-r0309-s00-c00.xtc

Q: can not read alignment parameters
A: parameter files are missing for this run.
For now add the warning message in PSCalib::CSPadCalibPars.cpp
DEFAULT ALIGNMENTENT PARAMETERS NEED TO BE SUBSTITUTED....

2012-05-09 Daniel Ratner, Despina Milathianaki, cxii0112

Q: How to write CSPad image in file (tiff, binary, or text)?
A: Use psana modules and configuration file as explained in Psana Module Examples - Old#Example for Package CSPadPixCoords

2012-04-05 Andreas Schropp, MEC

Q: How to get in psana the event number (index) for file name?
A: Use the time stamp:

#include "PSEvt/EventId.h"
#include "PSTime/Time.h"

shared_ptr<PSEvt::EventId> eventId = evt.get();
  if (eventId.get()) {
   PSTime::Time t = eventId->time();
   std::string t_str = (eventId->time()).asStringFormat( "%Y-%m-%d %H:%M:%S%f%z");
  }

2012-04-04 Stefan P. Moeller, sxrcom11

Q: How to extract arrays from the hdf5 files (r0059) ?
A: ~dubrovin/LCLS/PyApps/GetHDF5Dataset/SaveDatasetsInFile.pyhttps://confluence.slac.stanford.edu/display/PCDS/How+to+access+HDF5+data+from+Python

2012-03-23 Henrik Till Lemke, xppcom10

Q: PyCSPadImage does not provide correct alignment for xppcom10-r1437.h5
A: the run range in calib dir is 1423-end.data. Need to pass the run number to
calp.calibpars.setCalibParsForPath ( run = runnum, path = path_calib )
Q: Image orientation is wrong by 180 (or 90?) degree.
A: 1) use origin='lower' in plt.imshow
2) arr_rotated = np.rot90(arr,2) # where 2 stands for 2*90=180 degree

2012-03-07 Hartawan Laksmono, cxi49812

Q: How to reconstruct CSPad image in python code from hdf5 data?
A: https://confluence.slac.stanford.edu/display/PCDS/CSPad+image+producer+in+Python

2012-03-02 Rui Xu, cxi39112

Q: How to adjust the quad positions in geometry parameters?
A: For individual quad position alignment use file:
/reg/d/psdm/CXI/cxi39112/calib/CsPad::CalibV1/CxiDs1.0:Cspad.0/offset_corr/0-end.data

dXq1 dXq2 dXq3 dXq4
dYq1 dYq2 dYq3 dYq4
dZq1 dZq2 dZq3 dZq4

To move quads simultaneously w.r.t. the detector center use
/reg/d/psdm/CXI/cxi39112/calib/CsPad::CalibV1/CxiDs1.0:Cspad.0/marg_gap_shift/0-end.data

           offset of 2x1s in quad (for tilt)
          /    offset of quads in image (for tilt)
         /    /   gaps
        /    /   /   shifts
       /    /   /   /
X:   15   40   0  32
Y:   15   40   0  32
Z:    0    0   0   0

I would recommend to change shifts for X and Y.

2012-02-23 Hartawan Laksmono, cxi49812

Q: How to extract time variables?
A: https://confluence.slac.stanford.edu/display/PCDS/How+to+access+HDF5+data+from+Python
Section : Advanced operations -> Extract time,
Example 2: Extract and print the time variables

2012-02-22 Marc Messerschmidt, cxi

At the weekly meeting suggested to implement in psana a few algorithms:
1. common mode correction, based on average(A<30) for each ASIC
2. pixel counting above threshold for filter
3. peak finding was also discussed w/o detaios

2012-02-17 Hartawan Laksmono, cxi49812

Need to get HDF5 data in python.

Response: use pyana: https://confluence.slac.stanford.edu/display/PCDS/pyana
or stand-alone code: https://confluence.slac.stanford.edu/display/PCDS/How+to+access+HDF5+data+from+Python
Also useful is hdf5explorer: https://confluence.slac.stanford.edu/display/PCDS/HDF5+Explorer

Currently the hdf5explorer can be built locally with
addpkg PyCSPadImage

2012-01-18 Matthias Fuchs, xpp36211

Need to reconstruct CSPad alignment image in pyana ...

Response: use constans from
/reg/d/psdm/XPP/xpp36211/calib/CsPad::CalibV1/XppGon.0:Cspad.0/
with Ingrid code for image reconstruction in pyana.

2011-12-08 Marc, cxi49812, single photon gain

We need to produce a gain correction for the mini CSpad. I believe if you build histograms for cxi49812-r0203 you might be able to fit a 9.5 keV single photon peak everywhere.
Response:

2011-11-29 Marco Siano, Adi Natan, Richard J. Squibb, amo42112

For Opal camera need in peak finding C++ function with specified 2-threshold algorithm:

  • Input: raw image, gaussian sigma, center?, Thr1, 2, Edge
  • Processing
    • discard edges
    • discard pixels with amplitude below Thr1
    • smearing using convolution with Gaussian (sigma,center?)
    • peak finding above Thr2.
  • Output: Npeaks, X(Npeaks), Y(Npeaks)

Use MatLab at final stage.

Ingrid works on implementation in pyana.

2011-11-18 David Fritz, Jesse Clark, xpp44712

Need in precise CSPad geometry for images in MatLab + python + hdf5.

In responce I produced the description of the available package CSPad image producer in Python (DEPRICATED).

2011-10-10 Stefan Hau-Riege and Jan Feldkamp, cxi37411

Need in CSPad aligned image. Need in geometry constants for 2 CSPads in this experiment.
Constants were produced CSPAD Alignment. Can be used both in psana and python code.

2011-10-13 Marc, cxi37411

Experiment uses 2 CSPads. Need to be supported in HDF5Explorer.

HDF5Explorer is updated for multi-CSPad data.

2011-08-24 Johan Hattne, cxi22010 (cxi80410)

CSPad alignment and stimulated developement of the image producer as a psana module CSPAD Geometry Software (depricated)

2011-03-08 Adrian Mancuso, cxi80410

For the phase retrieval experiment it is essential to get a direct access to the three camera images from HDF5 files. Images need to be aligned with respect to each other; rotated and translated. Use MatLab at final stage.

Our solution is described in the Phase retrieval experiment.

  • No labels