Page History
...
- myana — A C++ program to analyze an xtc file. Provided (and used) by the DAQ group. How to set up your own myana executable is explained in the DAQ section "A Simple Online Analysis Example" .
- pyana — A python-based analysis framework. Pyana User Manual
- PSAna — A C++-based analysis framework, still in the design phase. Not usable yet.. Psana User Manual - Old
This document attempts to explain the names and functions found in the myana code and give some working examples on how to set up your analysis software. And we try to explain the structure of the data file and how to extract useful information from your data.
In several of these examples, we fill root histograms or NTuples. For more information on root, see http://root.cern.ch.
Disclaimer: There is no more complete or up-to-date documentation than the code itself, so regard this document as an introduction and a user guide, not a complete documentationIf you have questions or requests related to this user guide, feel free to send me an email (ofte at slac.stanford.edu).
Anchor | ||||
---|---|---|---|---|
|
The data file format: xtc
The data recorded from the LCLS experiments are stored in xtc (eXtended Tagged Container) files. These files contain This online format consists of "datagrams" which are an object of some type (TypeId) with associated status (Damage), structures that have fileds like TypeId, Damage, source (Src) and extent (size). It is Xtc files are not an indexed file and does not provide random access, and can only be read seqencially, one event (shot) at a time.. The data file contains only data, no metadata, so you depend on the pdsdata library (or similar) to make sense of the files. The only way to read it is using a special iterator (XtcIterator) and read the events seqencially, one shot at a time. The myana application does this loop for you, and you can customize the event()
function to read out the information you want from the xtc file.
You can explore the You can explore the contents of an xtc file by using the xtcreader
utility or pyxtcreader
utilities:
Code Block |
---|
pslogin ~ > xtcreader -f myxtcfile.xtc | less
pslogin ~ > pyxtcreader myxtcfile.xtc | less
|
Reading through the output, Reading through the output, you may see sections describing the various transitions in datataking. Look for these "headings" in the text output:
- Configure transition
- BeginRun transition
- BeginCalibCycle transition
- Enable transition
- L1Accept transition — This is the event data. Each event starts with "L1Accept transition:". From the text that follows, you can get an idea of what detector data is in the xtc file.
You also have the option of converting your experiments' xtc files into hdf5 files. More about data formats and where to find the experiment data files, see Analysis Workbook. Data Formats
...
The pdsdata
library
Myana uses the pdsdata library to access the datagrams in the xtc files. pds = photon data system. In your analysis directory you'll find it in release/pdsdata/
. The header files are in the top level directories of each package, and the implementation files are in the src directory of each package.
Panel | ||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
A new tool to list the contents of the file:
Code Block |
---|
pslogin ~ > xtcsummary.py
|
lists all detector and epics information found in Configure and L1Accept (event data) sections, and lists number of events per calibration cycle. Can be helpful when putting together your myana or pyana analysis script. Example output.
You can analyze the xtc data with the offline tools, myana
and pyana
. You also have the option of using the hdf5 data format (hierarchical data format 5), but you will have to wait for the xtc -> hdf5 translation which may take some time. Also, there is no support for hdf5 analysis by the offline group quite yet. hdf5 will be the standard offline LCLS data format, and tools are under development for analyzing these files. More about data formats and where to find the experiment data files, see Analysis Workbook. Data Formats
Anchor | ||||
---|---|---|---|---|
|
The pdsdata
library
See also pdsdata Reference Manual.
Myana uses the pdsdata library to access the datagrams in the xtc files, thus in this context pdsdata
defines the data structure. pds = photon data system. In your analysis directory you'll find it in release/pdsdata/
. The header files are in the top level directories of each package, and the implementation files are in the src directory of each package. Here's very briefly what the library contains:
Panel | ||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Anchor | ||||
---|---|---|---|---|
|
myana.cc .... an example C++ program to extract information from xtc file
This example fetches data for each event and writes it to a root histogram and stores the histogram in a root file. You may want to store your data differently, e.g. one histogram for each event, or everything in a root ntuple for further processing. Or you can write some other format that you'd like to work with (ascii file, ... ).
Panel |
---|
myana.cc - example code that makes a simple averaging histogram myana_morefeatures.cc - example code that does a little more than myana.cc |
The examples above are meant to show you how you can make your own code. With different experiments using different hardware and having different goals, these examples might not apply to your particular experiment / datafile. If so, you'll need to search the main code and libraries a bit to find something more suitable. Here's a brief description of the functions of the myana.cc example and main.cc:
This is the "user analysis module". This is where you fill in your own code to extract the information that you want from your experiment's xtc file. This module contain only the following functions:
Code Block |
---|
beginjob() // called at the beginning of an analysis job. You can for instance book histograms here.
beginrun() |
...
myana.cc .... an example C++ program to extract information from xtc file
This example fetches data for each event and writes it to a root histogram and stores the histogram in a root file. You may want to store your data differently, e.g. one histogram for each event, or everything in a root ntuple for further processing. Or you can write some other format that you'd like to work with (ascii file, ... ).
Panel |
---|
myana.cc - example code that makes a simple averaging histogram myana_morefeatures.cc - example code that does a little more than myana.cc |
The examples above are meant to show you how you can make your own code. With different experiments using different hardware and having different goals, these examples might not apply to your particular experiment / datafile. If so, you'll need to search the main code and libraries a bit to find something more suitable. Here's a brief description of the functions of the myana.cc example and main.cc:
This is the "user analysis module". This is where you fill in your own code to extract the information that you want from your experiment's xtc file. This module contain only the following functions:
Code Block |
---|
beginjob() // called at the beginning of an analysis job. You can for instance book histograms here.
beginrun() // called at the beginning of a run (the analysis job might analyze several runs)
begincalib() // called for each calibration cycle
event() // this is where you fetch, process and store information about each event (shot).
endcalib()
endrun()
endjob()
|
...
- Most of the functions return 0 if it was a successful function call, any other number means it failed.
Values are obtained through the arguments of the function calls. E.g. declare an array in your myana.cc, and {{Wiki Markup getXXXValue(&myarray
\[0
\])
}} will fill the array for you.- Enums: Several of the functions can be used to extract data from several of the detectors. Which detector is specified by an enum (named constant integers). You are encouraged to use the names instead of the numbers, in case the underlying order changes in a new version of the program.
...
There are several getters for fetching image data from the xtc file. Depending on which camera was in use, one of these should be appropriate:
- Opal1000 camera:
getOpal1kValue
getFrameValue
(an alias getOpal1kValue is provided for getFrameValuebackward compatibility)
- Pulnix6740CL camera:
getTm6740ValuegetFrameValue
(an alias getTm6740Value is provided for getFrameValuebackward compatibility)
- FrameDetector (general):
Gives you the width and height (in pixels) of the image, and a pointer to the start of the pixel array of a Pds::Camera::FrameV1 object. Specify the detector (using an appropriate enum).Code Block int getFrameConfig (FrameDetector det); int getFrameValue(FrameDetector det, int& frameWidth, int& frameHeight, unsigned short*& image );
Code Block Available frame detectors: AMO: AmoVmi - velocity map imaging AmoBps1 - beam position screen AmoBps2 - beam position screen SXR: SxrBeamlineOpal1 SxrBeamlineOpal2 SxrEndstationOpal1 SxrEndstationOpal2 SxrFccd XPP: XppSb1PimCvd XppMonPimCvd XppSb3PimCvd XppSb4PimCvd
- XPP CsPad detector:
Gives you a pointer to the first position in the array of pixel data from the XPP CsPad detector (or alternatively you can get a pointer to the Pds::CsPad::ElementV1 object itself).Code Block namespace Pds { namespace CsPad { class ConfigV1; }} int getCspadConfig (Pds::DetInfo::Detector det, unsigned& quadMask, unsigned& asicMask); int getCspadConfig (Pds::DetInfo::Detector det, Pds::CsPad::ConfigV1& cfg); namespace Pds { namespace CsPad { class ElementV1; }} int getCspadQuad (Pds::DetInfo::Detector det, unsigned quad, const uint16_t*& pixels); int getCspadQuad (Pds::DetInfo::Detector det, unsigned quad, const Pds::CsPad::ElementV1*& data);
For an example of how to draw an image as a 2D root histogram, see the myana_cspad.cc example.
- Fast CCD camera: camera:
Configures the information from the Fast CCD. Fills arguments with values depending on how the image/waveform data were taken. There is noCode Block int getFccdConfig(FrameDetector det, uint16_t& outputMode, bool& ccdEnable, bool& focusMode, uint32_t& exposureTime, float& dacVoltage1, float& dacVoltage2, float& dacVoltage3, float& dacVoltage4,
Code Block int getFccdConfig(FrameDetector det, uint16_t& outputMode, bool& ccdEnable, float& dacVoltage5, float& dacVoltage6, boolfloat& focusModedacVoltage7, uint32_tfloat& exposureTimedacVoltage8, float& dacVoltage1dacVoltage9, float& dacVoltage2dacVoltage10, float& dacVoltage3dacVoltage11, float& dacVoltage4dacVoltage12, float& dacVoltage5dacVoltage13, float& dacVoltage6dacVoltage14, float& dacVoltage7dacVoltage15, float& dacVoltage8dacVoltage16, float& dacVoltage9, float& dacVoltage10, float& dacVoltage11, float& dacVoltage12,dacVoltage17, floatuint16_t& dacVoltage13waveform0, floatuint16_t& dacVoltage14waveform1, floatuint16_t& dacVoltage15waveform2, floatuint16_t& dacVoltage16waveform3, uint16_t& waveform4, uint16_t& waveform5, uint16_t& floatwaveform6, uint16_t& dacVoltage17waveform7, uint16_t& waveform0waveform8, uint16_t& waveform1waveform9, uint16_t& waveform2waveform10, uint16_t& waveform3waveform11, uint16_t& waveform4waveform12, uint16_t& waveform5waveform13, uint16_t& waveform6, uint16_t& waveform7, uint16_t& waveform8, uint16_t& waveform9, uint16_t& waveform10, uint16_t& waveform11, uint16_t& waveform12, uint16_t& waveform13, uint16_t& waveform14);
getFccdValue
in main.hh, so I think you need to usegetFrameValue
for this .
- PnCCD camera (used by the CAMP collaboration):
This camera has 4 links, each link provides a 512 x 512 x 16 bit image. This function combines the four images to a single 1024 x 1024 x 16 bit image.Code Block int getPnCcdValue (int deviceId, unsigned char*& image, int& width, int& height );
deviceId
can be PnCcd0 or PnCcd1, width and height are the number of pixels in each direction.
Configures the information from the Fast CCD. Fills arguments with values depending on how the image/waveform data were taken.waveform14);
Fetches the FCCD image data. Specify the detector (only SxrFccd is available as of 2012).Code Block int getFrameValue(FrameDetector det, int& frameWidth, int& frameHeight, unsigned short*& image );
- PnCCD camera (used by the CAMP collaboration):
This camera has 4 links, each link provides a 512 x 512 x 16 bit image. This function combines the four images to a single 1024 x 1024 x 16 bit image.Code Block int getPnCcdValue (int deviceId, unsigned char*& image, int& width, int& height );
deviceId
can be PnCcd0 or PnCcd1, width and height are the number of pixels in each direction.
- Princeton camera:
To get the image data (array of unsigned short), usegetPrincetonValue
, and to get other information, like the image size, camera exposure, temperature etc, usegetPrincetonConfig
andgetPrincetonTemperature
:
Princeton camera:Code Block int getPrincetonConfig(Pds::DetInfo::Detector det, int iDevId, int& width, int& height, // image width and height in pixels int& orgX, int& orgY, // 0,0
fetches the configuration and data from the camera.Code Block int getPrincetonConfig(Pds::DetInfo::Detector det, int iDevId, int& widthbinX, int& height, int& orgXbinY); // 1,1 int getPrincetonValue(Pds::DetInfo::Detector det, int& orgY, int& binX, int&binY); int getPrincetonValue(Pds::DetInfo::Detector det, int iDevId,iDevId, unsigned short *& image); // pointer to first pixel element int getPrincetonTemperature(Pds::DetInfo::Detector det, int iDevId, float& temperature);
getPrincetonTemperature
is there to check the temperature of the camera at the time of data taking (not necessarily available for every shot).
...
- Encoder detector
Position of mirrors (SXISXR)Code Block int getEncoderConfig (Pds::DetInfo::Detector det, int iDevId); int getEncoderCount(Pds::DetInfo::Detector det, int iDevId, unsigned int& encoderCount);
- DiodeFex (Diode feature extraction)
Code Block int getDiodeFexConfig (Pds::DetInfo::Detector det, int iDevId, float* base, float* scale); int getDiodeFexValue (Pds::DetInfo::Detector det, int iDevId, float& value);
...
- Front end enclosure Gas detector
Gives you the shot energy to the arrayCode Block int getFeeGasDet (double* shotEnergy);
shotEnergy[4]
.
- Electron beam monitor
Gives electron beam values for each of these doubles. The measured charge of the beam (in nC),Code Block int getEBeam(double& charge, double& energy, double& posx, double& posy, double& angx, double& angy); int getEBeam(double& charge, double& energy, double& posx, double& posy, double& angx, double& angy, double& pkcurr);
the measured energy of the beam (in MeV), the 2D position of the beam (in mm) away from the origin
(nominal beam position), and 2D angular position (in mrad) off the assumed direction. and the
pkcurr = current? in (Amps)
...
- EPICS values (Process variables)
Get integers, floats, strings from any EPICS channel (PV = process variable)Code Block int getPvInt (const char* pvName, int& value); int getPvFloat (const char* pvName, float& value); int getPvString (const char* pvName, char*& value);