Page History

...

• myana — A C++ program to analyze an xtc file. Provided (and used) by the DAQ group. How to set up your own myana executable is explained in the DAQ section "A Simple Online Analysis Example" .
• pyana — A python-based analysis framework. Pyana User Manual
• PSAna — A C++-based analysis framework, still in the design phase. Not usable yet.. Psana User Manual - Old

This document attempts to explain the names and functions found in the myana code and give some working examples on how to set up your analysis software. And we try to explain the structure of the data file and how to extract useful information from your data.

In several of these examples, we fill root histograms or NTuples. For more information on root, see http://root.cern.ch.

Disclaimer: There is no more complete or up-to-date documentation than the code itself, so regard this document as an introduction and a user guide, not a complete documentation. If you have questions, requests or general feedback, If you have questions or requests related to this user guide, feel free to send me an email (ofte at slac.stanford.edu).

...

The data recorded from the LCLS experiments are stored in xtc (eXtended Tagged Container) files. These files contain This online format consists of "datagrams" which are an object of some type (TypeId) with associated status (Damage), structures that have fileds like TypeId, Damage, source (Src) and extent (size). It is Xtc files are not an indexed file and does not provide random access. The data file contains only data, and can only be read no metadata, so you depend on the pdsdata library (or similar) to make sense of the files. The only way to read it is using a special iterator (XtcIterator) and read the events seqencially, one event ( shot ) at a time. The myana application does this loop for you, and you can customize the event() function to read out the information you want from the xtc file.

You can explore the contents of an xtc file by using the xtcreader utility or pyxtcreader utilities:

pslogin ~ >  xtcreader -f myxtcfile.xtc | less
pslogin ~ >  pyxtcreader  myxtcfile.xtc | less

Reading through the output, you may see sections describing the various transitions in datataking. Look for these "headings" in the text output:

• Configure transition
• BeginRun transition
• BeginCalibCycle transition
• Enable transition
• L1Accept transition — This is the event data. Each event starts with "L1Accept transition:". From the text that follows, you can get an idea of what detector data is in the xtc file.

You also have the option of converting your experiments' xtc files into hdf5 files. More about data formats and where to find the experiment data files, see Analysis Workbook. Data Formats

...

The pdsdata library

Myana uses the pdsdata library to access the datagrams in the xtc files. pds = photon data system. In your analysis directory you'll find it in release/pdsdata/. The header files are in the top level directories of each package, and the implementation files are in the src directory of each package.

...

A new tool to list the contents of the file:

pslogin ~ >  xtcsummary.py

lists all detector and epics information found in Configure and L1Accept (event data) sections, and lists number of events per calibration cycle. Can be helpful when putting together your myana or pyana analysis script. Example output.

You can analyze the xtc data with the offline tools, myana and pyana. You also have the option of using the hdf5 data format (hierarchical data format 5), but you will have to wait for the xtc -> hdf5 translation which may take some time. Also, there is no support for hdf5 analysis by the offline group quite yet. hdf5 will be the standard offline LCLS data format, and tools are under development for analyzing these files. More about data formats and where to find the experiment data files, see Analysis Workbook. Data Formats

The pdsdata library

See also pdsdata Reference Manual.

Myana uses the pdsdata library to access the datagrams in the xtc files, thus in this context pdsdata defines the data structure. pds = photon data system. In your analysis directory you'll find it in release/pdsdata/. The header files are in the top level directories of each package, and the implementation files are in the src directory of each package. Here's very briefly what the library contains:

myana.cc .... an example C++ program to extract information from xtc file

This example fetches data for each event and writes it to a root histogram and stores the histogram in a root file. You may want to store your data differently, e.g. one histogram for each event, or everything in a root ntuple for further processing. Or you can write some other format that you'd like to work with (ascii file, ... ).

The examples above are meant to show you how you can make your own code.  With different experiments using different hardware and having different goals, these examples might not apply to your particular experiment / datafile. If so, you'll need to search the main code and libraries a bit to find something more suitable. Here's a brief description of the functions of the myana.cc example and main.cc:

myana.hh and myana.cc:

This is the "user analysis module". This is where you fill in your own code to extract the information that you want from your experiment's xtc file. This module contain only the following functions:

beginjob

...

myana.cc .... an example C++ program to extract information from xtc file

This example fetches data for each event and writes it to a root histogram and stores the histogram in a root file. You may want to store your data differently, e.g. one histogram for each event, or everything in a root ntuple for further processing. Or you can write some other format that you'd like to work with (ascii file, ... ).

The examples above are meant to show you how you can make your own code.  With different experiments using different hardware and having different goals, these examples might not apply to your particular experiment / datafile. If so, you'll need to search the main code and libraries a bit to find something more suitable. Here's a brief description of the functions of the myana.cc example and main.cc:

myana.hh and myana.cc:

This is the "user analysis module". This is where you fill in your own code to extract the information that you want from your experiment's xtc file. This module contain only the following functions:

beginjob()   // called at the beginning of an analysis job. You can for instance book histograms here.
beginrun()   // called at the beginning of a run (the analysis job might analyze several runs)
begincalib() // called for each calibration cycle
event()      // this is where you fetch, process and store information about each event (shot).
endcalib()
endrun()
endjob()

...

• Most of the functions return 0 if it was a successful function call, any other number means it failed.
• Wiki MarkupValues are obtained through the arguments of the function calls. E.g. declare an array in your myana.cc, and {{getXXXValue(&myarray\[0\])}} will fill the array for you.
• Enums: Several of the functions can be used to extract data from several of the detectors. Which detector is specified by an enum (named constant integers). You are encouraged to use the names instead of the numbers, in case the underlying order changes in a new version of the program.

...

There are several getters for fetching image data from the xtc file. Depending on which camera was in use, one of these should be appropriate:

• Opal1000 camera:
  getOpal1kValue getFrameValue (an alias getOpal1kValue is provided for getFrameValuebackward compatibility)

• Pulnix6740CL camera:
  getTm6740Value getFrameValue (an alias getTm6740Value is provided for getFrameValuebackward compatibility)

• FrameDetector (general):

  Code Block
  int getFrameConfig (FrameDetector det); int getFrameValue(FrameDetector det, int& frameWidth, int& frameHeight, unsigned short*& image );

  Gives you the width and height (in pixels) of the image, and a pointer to the start of the pixel array of a Pds::Camera::FrameV1 object. Specify the detector (using an appropriate enum).

  Code Block
  Available frame detectors: AMO: AmoVmi - velocity map imaging AmoBps1 - beam position screen AmoBps2 - beam position screen SXR: SxrBeamlineOpal1 SxrBeamlineOpal2 SxrEndstationOpal1 SxrEndstationOpal2 SxrFccd XPP: XppSb1PimCvd XppMonPimCvd XppSb3PimCvd XppSb4PimCvd

...

• Fast CCD camera:

  Code Block

  int getFccdConfig(FrameDetector det, uint16_t& outputMode, bool& ccdEnable, bool& focusMode, uint32_t& exposureTime, float& dacVoltage1, float& dacVoltage2, float& dacVoltage3, float& dacVoltage4, float& dacVoltage5, float& dacVoltage6, float& dacVoltage7, float& dacVoltage8, float& dacVoltage9, float& dacVoltage10, float& dacVoltage11, float& dacVoltage12, float& dacVoltage13, float& dacVoltage14, float& dacVoltage15, float& dacVoltage16, float& dacVoltage17, uint16_t& waveform0, uint16_t& waveform1, uint16_t& waveform2, uint16_t& waveform3, uint16_t& waveform4, uint16_t& waveform5, uint16_t& waveform6, uint16_t& waveform7, uint16_t& waveform8, uint16_t& waveform9, uint16_t& waveform10, uint16_t& waveform11, uint16_t& waveform12, uint16_t& waveform13, uint16_t& waveform14);

  Configures the information from the Fast CCD. Fills arguments with values depending on how the image/waveform data were taken. There is no getFccdValue in main.hh, so I think you need to use getFrameValue for this .

• PnCCD camera (used by the CAMP collaboration):

  Code Block
  int getPnCcdValue (int deviceId, unsigned char*& image, int& width, int& height );

  This camera has 4 links, each link provides a 512 x 512 x 16 bit image. This function combines the four images to a single 1024 x 1024 x 16 bit image.
  deviceId can be PnCcd0 or PnCcd1, width and height are the number of pixels in each direction.

• the information from the Fast CCD. Fills arguments with values depending on how the image/waveform data were taken.
  

  Code Block
  int getFrameValue(FrameDetector det, int& frameWidth, int& frameHeight, unsigned short*& image );

  Fetches the FCCD image data. Specify the detector (only SxrFccd is available as of 2012).

• PnCCD camera (used by the CAMP collaboration):

  Code Block
  int getPnCcdValue (int deviceId, unsigned char*& image, int& width, int& height );

  This camera has 4 links, each link provides a 512 x 512 x 16 bit image. This function combines the four images to a single 1024 x 1024 x 16 bit image.
  deviceId can be PnCcd0 or PnCcd1, width and height are the number of pixels in each direction.

• Princeton camera:
  To get the image data (array of unsigned short), use getPrincetonValue, and to get other information, like the image size, camera exposure, temperature etc, use getPrincetonConfig and getPrincetonTemperature:

  Code Block
  int getPrincetonConfig(Pds::DetInfo::Detector det, int iDevId, int& width, int& height, // image width and height in pixels int& orgX, int& orgY, // 0,0

  Princeton camera:

  Code Block

  int getPrincetonConfig(Pds::DetInfo::Detector det, int iDevId, int& widthbinX, int& height, int& orgXbinY); // 1,1 int getPrincetonValue(Pds::DetInfo::Detector det, int& orgYiDevId, int& binX, int&binY); int getPrincetonValue(Pds::DetInfo::Detector det, int iDevId, unsigned short *& image); // pointer to first pixel element int getPrincetonTemperature(Pds::DetInfo::Detector det, int iDevId, float& temperature);

  fetches the configuration and data from the camera. getPrincetonTemperature is there to check the temperature of the camera at the time of data taking (not necessarily available for every shot).

...

• Encoder detector

  Code Block
  int getEncoderConfig (Pds::DetInfo::Detector det, int iDevId); int getEncoderCount(Pds::DetInfo::Detector det, int iDevId, unsigned int& encoderCount);

  Position of mirrors (SXISXR)

• DiodeFex (Diode feature extraction)

  Code Block
  int getDiodeFexConfig (Pds::DetInfo::Detector det, int iDevId, float* base, float* scale); int getDiodeFexValue (Pds::DetInfo::Detector det, int iDevId, float& value);

...

• Front end enclosure Gas detector

  Code Block
  int getFeeGasDet (double* shotEnergy);

  Gives you the shot energy to the array shotEnergy[4].

• Electron beam monitor

  Code Block
  int getEBeam(double& charge, double& energy, double& posx, double& posy, double& angx, double& angy); int getEBeam(double& charge, double& energy, double& posx, double& posy, double& angx, double& angy, double& pkcurr);

  Gives electron beam values for each of these doubles. The measured charge of the beam (in nC),
  the measured energy of the beam (in MeV), the 2D position of the beam (in mm) away from the origin
  (nominal beam position), and 2D angular position (in mrad) off the assumed direction. and the
  pkcurr = current? in (Amps)

...

• EPICS values (Process variables)
  Get integers, floats, strings from any EPICS channel (PV = process variable)

  Code Block
  int getPvInt (const char* pvName, int& value); int getPvFloat (const char* pvName, float& value); int getPvString (const char* pvName, char*& value);

Further analysis help

CSPad analysis page