Versions Compared

Old Version 37

changes.mady.by.user Unknown User (ofte)

Saved on

compared with

New Version 38

changes.mady.by.user Unknown User (ofte)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This

...

page

...

is

...

under

...

construction.

Introduction

LCLS Data Analysis frameworks are under development, and currently three approaches are being used and/or developed:

  • myana ... simple C++ code to read xtc file. Provided (and used) by the DAQ group. Will likely be expanded in support of new hardware etc., and more examples might be provided, but otherwise no big changes anticipated.
  • pyana ... python-based analysis framework. Anticipate more tools and examples to appear for this one.
  • PSAna ... C++-based analysis framework, still in the design phase.

Currently only myana and pyana are usable for analysis.

This document attempts to explain some of the names and functions found in the myana code as well as the structure of the data file (xtc) and how to extract useful information from it.

Disclaimer: There is no more complete or up-to-date documentation than the code itself, so regard this document as an introduction and a user guide, not a complete documentation.

How to set up your own myana executable is explained in the DAQ section "A Simple Online Analysis Example".

myana makes use of the pdsdata library to read the datagrams from the xtc file.

Panel

pdsdata (pds = photon data system)
a library consisting of the follwing utility packages:

ipimb    -Intensity position, intensity monitor board
encoder  -
pnCCD   - for device used by CAMP
acqiris     - software for the aquiris digitizer hardware. Waveform data.
camera   - camera frames, configurations, feature extractions process
evr         - Event Receiver (event code, beam code?)
opal1k     - for Opal camera
pulnix      - for Pulnix TM6740CL monochrome camera
control    - utility for DAQ control, PV (process variable) control and monitoring
xtc          - This package defines all the datagrams for the xtc file.
epics      - interface to epics (process variables (PV))
bld         - defines some build data classes
princeton - software for the Princeton camera
fccd        - LBNL/ANL Fast CCD monochrome camera
cspad      - driver for the CsPad detector
lusi         - LCLS Ultrafast Science Instruments Configs for diode, ipm, pim.
app        - Xtc and Epics readers

The header files are in the top level directories of each package, and the
implementation files are in the src directory of each package.

xtc

The data from all the LCLS experiments are stored in xtc (extended container) files. These files contain "datagrams" which are an object of some type (TypeId) with associated status (Damage), source (Src) and extent (size). It is not an indexed file and does not provide random access, and can only be read seqencially. Thus, the example way to read the file shown here makes use of 'myana', a C++ executable that reads through the whole file and picks out the requested information. You can make your own version of 'myana' to extract other information.

...

MyAna .... C++ program to extract information from xtc file

...

This example fetches data for each event and writes it to a root histogram and stores the histogram in a root file. You may want to store your data differently, e.g. one histogram for each event, or everything in a root ntuple for further processing. Or you can write some other format that you'd like to work with (ascii file, ... ).

myana.cc - example code to make a simple averaging histogram
myana_morefeatures.cc - example code that does a little more.
main.cc - defines the functions used by myana.cc

Here's a brief description of the functions of myana.cc and main.cc:

myana.hh,cc:

Panel

This is the "user analysis module".  This is where you fill in your own code to extract the information that you want from your experiment's xtc file. This module contain only the following functions:

beginjob()     called at the beginning of an analysis job. You can for instance book histograms here.
beginrun()     called at the beginning of a run (the analysis job might analyze several runs)
begincalib() called for each calibration cycle
event()           this is where you fetch, process and store information about each event (shot).
endcalib()
endrun()
endjob()

main.hh,cc:

Panel

This is the main control of the analysis, but you should avoid editing this file. After the all utility functions (in main) and user functions (in myana) have been read, main() is executed and controls the flow of the analysis. For each xtc file it calls

_* h5. Introduction LCLS Data Analysis frameworks are under development, and currently three approaches are being used and/or developed: * myana ... simple C+\+ code to read xtc file. Provided (and used) by the DAQ group. Will likely be expanded in support of new hardware etc., and more examples might be provided, but otherwise no big changes anticipated. * [pyana|https://confluence.slac.stanford.edu/display/PCDS/Pyana+User+Manual] ... python-based analysis framework. Anticipate more tools and examples to appear for this one. * PSAna ... C++-based analysis framework, still in the design phase. Currently only myana and pyana are usable for analysis. This document attempts to explain some of the names and functions found in the myana code as well as the structure of the data file (xtc) and how to extract useful information from it. Disclaimer: There is no more complete or up-to-date documentation than the code itself, so regard this document as an introduction and a user guide, not a complete documentation. How to set up your own myana executable is explained in the DAQ section ["A Simple Online Analysis Example"|https://confluence.slac.stanford.edu/display/PCDS/A+Simple+Online+Analysis+Example]. myana makes use of the pdsdata library to read the datagrams from the xtc file. {panel} *pdsdata* (pds = photon data system) a library consisting of the follwing utility packages: ipimb    \-Intensity position, intensity monitor board encoder  - pnCCD   - for device used by CAMP acqiris     - software for the aquiris digitizer hardware. Waveform data. camera   - camera frames, configurations, feature extractions process evr         - Event Receiver (event code, beam code?) opal1k     - for Opal camera pulnix      - for Pulnix TM6740CL monochrome camera control    - utility for DAQ control, PV (process variable) control and monitoring xtc          - This package defines all the datagrams for the xtc file. epics      - interface to epics (process variables (PV)) bld         - defines some build data classes princeton - software for the Princeton camera fccd        - LBNL/ANL Fast CCD monochrome camera cspad      - driver for the CsPad detector lusi         - LCLS Ultrafast Science Instruments Configs for diode, ipm, pim. app        - Xtc and Epics readers The header files are in the top level directories of each package, and the implementation files are in the src directory of each package. {panel} h5. xtc The data from all the LCLS experiments are stored in xtc (extended container) files. These files contain "datagrams" which are an object of some type (TypeId) with associated status (Damage), source (Src) and extent (size). It is not an indexed file and does not provide random access, and can only be read seqencially. Thus, the example way to read the file shown here makes use of 'myana', a C+\+ executable that reads through the whole file and picks out the requested information. You can make your own version of 'myana' to extract other information. h6. MyAna .... C+\+ program to extract information from xtc file This example fetches data for each event and writes it to a root histogram and stores the histogram in a root file. You may want to store your data differently, e.g. one histogram for each event, or everything in a root ntuple for further processing. Or you can write some other format that you'd like to work with (ascii file, ... ). myana.cc - example code to make a simple averaging histogram myana_morefeatures.cc - example code that does a little more. main.cc - defines the functions used by myana.cc Here's a brief description of the functions of myana.cc and main.cc: myana.hh,cc: {panel} This is the "user analysis module".  This is where you fill in your own code to extract the information that _you_ want from your experiment's xtc file. This module contain only the following functions: {{beginjob()}}     called at the beginning of an analysis job. You can for instance book histograms here. {{beginrun()}}     called at the beginning of a run (the analysis job might analyze several runs) {{begincalib()}} called for each calibration cycle {{event()}}           this is where you fetch, process and store information about each event (shot). {{endcalib()}} {{endrun()}} {{endjob()}} {panel} main.hh,cc: {panel} This is the main control of the analysis, but you should avoid editing this file. After the all utility functions (in main) and user functions (in myana) have been read, {{main()}} is executed and controls the flow of the analysis. For each xtc file it calls {code}
Code Block

anafile(xtcname, maxevt, skip, iDebugLevel);
{code} 
which 
 
 iterates 
 
 through 
 
 the 
 
 xtc 
 
 file, 
 
 keeps 
 
 track 
 
 of 
 
 all 
 
 the 
 
 datagrams 
 
 in 
 
 it, 
 
 and 
 
 makes 
 
 sure 
 
 to 
 
 execute 
 
 your 
 {{
beginjob()
}} 
 and 
 {{
event()
}} 
 functions.

{panel}



All the functionality needed to get data from the xtc file is (or should be) defined in main.cc and in the files it includes (including the pdsdata library). Get an uppdated list of all the available functions by looking at main.hh (implementations are in main.cc).

----
h6. Configuration and L1Accept Data retrieval functions:

The following contains a few lines of explanation for some of the functions defined in main.
But first some general remarks:

* Most of the functions return 0 if it was a successful function call, any other number means it failed.
* Values are obtained through the arguments of the function calls. E.g. declare an array in your myana.cc, and {{getXXXValue(&myarray{}}}{{[0]}}{{)}} will fill the array for you.
* Enums: Several of the functions can be used to extract data from several of the detectors. Which detector is specified by an enum (named constant integers). You are encouraged to use the names instead of the numbers, in case the underlying order changes in a new version of the program.


{panel}
Acquiris digitizer:

{code}
All the functionality needed to get data from the xtc file is (or should be) defined in main.cc and in the files it includes (including the pdsdata library). Get an uppdated list of all the available functions by looking at main.hh (implementations are in main.cc).
...
Configuration and L1Accept Data retrieval functions:
...
The following contains a few lines of explanation for some of the functions defined in main.
 But first some general remarks:
  • Most of the functions return 0 if it was a successful function call, any other number means it failed.
  • Values are obtained through the arguments of the function calls. E.g. declare an array in your myana.cc, and getXXXValue(&myarray0) will fill the array for you.
  • Enums: Several of the functions can be used to extract data from several of the detectors. Which detector is specified by an enum (named constant integers). You are encouraged to use the names instead of the numbers, in case the underlying order changes in a new version of the program.
 Panel
Acquiris digitizer:
 Code Block
int getAcqConfig(AcqDetector det, int& numChannels, int& numSamples, double& sampleInterval);
{code}

Fetches 
 
 the 
 
 configuration 
 
 information 
 
 for 
 
 any 
 
 of 
 
 the 
 
 Acquiris 
 
 devices. 
 
 Returns 
 
 1 
 
 if 
 
 the 
 
 requested 
 
 detector 
 
 does 
 
 not 
 
 exist, 
 
 and 
 
 2 
 
 if 
 
 it 
 
 was 
 
 not 
 
 in 
 
 use. 
 
 Tells 
 
 you 
 
 the 
 
 number 
 
 of 
 
 channels 
 
 used 
 
 for 
 
 this 
 
 device, 
 
 the 
 
 number 
 
 of 
 
 samples 
 
 collected 
 
 and 
 
 the 
 
 sample 
 
 interval. 
 
 This 
 
 is 
 
 typically 
 
 done 
 
 in 
 
 the 
 {{
beginjob()
}} 
 or 
 {{
beginrun()
}} functions.

{code}
 functions.
 Code Block
int getAcqValue(AcqDetector det, int channel, double*& time, double*& voltage);
int getAcqValue(AcqDetector det, int channel, double*& time, double*& voltage, double& trigtime);
{code}

Fetches 
 
 waveform 
 
 data 
 
 from 
 
 any 
 
 of 
 
 the 
 
 Acquiris 
 
 devices. 
 
 Fills 
 
 your 
 
 arrays 
 
 with 
 
 the 
 
 waveform 
 
 time 
 
 and 
 
 voltage, 
 
 and 
 
 optionally 
 
 gives 
 
 you 
 
 the 
 
 trigger 
 
 time.



In 
 
 the 
 
 myana.cc 
 
 example, 
 
 we 
 
 fetch 
 
 data 
 
 from 
 
 the 
 
 AmoITof 
 
 device 
 
 (AMO 
 
 Ion 
 
 Time-of-flight).



 Other 
 
 Acquiris 
 
 devices 
 
 (see 
 
 main.hh 
 
 for 
 
 an 
 
 up-to-date 
 
 list):

{
 Code Block
}

AMO:
   AmoIms      - ion momentum spectrometer
   AmoGasdet   - gas detector (in the Front End Enclusure)
   AmoETof     - electron time-of-flight
   AmoMbes     - magnetic bottle electron spectrometer
   AmoVmiAcq   - (Vmi = Velocity mapping imaging)
   AmoBpsAcq   - (Bps = Beam position screen)
   Camp        - for the CAMP experimental setup
SXR:
   SxrBeamlineAcq1
   SxrBeamlineAcq2
   SxrEndstationAcq1
   SxrEndstationAcq2

{
 Code Block

 ------------------------------------------------------------------------------------------------------
 To get an image Frame
 ------------------------------------------------------------------------------------------------------

 int getFrameConfig   (FrameDetector det);

 -->  Checks that the xtc file has got data from this device.

 int getFrameValue(FrameDetector det, int& frameWidth, int& frameHeight, unsigned short*& image );

 --> fetches an image from the FrameDetector of your choice.
     Specify the detector (using an appropriate enum), give pointers to
                  int frame width, frame height and short imgage of the Camera::FrameV1.

  Other frame detectors:
  AMO:
      AmoVmi
      AmoBps1
      AmoBps2
  SXR:
      SxrBeamlineOpal1
      SxrBeamlineOpal2
      SxrEndstationOpal1
      SxrEndstationOpal2
      SxrFccd
   XPP:
      XppSb1PimCvd
      XppMonPimCvd
      XppSb3PimCvd
      XppSb4PimCvd


 }
{panel}

----
 Any particular hardware?

 int getFrameConfig   (FrameDetector det);

 -->  Checks that the xtc file has got data from this device.

 int getFrameValue(FrameDetector det, int& frameWidth, int& frameHeight, unsigned short*& image );

 --> fetches an image from the FrameDetector of your choice.
     Specify the detector (using an appropriate enum), give pointers to
                  int frame width, frame height and short imgage of the Camera::FrameV1.

  Other frame detectors:
  AMO:
      AmoVmi
      AmoBps1
      AmoBps2
  SXR:
      SxrBeamlineOpal1
      SxrBeamlineOpal2
      SxrEndstationOpal1
      SxrEndstationOpal2
      SxrFccd
   XPP:
      XppSb1PimCvd
      XppMonPimCvd
      XppSb3PimCvd
      XppSb4PimCvd


 -----------------------------------------------------------------------------------------------------\-
 Princeton camera
 ------------------------------------------------
 Princeton camera
 -----------------------------------------------------\-------------------------------------------------

  int getPrincetonConfig(Pds::DetInfo::Detector det, int iDevId,
                                               int& width, int& height, int& orgX, int& orgY, int& binX, int&binY);
  -->

  int getPrincetonValue(Pds::DetInfo::Detector det, int iDevId, unsigned short \*& image);
 - -->

  int getPrincetonTemperature(Pds::DetInfo::Detector det, int iDevId, float& temperature);

 -----------------------------------------------------------------------------------------------------\-
  Ipimb detector
  (Intensity Position, Intensity Monitor Board)
 -----------------------------------------------------------------------------------------------------\-
  int getIpimbConfig(Pds::DetInfo::Detector det, int iDevId);
  -->

  int getIpimbVolts(Pds::DetInfo::Detector det, int iDevId,
                                     float &channel0, float &channel1, float &channel2, float &channel3);
  -->


  -----------------------------------------------------------------------------------------------------\-
  Encoder detector
  -----------------------------------------------------------------------------------------------------\-

   int getEncoderConfig     (Pds::DetInfo::Detector det, int iDevId);
   \-->

   int getEncoderCount(Pds::DetInfo::Detector det, int iDevId, unsigned int& encoderCount);
   \-->

 -----------------------------------------------------------------------------------------------------\-
 Other functions that do not require (or have) configuration
  FCCD (Fast CCD)
 -----------------------------------------------------------------------------------------------------\-


 int getFeeGasDet  (double\* shotEnergy);
 --> Gives you the shot energy to the array shotEnergy\[4\]. This information is obtained from
     the Front End Enclosure Gas Detector.



 int getEBeam(double& charge, double& energy, double& posx, double& posy,
              double& angx, double& angy);
 int getEBeam(double& charge, double& energy, double& posx, double& posy,
              double& angx, double& angy, double& pkcurr);
  \--> Gives electron beam values for each of these doubles. The measured charge of the beam (in nC),
      the measured energy of the beam (in MeV), the 2D position of the beam (in mm) away from the origin
      (nominal beam position), and 2D angular position (in mrad) off the assumed direction. and the
      pkcurr = current? in (Amps)



 int getPhaseCavity(double& fitTime1, double& fitTime2, double& charge1,  double& charge2);
 --> Gives you the phase cavity fit time (low and high?) and charges (before and after?).

 int getPnCcdValue (int deviceId, unsigned char*& image, int& width, int& height );


 int getEvrDataNumber();

 int getEvrData    ( int id, unsigned int& eventCode, unsigned int& fiducial, unsigned int& timeStamp );



 ------------------------------------------------------------ int getFccdConfig(FrameDetector det, uint16_t& outputMode, bool& ccdEnable, bool& focusMode, uint32_t& exposureTime,
                  float& dacVoltage1, float& dacVoltage2, float& dacVoltage3, float& dacVoltage4,
                  float& dacVoltage5, float& dacVoltage6, float& dacVoltage7, float& dacVoltage8,
                  float& dacVoltage9, float& dacVoltage10, float& dacVoltage11, float& dacVoltage12,
                  float& dacVoltage13, float& dacVoltage14, float& dacVoltage15, float& dacVoltage16,
                  float& dacVoltage17,
                  uint16_t& waveform0, uint16_t& waveform1, uint16_t& waveform2, uint16_t& waveform3,
                  uint16_t& waveform4, uint16_t& waveform5, uint16_t& waveform6, uint16_t& waveform7,
                  uint16_t& waveform8, uint16_t& waveform9, uint16_t& waveform10, uint16_t& waveform11,
                  uint16_t& waveform12, uint16_t& waveform13, uint16_t& waveform14);
 --> Configures the information from the Fast CCD. Fills arguments with values depending on
     how the image/waveform data were taken.

 -----------------------------------------\-
 EPICS
 -------------------------------------------------------------
 DiodeFex
 ------------------------------------------------------\-

 Get integers, floats, strings from any EPICS channel (PV = process variable)
 int getPvInt      (const char\* pvName, int& value);
 int getPvFloat    (const char\* pvName------------------------------------------------
 int getDiodeFexConfig (Pds::DetInfo::Detector det, int iDevId, float* base, float* scale);
 -->
 int getDiodeFexValue (Pds::DetInfo::Detector det, int iDevId, float& value);
 int getPvString   (const char\* pvName, char*& value);


 /\*
  * Control data retrieval functions
  \*/
 int getControlPvNumber();
 int getControlPvName  ( int pvId, const char*& pvName, int& arrayIndex );
 int getControlValue   (const char\* pvName, int arrayIndex, double& value );

 int getMonitorPvNumber();
 int getMonitorPvName  ( int pvId, const char*& pvName, int& arrayIndex );
 int getMonitorValue   (const char\* pvName, int arrayIndex, double& hilimit, double& lolimit );

 /\*
  * Advanced Interface
  \*/
 int   getEpicsPvNumber();
 int   getEpicsPvConfig( int iPvId, const char*& sPvName, int& iType, int& iNumElements );
 int   getEpicsPvValue ( int pvId, const void*& value, int& dbrype, struct tm& tmTimeStamp, int& nanoSec );

 void  fillConstFrac(double\* t, double\* v, unsigned numSamples, float baseline,
                    float thresh, TH1\* hist);
 void  fillConstFrac(double\* t, double\* v, unsigned numSamples, float baseline,
                    float thresh, double\* edge, int& n);

   main.{hh,cc} defines a class myLevelIter, which inherits from XtcIterator. A utility class to
   traverse the xtc file. It has several process memeber functions, depending on which detector
   is being used.

   The MAIN function is making an output root file with the same name as the input xtc file.
   You can control through the myana.cc file what gets stored in the root output file.
   The main function calls the function anafile for one xtc file or for a list of xtc files.
   anafile opens the xtc file, makes and XtcFileIterator object, and loops through all the
   datagrams in the file. Each datagram corresponds to one event?


   EPICS = Experimental Physics and Industrial Control System, used for control of
   hardware, and EPICS also outputs all kinds of hardware related values, like voltages
   and currents at all given times of data collection.
   Example to get the value from a given Epics channel (e.g. "SXR:EXP:MMS1.RBV") for a given event,
   within the event() function, do:
     // get one PV
     float value;
     fail = getPvFloat( "SXR:EXP:MMS1.RBV", value );
   This gives the value to your declared float 'value'.


   * Get encoder data for DetInfo::SxrBeamline device ID 0
   \*/
  unsigned int count;
  fail = getEncoderCount(DetInfo::SxrBeamline, 0, count); -->

 ------------------------------------------------------------------------------------------------------
 Imp detector Fex (feature extraction)
 ------------------------------------------------------------------------------------------------------
 int getIpmFexConfig   (Pds::DetInfo::Detector det, int iDevId,
                       float* base0, float* scale0,
                       float* base1, float* scale1,
                       float* base2, float* scale2,
                       float* base3, float* scale3,
                       float& xscale, float& yscale);
 int getIpmFexValue   (Pds::DetInfo::Detector det, int iDevId,
                      float* channels, float& sum, float& xpos, float& ypos);

 ------------------------------------------------------------------------------------------------------
 CsPad detector
 ------------------------------------------------------------------------------------------------------
 namespace Pds { namespace CsPad { class ConfigV1; }}
 int getCspadConfig (Pds::DetInfo::Detector det, unsigned& quadMask, unsigned& asicMask);
 int getCspadConfig (Pds::DetInfo::Detector det, Pds::CsPad::ConfigV1& cfg);

 namespace Pds { namespace CsPad { class ElementV1; }}
 int getCspadQuad  (Pds::DetInfo::Detector det, unsigned quad, const uint16_t*& pixels);
 int getCspadQuad  (Pds::DetInfo::Detector det, unsigned quad, const Pds::CsPad::ElementV1*& data);

------------------------------------------------------------------------------------------------------
 Other functions that do not require (or have) configuration
 ------------------------------------------------------------------------------------------------------


 int getFeeGasDet  (double* shotEnergy);
 --> Gives you the shot energy to the array shotEnergy[4]. This information is obtained from
     the Front End Enclosure Gas Detector.



 int getEBeam(double& charge, double& energy, double& posx, double& posy,
              double& angx, double& angy);
 int getEBeam(double& charge, double& energy, double& posx, double& posy,
              double& angx, double& angy, double& pkcurr);
  --> Gives electron beam values for each of these doubles. The measured charge of the beam (in nC),
      the measured energy of the beam (in MeV), the 2D position of the beam (in mm) away from the origin
      (nominal beam position), and 2D angular position (in mrad) off the assumed direction. and the
      pkcurr = current? in (Amps)



 int getPhaseCavity(double& fitTime1, double& fitTime2, double& charge1,  double& charge2);
 --> Gives you the phase cavity fit time (low and high?) and charges (before and after?).


 int getPnCcdValue (int deviceId, unsigned char*& image, int& width, int& height );
 --> The PnCCD camera (used by CAMP) has 4 links, each link provides a 512 x 512 x 16 bit image.
     This function combines the four images to a single 1024 x 1024 x 16 bit image.


 int getEvrDataNumber();

 int getEvrData    ( int id, unsigned int& eventCode, unsigned int& fiducial, unsigned int& timeStamp );
     - eventCode tells you something about the beam quality of this event. Usually the event code is  140,
       meaning electrons were produced upstream (beam was on). It does not tell you about the photon status.
       Other codes:

     - fiducial
     - timestamp


 ------------------------------------------------------------------------------------------------------
 EPICS
 ------------------------------------------------------------------------------------------------------

 Get integers, floats, strings from any EPICS channel (PV = process variable)
 int getPvInt      (const char* pvName, int& value);
 int getPvFloat    (const char* pvName, float& value);
 int getPvString   (const char* pvName, char*& value);