1. About this Tutorial

The main objective of this session is to introduce and to explain the new Python interface for accessing LCLS data. This software framework is known as the "Interactive psana". The first idea of implementing such tool was suggested around 1.5 years ago at the joint PCDS/SRD meeting (look for Interactive Psana). Though its underlying machinery is largely based on the batch version of psana modules the interface to the data is simpler, more intuitive, and it requires less code to be written by a user in order to get to that CSPad image or that EPICS PV. The new framework won't work for everyone, specifically for those users who have either heavily invested into the modular code of the batch framework, or who need the performance of modules written in C++. Still our intent is to demonstrate the power of the new approach and to encourage using the tool where appropriate.

...

Users of pyana and myana, attention: The pyana and myana frameworks will be phased out at some point. As our understanding of what kind of analysis framework works better for the users and for ourselves as developers grew evolved, we realized that we needed to develop a tool which would have an easier interface to the data, a better internal architecture and be easier to maintain and extend for new data types. Hence we developed psana modules framework which has a number of advantages:

...

The last two features are opening the powerful possibility of using psana for real-time monitoring of data while benefiting from reusing the same code which might be developed for the traditional OFFLINE processing/analysis.

2. Getting Started

Example Location

All examples can be found under the following directory:

/reg/g/psdm/tutorials

Data Files

In order to make our examples as close to the "real" analysis environment as possible we chose to create 6 pseudo experiments (one per instrument):

...

All data files are open for reading by anyone who can log onto LCLS computers. Moreover, those directories (like scratch/, ftc/) are open for writing by anyone. And yes, one can also see these experiments in the Web Portal.

Setting up the Environment

• Make sure you can run X11 applications. Most examples of this tutorial will do a simple visualization. You can pass the -X or -Y argument to ssh to make sure you can forward the screens to your local machine. Log onto any machine of the interactive pools psananeh or psanafeh, eg:

  Code Block
  ssh -Y psananehpsana

• Make sure you source (just once) one of the following scripts. When using the bash shell:

  Code Block
  . /reg/g/psdm/etc/ana_env.sh

  Or, when using the csh shell family:

  Code Block
  source /reg/g/psdm/etc/ana_env.csh

  Note that the default shell for most LCLS users is bash.

  Run (just once) the following command which will set up a proper analysis environment for the latest analysis release:

  Code Blocksit_setup ana-current

   This gives you access to the latest psana software release (there are other commands to use an older, or local release)

   

At this point you are ready to go. To test that the your environment is set up correctly try running psana without any parameters. If your environment is properly set you should see something like this:

psana
[error:2013-06-06 20:54:44.131:PSAnaApp.cpp:218] no analysis modules specified

3. Basic Examples

This section presents a few simple scripts which have been developed to underline the main ideas behind the framework's API. The code of the examples along with a simple HOWTO file can be found at:

/reg/g/psdm/tutorials/common/data_access_methods/

Printing Identifiers for all Events in a Run

First try this:

./print_event_id.py

...

./print_event_id.py
     0: XtcEventId(run=366, time=2013-04-21 04:37:39.343773772-07, fiducials=38877, ticks=329342, vector=19553)
     1: XtcEventId(run=366, time=2013-04-21 04:37:39.360457259-07, fiducials=38883, ticks=331442, vector=19554)
     2: XtcEventId(run=366, time=2013-04-21 04:37:39.377123777-07, fiducials=38889, ticks=330560, vector=19555)
     3: XtcEventId(run=366, time=2013-04-21 04:37:39.393797466-07, fiducials=38895, ticks=329762, vector=19556)
     4: XtcEventId(run=366, time=2013-04-21 04:37:39.410477971-07, fiducials=38901, ticks=331204, vector=19557)
     5: XtcEventId(run=366, time=2013-04-21 04:37:39.427145705-07, fiducials=38907, ticks=331036, vector=19558)
     6: XtcEventId(run=366, time=2013-04-21 04:37:39.443816588-07, fiducials=38913, ticks=329370, vector=19559)
     7: XtcEventId(run=366, time=2013-04-21 04:37:39.460499778-07, fiducials=38919, ticks=331414, vector=19560)
     8: XtcEventId(run=366, time=2013-04-21 04:37:39.477167658-07, fiducials=38925, ticks=330616, vector=19561)
     9: XtcEventId(run=366, time=2013-04-21 04:37:39.493840079-07, fiducials=38931, ticks=329720, vector=19562)
    10: XtcEventId(run=366, time=2013-04-21 04:37:39.510520348-07, fiducials=38937, ticks=331218, vector=19563)

Printing a Catalog of Event Components

The example can be run like this:

...

You already encountered one of these get functions in the very first example where you were extracting the event identifier. The type field indicates what kind of data you are accessing, the source field indicates the instance of that particular detector. In this example there are two acqiris digitizers with the same data type. Note that event components obtained through this API will be objects of various classes. A full catalog of those objects can be found in the DOXYGEN documentation which is auto-generated from the code of the OFFLINE releases.

Iterating over steps and events

Some of our experiments (in particular XPP) are heavily relying on so called steps, which are also known as Calibration Transitions. Each DAQ run has one or many such steps. Events are recorded in a scope of a particular step. The new framework has a special provision for scans through the iterator of steps. The idea behind the following example is:

...

 These two scripts perform the same processing, the only difference being which data format they're accessing. The first example will read XTC files, while the second one will read HDF5 files. When running these examples you will notice that the HDF5 version is significantly faster. This is due to the fact that we don't yet support indexing for XTC files.

Extracting values of EPICS variables

In this example we will demonstrate how to access a value of an EPICS variable. The application will monitor changes in the value and report event numbers at which changes happen:

...

pv = epics.value('VGP:FEE1:311:P')
pv = epics.value('VGP:FEE1:311:P',0)
pv = epics.getPV('VGP:FEE1:311:P').value(0)

4. Instrument Specific Examples

This section includes a number of examples which are relevant to different instruments. Their primary goal is to illustrate how to access data objects specific to each instrument.

XCS

Princeton Movie

The code for these examples is found at:

/reg/g/psdm/tutorials/xcs/princeton_movie/

SXR

Correlation Plots for Signals from GDM and Diode

The code for these examples is found at:

/reg/g/psdm/tutorials/sxr/gmd_vs_diode/

CXI

Diffraction Patterns on the CSPad Detector

The code for these examples is found at:

...

./dump_2x1_elements.py
./frame_reco.py
./frame_reco_calib.py

5 Wrapping up

Where to look for support

• send e-mail to: pcds-help@slac.stanford.edu

Documentation

• DOXYGEN documentation for psana
• Batch version of psana
• psana - Module Catalog - comprehensive catalog of psana modules in the latest Analysis Release
• psana - Module Examples