Table of Contents

maxLevel	3

XtcExplorer (GUI interface to pyana)

This is a description and documentation of a GUI tool to explore the xtc data. It is written in python, relying on PyQt4 for graphical user interface. The data processing is done via the pyana framework and visualization provided by matplotlib. It can be run on any xtc file, including the file being written during data aquisition (DAQ).

...

For a similar tool to explore the HDF5 data format, look at HDF5 Explorer.

How to get started

Set up your release work directory:
You should always run this code from your own release working directory.
See the Account Setup section to set up the analysis environment. Set up your release working directory from pslogin:
Code Block
none
none
[user@pslogin ~] newrel ana-current myrelease [user@pslogin ~] cd myrelease [user@pslogin myrelease] sit_setup
Add the package:
This step is optional. Since release ana-0.3.7, xtcexplorer has been built into the release so you can skip to (3).
Checking out a local copy is beneficial, though! That will enable you to edit the source files and customise the analysis.

You need an afs token to check out the package, so if you didn't already you need to run 'kinit' first:
Code Block
none
none
[user@pslogin myrelease] kinit Password for user@SLAC.STANFORD.EDU:
Now you're ready to check out the package and "compile":
Code Block
none
none
[user@pslogin myrelease] addpkg XtcExplorer [user@pslogin myrelease] scons
Note! You can also do 'addpkg XtcExplorer VXX-XX-XX' to get a specific version of the package, or to get the most recent one, do 'addpkg XtcExplorer HEAD', but be aware that it may look different from described here, and might have fresh bugs as well as new features.
Run the program:
Launch the GUI with the command 'xtcexplorer' and optionally give the input xtc files that you want to read
as arguments. You can also browse to find files after launching the browser.

To have access to the xtc files, you need to login to psana for this:
Code Block
[user@psanaXXX myrelease] sit_setup [user@psanaXXX myrelease] xtcexplorer /reg/d/psdm/CXI/cxi22010/xtc/e60-r0465-s0*

Info

If you encounter any problems, ask for help! Don't be stuck. This is supposed to be easy. If it's not, I did something wrong, so let me know! Send me an email (ofte at slac.stanford.edu)
xtcexplorer writes out pyana configuration files, based on selections made in the GUI. Look out for files with names xb_pyana_XXXX.cfg, where XXXX is a random four-digit number. You can use them to run pyana directly from the command line. But they accumulate, so you might want to remove these regularly...
LCLS data is very diverse, and not all features for all detectors are implemented (yet). Feel free to contact me if you have requests and suggestions.

Description of the GUIs

LCLS Xtc Explorer

xtcexplorer will launch the main GUI. At startup, it has two sections:

...

	The main window before any file selection. You can select files either by editing the filename by hand or by opening the "File Browser...". If you select instrument and experiment the browser will take you directly to the right directory, or if you type in the run number it will load all files for this run number.
	A standard file browser pops up. Navigate the file browser till you find the xtc files for your experiment. (Remember, if this seems cumbersome, you can also give the file name(s) as argument to xtcexplorer when you first launch the GUI)
	Mark the files you want to look at (Hold down the Ctrl-key to select more than one file). Note, you should only look at one run number at a time, since different runs might have different configurations.
	This is the main window after a file has been selected. File name and file size is shown in the GUI. If the file is not too big, you can click the "Scan File(s)" button to get exact contents of the whole file. If the file is big, it's better to do a "Quick Scan" which will tell you all you need to know (except count number of events and calibration cycles).

Pyana Control Center

After scanning, a new GUI will pop up showing you a list of detectors/devices found in the file. A little more information is written to the terminal window too.

...

More information on how to run pyana by itself (see 'pyana -h' for more help, or the pyana section of confluence).

The pyana modules

The GUI as described above prepares a configuration file for you to run pyana. You can either run it from the GUI or you can run it from the command line. The configfile sets input parameters for the pyana modules in this package. Feel free to use one or more of these modules as a starting point for your more elaborate pyana analysis!

...

A few things to note about the different detectors / pyana modules:

Common configuration parameters

A few parameters are common to all modules, and their default values are set to be the same, but can also be set individually for each module. They are the following:

Code Block

	none
	none

# parameter name   default value   comment
  plot_every_n      = 100         # If 0, plot only at the end. Else, display every N events
  accumulate_n      = 0           # If 0, accumulate all (no reset). Else, reset stored arrays every N events
  fignum            = 1           # matplotlib figure number

pyana_plotter

This module is added to the end of the job, after other modules. It controls the display mode and ipython.

The default display mode is SlideShow mode, where the plot window stays open but updates every n events. In Interactive mode, the figure window must be closed before the next event gets processed. The other modules are set up to work with this one, but should with minor modifications also run fine without it. (The minor modifications would be to issue a "plt.ion()" before the figures are created or replace "plt.draw()" with "plt.show()" to make sure the plot windows stay open.)

There's also a handle to allow you to use ipython at the end of the job (or after each plot) to investigate your plottet arrays further.

Code Block

	none
	none

  display_mode      = 2           #  Interactive (1) or SlideShow (2) or NoDisplay (0)
  ipython           = False       #  Drop into ipython after each plot (to allow further investigation of data)

pyana_bld

This module collects data from the BeamLine Detectors (Bld) and plots them in separate windows.

...

pyana -n 300 -c xb_pyana_5299.cfg -m XtcExplorer.pyana_bld -m XtcExplorer.pyana_plotter <xtc files>

pyana_waveform

Plots waveform data from Acqiris digitizers. Takes a list of addresses, currently will make one plot for each channel of each device. Plan to add more options.

...

pyana -n 300 -c xb_pyana_5299.cfg -m XtcExplorer.pyana_waveform <xtc files>

pyana_cspad

CsPad data is reconstructed in pyana_cspad.py.

...

pyana -n 10 -c xb_pyana_5299.cfg -m XtcExplorer.pyana_cspad -m XtcExplorer.pyana_plotter <xtc files>

pyana_image.py

This module processes generic camera frames, e.g from Pulnix TM6740 device. It allows any number of images, given as a space-separated list of addresses in the configuration file.

...

pyana -n 10 -c xb_pyana_5299.cfg -m XtcExplorer.pyana_image -m XtcExplorer.pyana_plotter <xtc files>

pyana_ipimb

This module extracts and displays data from IPIMB type detectors. There are two types of information: IpimbData (raw data) and IpmFex (feature extracted data). Only the last type of data is currently implemented, but other data can also be added. Currently the display will show a plot of the intensity measured in each of the four channels, the beam spot position estimated from sums and differences of the individual intensity measurements, and the sum of the four measurements as a function of time. The plot included below has these plots for 4 different IPIMB detector sources.

...

pyana -n 300 -c xb_pyana_5299.cfg -m XtcExplorer.pyana_ipimb -m XtcExplorer.pyana_plotter <xtc files>

pyana_epics

Code Block

	none
	none

pv                  # Name(s) of the EPICS PV(s) to dump

...

pyana -n 200 -c xb_pyana_5299.cfg -m XtcExplorer.pyana_epics -m XtcExplorer.pyana_plotter <xtc files>

pyana_scan

This module is different from the others. In runs with several calibration cycles and one (or more) control PV (process variable), this module displays scalar values from other detectors as a function of scan step of the control PV. It currently takes two scalar type inputs to evaluate the scan:

...

pyana -n 200 -c xb_pyana_5299.cfg -m XtcExplorer.pyana_scan -m XtcExplorer.pyana_plotter <xtc files>

Anchor

	IPython
	IPython

Interactive plotting with IPython

Since XtcExplorer V00-01-04, you also have the option of launching ipython at the end of the job. The basic support for this has been added to all modules. Each pyana module stores some data (primarily as numpy arrays) which it passes to the pyana_plotter module which launches iPython. In the session you can investigate the data further and make additional plots.

...

A couple of close-ups of the ipython terminal:

xtcscanner

This tool also belongs to the XtcExplorer package, and is used by the GUI. But the tool can also be run directly from the command line:

...

Code Block

	none
	none
title	xtcscanner -n 200 /reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s0*

Scanning....
Start parsing files:
['/reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s00-c00.xtc', '/reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s01-c00.xtc']
  201 datagrams read in 0.070000 s .   .   .   .   .   .   .
-------------------------------------------------------------
XtcScanner information:
  - 1 calibration cycles.
  - Events per calib cycle:
   [197]

Information from  0  control channels found:
Information from  9  devices found
                      BldInfo:EBeam:             EBeamBld (197)
            BldInfo:FEEGasDetEnergy:             FEEGasDetEnergy (197)
        DetInfo:AmoETof-0|Acqiris-0:  (5 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
      DetInfo:AmoGasdet-0|Acqiris-0:  (2 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
        DetInfo:AmoITof-0|Acqiris-0:  (1 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
        DetInfo:AmoMbes-0|Acqiris-0:  (1 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
     DetInfo:EpicsArch-0|NoDevice-0:             Epics_V1 (688)
         DetInfo:NoDetector-0|Evr-0:             EvrConfig_V2 (1)
                          ProcInfo::             RunControlConfig_V1 (11)
XtcScanner is done!
-------------------------------------------------------------

Further analysis with `pyana`

Any serious data analysis will need more customized tools than we can provide in this general GUI interface.

This will require the user / analyst to program his/her own tools. Pyana is a complete framework for programming a user analysis in python.
"Blank" analysis code can be generated with Andy's codegen script (try codegen -h and codegen -p for options). Extensive information about pyana is available on confluence.

Running this package's pyana modules outside of the XtcExplorer

Feel free to use any or all of the XtcExplorer modules as examples or starting point for your analysis. If you have suggestions for features or algorithms that might benefit others too, let me know and perhaps we can add it to the XtcExplorer package.

...

pyana -n 300 -c xb_pyana_5299.cfg -m XtcExplorer.pyana_module <xtc files>

Submitting to batch

Once you've identified the command line to run your pyana job, you can submit this to the LFS batch queue system as described in Computing.

...

Configure pyana to plot as little as possible by setting the display mode to NoDisplay
Add the following two lines to XtcExplorer/src/utilities.py :
import matplotlib
matplotlib.use('PDF')

A few words on time and performance

The explorer hasn't been optimized for speed properly, but I strive to keep the pyana examples relatively efficient. But keep in mind
that plotting with matplotlib does take a substansial amount of time. An example:

...

Page tree

Versions Compared

Old Version 162

New Version 163

Key

XtcExplorer (GUI interface to pyana)

How to get started

Description of the GUIs

LCLS Xtc Explorer

Pyana Control Center

The pyana modules

Common configuration parameters

pyana_plotter

pyana_bld

pyana_waveform

pyana_cspad

pyana_image.py

pyana_ipimb

pyana_epics

pyana_scan

Interactive plotting with IPython

xtcscanner

Further analysis with `pyana`

Running this package's pyana modules outside of the XtcExplorer

Submitting to batch

A few words on time and performance

Page tree

Page History

Versions Compared

Old Version 162

New Version 163

Key

XtcExplorer (GUI interface to pyana)

How to get started

Description of the GUIs

LCLS Xtc Explorer

Pyana Control Center

The pyana modules

Common configuration parameters

pyana_plotter

pyana_bld

pyana_waveform

pyana_cspad

pyana_image.py

pyana_ipimb

pyana_epics

pyana_scan

Interactive plotting with IPython

xtcscanner

Further analysis with pyana

Running this package's pyana modules outside of the XtcExplorer

Submitting to batch

A few words on time and performance

Further analysis with `pyana`