• Created by Unknown User (ofte), last modified on May 12, 2011

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 105 Next »

XtcExplorer (GUI interface to pyana)

Note! This package is a new version of the "XtcEventBrowser" which is the old name of the package.

This is a description and documentation of a GUI tool to explore the xtc data. It is written in python, relying on PyQt4 for graphical user interface. The data processing is done via the pyana framework and visualization provided by matplotlib. It can be run on any xtc file, including the file being written during data aquisition (DAQ).

Xtc files contain the raw data (datagrams) streamed from the DAQ online system. The datagrams occur in the order they were flushed from the DAQ system, and thus do not necessarily appear in chronological order. The datagrams are not indexed, so the file must be read sequentially. The pyana and psana frameworks are designed to process these datagrams and present them to you shot by shot (or event by event, in pyana/psana terminology).

The XtcExplorer allows a simple-to-run interface to the xtc files. The package name is XtcExplorer, the executables (xtcexplorer and xtcscanner) are found in the app subdirectory of this package, and all other source code is in the src subdirectory.

How to get started

You should always run this code from your own release working directory. It might work from other directories, but be aware that configuration files will be written to your current directory. Look out for files with names xb_pyana_XXXX.cfg, where XXXX is a random four-digit number. You can use them to run pyana directly from the command line. But they accumulate, so you might want to remove these regularly...

See the Account Setup section to set up the analysis environment. Set up your release from pslogin:

[user@pslogin ~] newrel ana-current myrelease
[user@pslogin ~] cd myrelease
[user@pslogin myrelease] sit_setup

Optional: Add the XtcExplorer package to your analysis release and "compile":

[user@pslogin myrelease] addpkg XtcExplorer V00-01-01
[user@pslogin myrelease] scons

Note! If you don't check out this local copy, you will run whichever version is in the release. You will not be able to edit any of the source files to tweak your analysis.
Note! You can omit the "tag" (VXX-XX-XX) to get the latest and greatest version in the svn repository, but this may look different from described here.

Run the program with the command 'xtcexplorer' and optionally give the input xtc files that you want to read as arguments. You can also browse to find files after launching the browser.

To have access to the xtc files, you need to login to psana for this:

[user@psanaXXX myrelease] sit_setup
[user@psanaXXX myrelease] xtcexplorer /reg/d/psdm/cxi/cxi80410/xtc/e55-r0581*

If you encounter any problems, ask for help! Don't be stuck. This is supposed to be easy. If it's not, I did something wrong, so let me know! (smile) Send me an email (ofte at slac.stanford.edu)

Description of the GUIs

LCLS Xtc Explorer

xtcexplorer will launch the main GUI. It currently has two sections:

  • File section: Shows a list of currently selected file(s). As you may have guessed, "File Browser" opens a file browser and "Clear File List" clears the current list of files. This section also allows you to add a file name by hand (or paste).
  • Scan section: The two buttons to the left allows you to scan the xtc file to get a summary of what datagrams are stored in it. Note, for most purposes, a "Quick Scan" is sufficient. If you need to scan the whole file, e.g. if you want to know the total number of events, number of calibration cycles, etc, you can enable the "Scan File(s)" button. If the files are big, this will take a lot of time...

Main window before any file selection.
Click on "File Browser..." to select file(s).

Main window after a file has been selected. File name and file size is shown in the GUI. If the file is not too big, you can click the "Scan File(s)" button to get exact contents of the whole file. If the file is big, it's better to do a "Quick Scan" which will tell you all you need to know (except count number of events and calibration cycles).

Pyana Control Center

After scanning, a new GUI will pop up showing you a list of detectors/devices found in the file. A little more information is written to the terminal window too.

Main window (top) after the file scan. In this case the file contain several "calibration cycles" (motor scan steps), and the GUI lists number of calibration cycles and number of events. Some more information is printed to the terminal window from which the xtcbrowser was launched.

Another window, pyana control center (bottom), also pops up, which has a few fields. "In the file(s):" In front of each detector/device name is a checkbox, where you can select which datagrams you are interested in analysing / plotting. To the right of this is a field with some general information and where you can set general parameters for pyana processing and plotting in this GUI, among them how often to update plots (default is every 100 events).

Once you checkmark the detectors you want to display information from, another tab will pop up showing pyana configuration text. "Current pyana configuration": as you select devices from the list, a tentative configuration file for running pyana is written and shown in this field.

If a ControlPV is present and checked off, only a pyana_scan module will be used. All the other devices you check will be added to the input of the scan.
If no ControlPV is used, other pyana modules will be configured as appropriate to display a variety of information from the events.
If "Epics Process Variables" are checked off, another Gui appears that lists all the epics variables. Select the ones you want to display.

Press the "Write configuration to file" button once you're done. You can further edit the file by hand if you want. Once a file is written, a "Run pyana" button will appear.

"Run pyana" lauches an input GUI that shows you the runstring. You can use the same runstring from the command line. Or hit "OK" and it'll run.

After launching pyana, another button "Quit pyana" appears... If you see you need to change parameters, you can stop pyana, edit the configuration file, and start over again.

More information on how to run pyana by itself (see 'pyana -h' for more help, or the pyana section of confluence).

The pyana modules

The GUI as described above prepares a configuration file for you to run pyana. You can either run it from the GUI or you can run it from the command line. The configfile sets input parameters for the pyana modules in this package. Feel free to use one or more of these modules as a starting point for your more elaborate pyana analysis!

XtcExplorer/src/pyana_bld.py          # display of Beam-line data
XtcExplorer/src/pyana_cspad.py        # display of CsPad image data
XtcExplorer/src/pyana_image.py        # display of camera image data
XtcExplorer/src/pyana_ipimb.py        # display of diode data from IPIMB and PIM
XtcExplorer/src/pyana_waveform.py     # display of waveform data
XtcExplorer/src/pyana_epics.py        # display of Epics PV data
XtcExplorer/src/pyana_scan.py         # display of motor scan data
XtcExplorer/src/pyana_plotter.py      # a plotter module to control the event display

A few things to note about the different detectors / pyana modules:

Common configuration parameters

A few parameters are common to all modules, and their default values are set to be the same, but can also be set individually for each module. They are the following:

# parameter name   default value   comment
  plot_every_n      = 100         # If 0, plot only at the end. Else, display every N events
  fignum            = 1           # matplotlib figure number

pyana_plotter

This module is added to the end of the job, after other modules. It does nothing (yet) than control the display mode. Default is SlideShow mode. There's also a handle to allow you to use ipython at the end of the job (or after each plot) to investigate your plottet arrays further. This is not functional yet. 

  display_mode      = 2           #  Interactive (1) or SlideShow (2) or NoDisplay (0)
  ipython           = False       #  Drop into ipython after each plot (to allow further investigation of data)

pyana_bld

This module collects data from the BeamLine Detectors (Bld) and plots them in separate windows. In SlideMode, the plots accumulate from one plot to the next.
To run this module by itself with pyana:

pyana -m XtcExplorer/src/pyana_waveform.py <xtc files> 

 do_ebeam           = False       #  Plot data from EBeam object
 do_gasdetector     = False       #  Plot data from GasDetector
 do_phasecavity     = False       #  Plot data from PhaseCavity
 do_ipimb           = False       #  Plot data from SharedIpimb

pyana_waveform

Plots waveform data from Acqiris digitizers. Takes a list of addresses, currently will make one plot for each channel of each device.
To run this module by itself with pyana:

pyana -m XtcExplorer/src/pyana_waveform.py <xtc files>

Configurable options:

  sources           =             # string, space-separated list of addresses (Detector-Id|Device-ID)

pyana_cspad

CsPad data is reconstructed in pyana_cspad.py. The image plot value limits are adjusted automatically, but if
you want to change them, click on the color bar (left-click for low limit, right-click for high limit).
The successive events will be plotted with the new limits. Revert to the original by middle-clicking.
To run this module by itself with pyana:

pyana -m XtcExplorer/src/pyana_cspad.py <xtc files>

Options must be specified in a configuration file. Only the img_sources parameter is required.

  img_sources       =             # string, Address of Detector-Id|Device-ID
  dark_img_file     =             # filename, Dark image file to be loaded, if any
  out_img_file      =             # filename (If collecting: write the average image to this file)
  plot_vrange       =             # range=vmin,vmax of values for plotting (pixel intensity)
  threshold         =             # threshold for image intensity in active threshold area
  thr_area          =             # active threshold area (range=xmin,xmax,ymin,ymax). None == whole image.

Image display of the CSPad detector, background subtracted. This is currently the only display that has interactive features. A filter allows you to select events within requested intensity range.

pyana_image.py

This module processes generic camera frames, e.g from Pulnix TM6740 device. It allows any number of images, given as a space-separated list of addresses in the configuration file.

  • You can set ranges to define good images and dark images. If both are set, you have the option to display good images background subtracted, where background subtraction is based on the average of background images so far collected.
  • Each image can be separately rotated, shifted and scaled (zoomed in/out).
  • Nicknames can be given to the input images. Defaults are Im1, Im2... etc. These names will be used if you plot differences, or other manipulations of the original images.
  • The images are subtracted and differences displayed as well as fourier transform of differences. Examples of what may be displayed. To display other things, at this stage you have to edit pyana_image.py to change this behaviour. 
     image_addresses     # address string of Detector-Id|Device-ID
 good_range          # threshold values selecting images of interest (Format: low,high)
 dark_range          # threshold values selecting dark images (Format: low,high)
 image_rotations     # rotation angle, in degrees, to be applied to image(s)
 image_shifts        # npixel shifts, format (nx,ny), to be applied to image(s)
 image_scales        # scale factor (float) to be applied to images
 image_nicknames     # nicknames for plot titles
 image_manipulations # String containing keywords: Diff, FFT
 output_file         # filename. Valid extensions are .hdf5, .txt (ascii) or .npy (numpy binary)
 n_hdf5              # if output file is hdf5, combine n events in each output file.

    Displays of three different Pulnix TM6740 images of YAG screens, after rotation/translation. Also shown, differences between images and FFT of differences.

pyana_ipimb

ipimb_addresses     # list of IPIMB addresses

pyana_epics

pv                  # Name(s) of the EPICS PV(s) to dump

pyana_scan

This module is different from the others. It does a "motor scan", displays certain values as a function of scan step of a motor scan. The motor name is given in the xtc file as the Control PV. There may be more than one control PV. It currently takes two scalar type inputs to evaluate the scan:

input_epics         # Name(s) of other scalars to correlate in scan
input_scalars       # Name(s) of other scalars to correlate in scan

xtcscanner

This tool also belongs to the XtcExplorer package, and is used by the GUI. But the tool can also be run directly from the command line: 

usage: xtcscanner [options] xtc-files ...

options:
  -h, --help            show this help message and exit
  -n NDATAGRAMS, --ndatagrams=NDATAGRAMS
  -v, --verbose
  -l L1_OFFSET, --l1-offset=L1_OFFSET

Example:

xtcscanner -n 200 /reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s0*
Scanning....
Start parsing files:
['/reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s00-c00.xtc', '/reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s01-c00.xtc']
  201 datagrams read in 0.070000 s .   .   .   .   .   .   .
-------------------------------------------------------------
XtcScanner information:
  - 1 calibration cycles.
  - Events per calib cycle:
   [197]

Information from  0  control channels found:
Information from  9  devices found
                      BldInfo:EBeam:             EBeamBld (197)
            BldInfo:FEEGasDetEnergy:             FEEGasDetEnergy (197)
        DetInfo:AmoETof-0|Acqiris-0:  (5 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
      DetInfo:AmoGasdet-0|Acqiris-0:  (2 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
        DetInfo:AmoITof-0|Acqiris-0:  (1 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
        DetInfo:AmoMbes-0|Acqiris-0:  (1 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
     DetInfo:EpicsArch-0|NoDevice-0:             Epics_V1 (688)
         DetInfo:NoDetector-0|Evr-0:             EvrConfig_V2 (1)
                          ProcInfo::             RunControlConfig_V1 (11)
XtcScanner is done!
-------------------------------------------------------------

Further analysis with pyana

Any serious data analysis will need more customized tools than we can provide in a GUI interface. This will require the user / analyst to program his/her own tools. Pyana is a complete framework for programming a user analysis in python. The Gui Event Browser can provide simple analysis code that can be expanded by the user. "Blank" analysis code can also be generated with Andy's codegen script (try codegen -h and codegen -p for options).

More information about pyana can be found on confluence.

  • No labels