You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

The timetool camera measures the time difference between laser and FEL in one of two methods:

  1. spatial encoding, where the X-rays change the reflectivity of a material and the laser probesthat change by the incident angle of its wavefront; or
  2. spectral encoding, where the X-rays change the transmission of a material and the chirped laser probes it by a change in the spectral components of the transmitted laser.

TimeTool results can be computed by the DAQ while data is being recorded and written directly into the .xtc files, or after data has been recorded. In AMO/XPP the time tool analysis is done by the DAQ while data is being recorded. Results can be accessed using the following epics-variables names: TTSPEC:AMPL , TTSPEC:AMPLNXT, TTSPEC:FLTPOS, TTSPEC:FLTPOSFWHM, TTSPEC:FLTPOS_PS, TTSPEC:REFAMPL or by getting the appropriate Psana.TimeTool.DataV* object from the event (either V1 or V2) from the opal camera source.

This document covers how to run the TimeTool after data has been recorded. Starting with ana-0.16.9, a Python wrapper to the TimeTool is provided which is the preferred way to run the TimeTool algorithms. Previous to ana-0.16.9 one would use Psana modules. One would loadi the C++ TimeTool.Analyze module via a psana config file.

Note: specific examples for the time tool can be found in /reg/g/psdm/sw/releases/ana-current/TimeTool/examples/.

Python Interface

In the example directory mentioned above, see:

  • pyxface_evr_bykick.py
  • pyxface_calib_db_ref.py

The python interface contains extensive documentation in its docstrings. To access this, from a ipython session, do

import TimeTool

TimeTool.PyAnalyze?
TimeTool.AnalyzeOptions?

Further documentation can be found on the PyAnalyze functions.

The docstrings will contain the most current documentation. For convenience, we include this documentation here as well:

PyAnalyze

Python interface to the TimeTool.Analyze C++ Psana module to allow conditional execution.

Basic Usage

There are several steps to using the module demonstrated in this example:

ttOptions = TimeTool.AnalyzeOptions(get_key='TSS_OPAL',
                                    eventcode_nobeam = 162)
ttAnalyze = TimeTool.PyAnalyze(ttOptions)
ds = psana.DataSource(self.datasource, module=ttAnalyze)

for evt in ds.events():
    ttResults = ttAnalyze.process(evt)

The steps are

  • Create an instance of the AnalyzeOptions class. See AnalyzeOptions docstring for detailed documentation on options.

  • Construct an instance of PyAnalyze (called ttAnalyze above) by passing this options object (called ttOptions above).

  • construct your psana DataSource by passing the PyAnalyze instance through to the module argument.

  • call the ttAnalyze.process() method on each event you want to process.

Parallel Processing

When doing parallel processing and distributing events among different ranks, each rank typically processes a fraction of all the events. Howevever it is important that each rank also process all events that include reference shots for the TimeTool. One can check if a shot is a reference shot or not with the isRefShot(evt) function. For example, in the above, suppose the variables numberOfRanks and rank are defined so that we can implement a round robin strategy to distribute event processing. One could implement this, while making sure all ranks process reference shots, as follows:

        for idx, evt in enumerate(ds.events()):
            if ttAnalyze.isRefShot(evt): 
                ttAnalyze.process(evt)
            if idx % numberOfRanks != rank: 
                continue
            ttResults = ttAnalyze.process(evt)

Note that it is Ok to call the PyAnalyze.process(evt) function more than once on the same event. The PyAnalyze class caches the results of the first call so as to not call the underlying C++ TimeTool.Analyze module twice.

Controlling Beam Logic

This is an unsual use case. It happens when users are using EPICSs to move things around to drop the beam, rather than using an event code. In this case the user will want to look at the EPICS variables and instruct the TimeTool of when the beam is off so that it can build up its reference. To do this, set controlLogic=True in AnalyzeOptions and use the controlLogic(beamOn, laserOn) function. When controlLogic is true, you must call this function for each event you subsequently call process on (if you forget, TimeTool will stop with an error).

AnalyzeOptions

Specify configuration options for the TimeTool Analyze module.

There are many options for TimeTool.Analyze, most can be left at their default values. First we document options users may want to change, then more specialized options.

Note that all options are either a Python str, or Python int, or float. Even though some options represent a list of numbers, the argument must be formatted as a string with only whitespace separating numbers (no commas, etc).

Common Options

    Option    default    data type / explanation
    ------    -------    -----------
    get_key  'TSS_OPAL'  str, this is the psana source for the TimeTool
                         opal camera. The default, 'TSS_OPAL', is a common
                         DAQ alias for this source
    
    eventcode_nobeam 0   int, for BYKICK experiments, where an evr code 
                         specifies when the beam is NOT present, specify that
                         EVR code here. If the beam is always present, keep
                         the default value of 0. Note - there are specialized
                         options to control laser on/off beam on/off logic 
                         below.
    ref_avg_fraction 0.05 float, weight assigned to next reference shot in
                          rolling average. 1.0 means replace reference with
                          next shot (no rolling average). 0.05 means next 
                          shot is 5% of average and all previous is 95%.
    sig_roi_y '425 725' str, signal roi in y, or the rows of the Opal camera
                         that the user may want to adjust based on where signal is.
    sig_roi_x   '0 1023' str, signal roi in x or columns, default is all.
    calib_poly '0 1 0'   str, TimeTool.Analyze returns results as both a pixel
                         location on the Detector, as well as a conversion 
         to femtoseconds by applying this quadratic polynomial. Typically a 
         special calibration run is performed to compute the mapping from 
         position on the detector to femtoseconds and the results of this 
         analysis are passed through this parameter. If calib_poly is 'a b c'
         Then femtosecond_result = a + b*x + c*x^2 wher x is the detector position.
    ** Uncommon Options**
    Option    default    data type / explanation
    ------    -------    -----------
    projectX   True     bool, if true, project down to the X axis of the opal.
    eventcode_skip  0   int, EVR code for events that should be skipped from
                        TimeTool processing.
    ipm_get_key    ''   str, in addition to the evr code above, the timetool 
                        will look at the threshold on a specified ipmb to decide
                        if the beam is present. Default is to not look at a 
                        ipmb. Here one can specify the psana source for the
                        desired ipimb.
                 
    ipm_beam_threshold  float, threshold for determining if beam present from
                        an ipm.
                       
    weights  '0.00940119 ...' str, this defaults to a long string of the 
                        weights that the TimeTool uses when performing signal
                        processing on the normalized, reference divided signal,
                        to turn a sharp drop into a peak. It is unusual for
                        users to modify this string. The full string can be
                        found in the code.
    weights_file ''     str, the weights can be put into a file as well
    use_calib_db_ref  False  bool, get the initial reference from the 
                       calibration database. This reference can be deployed
                       using calibman by making a pedestal for the appropriate
                       opal. Example use case is creating references not by
                       dropping the beam from shots, but rather making 
                       reference runs (dropped shots tend to give better results).
    
    ref_load  ''       str, filename to load reference file from.
    ref_store ''       str, filename to store reference into.
    controlLogic      False,  bool, to bypass the normal mechanism of letting
                      the timetool identify when the beam or laser is on, 
                      the user can control this. If this is set, then for
                      every event, one must call controlLogic before calling
                      process with the TimeTool.Analyze python class.
    proj_cut   -(1<<31), int, after projecting of the opal camera to create the 
                     signal, one can require that at least one value in the signal
                     be greater than this parameter in order to continue processing.
                     Default is int_min, which means one always processes when the
                     laser is on (and a reference is available).
    sb_roi_x   '', str  give something like 1 10 to do sideband analysis and specify
                   the sideband ROI. This feature, while it exists in the code, is not
                   maintained. Experts interested in this feature may need to contact
                   their POC for support.
                   
    sb_roi_y   '', str, see sb_roi_x, this is for the y region (rows)
    sb_avg_fraction 0.05, float, see sb_roi_x and sig_avg_fraction for information on
                    rolling averages.
    
    analyze_event -1, int, special options for analyzing the first few events with respect
                  to a reference - feature not maintained.
    dump         0, int, setting this option is not reccommended. It instructs TimeTool
                 to use the psana root based historgramming method - however the root
                 files interfere with MPI based analysis. See the eventdump option below
                 to access intermediate stages of TimeTool processing.
    eventdump   False, bool, setting this option to True will cause the underlying C++
                Psana TimeTool.Analyze module to return extra data that the Psana PlotAnalyze
                module can use (or users can get directly) but presently, the Python wrapper
                does not expose this, or setup the plotter in a convenient way. However One can
                get the ndarrays directly from the event following a call to PyAnalyze.Process().
    psanaName TimeTool.Analyze, str, the logging name passed to this instance of the
                underlying C++ Psana Module called TimeTool.Analyze. There should be no
                reason to modify this - unless for some reason you want to configure two
                separate instances of the Psana Module.
    put_key TTANA, str, should be little reason to modify this. This is the key used to get
                 results back from the C++ TimeTool.Analyze module.
    beam_on_off_key ControlLogicBeam, str, there should be no reason to modify this.
                if controlLogic is true, this is the internal key string used to 
                communicate beam on/off with the TimeTool.Analyze module.
    laser_on_off_key ControlLogicLaser, str, as above, should be no reason to modify this.

Users migrating older code will need to discard psana config files, or remove the TimeTool.Analyze configuration from the psana config file AS WELL AS THE TimeTool.Analyze C++ Psana Module from the list of psana modules in the config file. If mixing old style Psana modules with PyAnalyze, be advised that PyAnalyze will run after any modules listed in the psana config file (so the TimeTool results will not be available to Psana modules listed in the config file). It should be easy to move Python Psana modules into the new style (C++ modules are not easy to move).

C++ Psana Module Interface

Next we disuss the older, C++ Psana module interface to the TimeTool. The package includes sample configuration files that describe all the options. From a psana release directory, users are encouraged to add the TimeTool package to obtain the latest source. For instance:

newrel ana-current myrel
cd myrel
kinit             # get ticket to check out package from svn repository
addpkg TimeTool
sit_setup
scons
# now examine the files in TimeTool/data:  sxr_timetool.cfg  sxr_timetool_setup.cfg  timetool_setup.py  xpp_timetool.cfg  xpptut.cfg

 timetool_setup.py is a python script to calculate the digital filter weights.

Module Analyze

A module that analyzes the camera image by projecting a region of interest onto an axis and dividing by a reference projection acquired without the FEL.  The resulting projection is processed by a digital filter which yields a peak at the location of the change in reflectivity/transmission.  The resulting parameters are written into the psana event. The type of the parameter depends on the release. Starting with ana-0.13.10, a TimeTool::DataV2 object in put in the event store. ana-0.13.3 put a TimeTool::DataV1 object in the event store. In ana-0.14.4 and later, this is how one gets the data, a TimeTool::DataV2 object. Older releases would also add the output as doubles or ndarrays, but this is no longer the case with ana-0.14.4 and later.

Accessing Results from Analyze (ana-0.14.4 and later)

ttData = evt.get(TimeTool.DataV2, self.timeToolKey)
ttdata.position_pixel() # position of edge, in pixels
ttdata.amplitude()      # amplitude of edge, in pixels
ttdata.nxt_amplitude()  # amplitude of second-most-significant edge, in pixels
ttdata.position_fwhm()  # FullWidthHalfMax of the differentiated signal (corresponds to slope of edge) in pixels
ttdata.position_time()  # position of the most significant edge (see note below)

Note that the position_time() results depend on have appropriate calibration constants deployed to the TimeTool configuration in a variable like this (in this case done as an argument to psana.setOptions()):

'TimeTool.Analyze.calib_poly':'0 1 0'
 

The three variables are coefficients of a quadratic polynomial that convert pixel number into time.  These constants are typically determined by the hutch scientists.

 

Controlling Laser/Beam Logic

TimeTool.Analyze is often used on experiments where both the laser and beam fire at different times. TimeTool.Analyze does the following based on what it determines about the laser and beam:

  • laser on, beam off: builds a reference/background based on just the laser. The user may configure TimeTool.Analyze to load the reference from a file, in case no "beam off" data was acquired in the run.

  • laser on, beam on: when it has a reference of just the laser background, computes its results and puts them in the Event.
  • laser off: nothing

The laser on/off beam on/off logic is typically determined based on evr codes, and looking at energy in the beam monitors (ipmb data)  - which evr codes and ipmb's is configurable. However for some experiments, users need to override this logic and make their own decision. Starting in ana-0.13.17, this can be done as follows

  • configure TimeTool.Analyze to get laser and/or beam logic from strings in the Event
  • Write a Psana Module that puts "on" or "off" in the Event for the laser and/or beam based on your own logic
  • Load this Psana Module before loading TimeTool.Analyze

The parameters to tell TimeTool.Analyze to get laser/beam logic are "beam_on_off" and "laser_on_off". For example, if you do

# in a config file
[TimeTool.Analyze]
beam_on_off_key=beam_on_off
laser_on_off_key=laser_on_off

then TimeTool.Analyze will bypass it's own logic for determining if the laser as well as the beam is on or off, and get if from variables in the event that are strings, with the keys "beam_on_off" and "laser_on_off" (you can set those to whatever you like, and you need not specify both if you only want to control the beam logic, or laser logic, respectively).

Next one needs to write a Psana Module (not a standard Python script) that adds these variables into the event. A good reference for Psana Modules is psana - User Manual. Note - this link is different then the links that discuss writing Python scripts, such as  psana - Python Script Analysis Manual. The Psana module will have to add the variables for every event - once you specify a value for beam_on_off_key, or laser_on_off_key, those keys need to be present for all events. An example Psana Module written in Python might be

class MyMod(object):
    def event(self, evt, env):
        evt.put("on","beam_on_off")
        evt.put("off","laser_on_off")

Now, assuming this Psana Module called MyMod was in a Package called MyPkg (so it resied in a file in your test release, MyPkg/src/MyMod.py) if one were to set the psana modules option like so

[psana]
modules=MyPkg.MyMod,TimeTool.Analyze

then TimeTool.Analyze would treat the beam as on and the laser as off for every event.

Plotting and Details about Analyze

A general feature of psana is to control the level of output that differnent modules cary out. To see the trace and debug messages of TimeTool.Analyze, set the following environment variable before running your code

MSGLOGCONFIG=TimeTool.Analyze=debug

Starting with ana-0.14.4, you can also set the following configuration options:

[TimeTool.Analyze]
eventdump=True

This adds a number of intermediate calculations into the event store. The TimeTool package includes a Python Psana module that will look for these results and plot them. To use this module, include it in the Psana module chain after TimeTool.Analyze, i.e:

[psana]
modules = TimeTool.Analyze TimeTool.PlotAnalyze

and then configure PlotAnalyze as follows:

[TimeTool.PlotAnalyze]
tt_get_key = TSS_OPAL # or whatever the input key is for TimeTool.Analyze to find the camera frame
tt_put_key = TTANA # or whatever the output key, put_key is for TimeTool.Analyze
fignumber = 11 # starting matplotlib figure number, change to not interfere with other figure numbers you may be using
pause = 1 # set to 0 to go through events without pause, otherwise module stops each time the laser is on

Examples

The TimeTool package contains two examples (see  /reg/g/psdm/sw/releases/ana-current/TimeTool/examples/)

EVR BYKICK signals no beam

A common way to run the TimeTool is to have the laser always on, but the beam is not present when the evr bykick code is present. The TimeTool.Analyze module will build up a reference during the BYKICK shots, and attempt to compute results for all other shots. There are a few reasons why it may fail and return no results - usually related to a very poor signal during that event. To run this example, do

python TimeTool/examples/plot_analyze.py -h

You see that it is a script with a help message. If you then run it as

python TimeTool/examples/plot_analyze.py -d sxri0214 -r 158 -n 100

The -n option is a testing option that limits the number of events to the first 100. The script loads TimeTool.Analyze to get results, but it also configures TimeTool.Analyze to put extra information in the event store. This is done with the generated a log plot of the image and plots the time tool pixel position result over the plot.

Manage References using Calibration Manager

Some experiments use certain runs to form the reference for the TimeTool. They take a run where the laser is ON, but the beam is blocked. TimeTool.Analyze contains two parameters, ref_load and ref_store. These could be used to have TimeTool save a reference from such a run, and then load it when processing another run. Another option is to form a pedestal file for the camera in question using calibman. Once this pedestal file has been deployed, you can set the option

        use_calib_db_ref=1

In the TimeTool.Analyze configuration. It will then detect which run the events are coming from and attempt to load the pedestal file from the calibration manager. Look at

python TimeTool/examples/refCalibExample.py -h

for details.

Module Check

a module that retrieves results from the event for either the above module or from data recorded online.

Module Setup

a module that calculates the reference autocorrelation function from events without FEL for use in the digital filter construction.

  • No labels