The timetool camera measures the time difference between laser and FEL in one of two methods:
- spatial encoding, where the X-rays change the reflectivity of a material and the laser probesthat change by the incident angle of its wavefront; or
- spectral encoding, where the X-rays change the transmission of a material and the chirped laser probes it by a change in the spectral components of the transmitted laser.
TimeTool results can be computed by the DAQ while data is being recorded and written directly into the .xtc files, or after data has been recorded. In AMO/XPP the time tool analysis is done by the DAQ while data is being recorded. Results can be accessed using the following epics-variables names: TTSPEC:AMPL , TTSPEC:AMPLNXT, TTSPEC:FLTPOS, TTSPEC:FLTPOSFWHM, TTSPEC:FLTPOS_PS, TTSPEC:REFAMPL or by getting the appropriate Psana.TimeTool.DataV* object from the event (either V1 or V2) from the opal camera source.
This document covers how to run the TimeTool after data has been recorded. Starting with ana-0.16.9, a Python wrapper to the TimeTool is provided which is the preferred way to run the TimeTool algorithms. Previous to ana-0.16.9 one would use Psana modules. One would loadi the C++ TimeTool.Analyze module via a psana config file.
Note: specific examples for the time tool can be found in /reg/g/psdm/sw/releases/ana-current/TimeTool/examples/.
Python Interface
In the example directory mentioned above, see:
- pyxface_evr_bykick.py
- pyxface_calib_db_ref.py
The python interface contains extensive documentation in its docstrings. To access this, from a ipython session, do
import TimeTool TimeTool.PyAnalyze? TimeTool.AnalyzeOptions?
Further documentation can be found on the PyAnalyze functions.
The docstrings will contain the most current documentation. For convenience, we include this documentation here as well:
PyAnalyze
Python interface to the TimeTool.Analyze C++ Psana module to allow conditional execution.
Basic Usage
There are several steps to using the module demonstrated in this example:
ttOptions = TimeTool.AnalyzeOptions(get_key='TSS_OPAL',
eventcode_nobeam = 162)
ttAnalyze = TimeTool.PyAnalyze(ttOptions)
ds = psana.DataSource(self.datasource, module=ttAnalyze)
for evt in ds.events():
ttResults = ttAnalyze.process(evt)
The steps are
Create an instance of the AnalyzeOptions class. See AnalyzeOptions docstring for detailed documentation on options.
Construct an instance of PyAnalyze (called ttAnalyze above) by passing this options object (called ttOptions above).
construct your psana DataSource by passing the PyAnalyze instance through to the module argument.
call the ttAnalyze.process() method on each event you want to process.
Parallel Processing
When doing parallel processing and distributing events among different ranks, each rank typically processes a fraction of all the events. Howevever it is important that each rank also process all events that include reference shots for the TimeTool. One can check if a shot is a reference shot or not with the isRefShot(evt) function. For example, in the above, suppose the variables numberOfRanks and rank are defined so that we can implement a round robin strategy to distribute event processing. One could implement this, while making sure all ranks process reference shots, as follows:
for idx, evt in enumerate(ds.events()):
if ttAnalyze.isRefShot(evt):
ttAnalyze.process(evt)
if idx % numberOfRanks != rank:
continue
ttResults = ttAnalyze.process(evt)
Note that it is Ok to call the PyAnalyze.process(evt) function more than once on the same event. The PyAnalyze class caches the results of the first call so as to not call the underlying C++ TimeTool.Analyze module twice.
Controlling Beam Logic
This is an unsual use case. It happens when users are using EPICSs to move things around to drop the beam, rather than using an event code. In this case the user will want to look at the EPICS variables and instruct the TimeTool of when the beam is off so that it can build up its reference. To do this, set controlLogic=True in AnalyzeOptions and use the controlLogic(beamOn, laserOn) function. When controlLogic is true, you must call this function for each event you subsequently call process on (if you forget, TimeTool will stop with an error).
AnalyzeOptions
Specify configuration options for the TimeTool Analyze module.
There are many options for TimeTool.Analyze, most can be left at their default values. First we document options users may want to change, then more specialized options.
Note that all options are either a Python str, or Python int, or float. Even though some options represent a list of numbers, the argument must be formatted as a string with only whitespace separating numbers (no commas, etc).
Common Options
Option default data type / explanation
------ ------- -----------
get_key 'TSS_OPAL' str, this is the psana source for the TimeTool
opal camera. The default, 'TSS_OPAL', is a common
DAQ alias for this source
eventcode_nobeam 0 int, for BYKICK experiments, where an evr code
specifies when the beam is NOT present, specify that
EVR code here. If the beam is always present, keep
the default value of 0. Note - there are specialized
options to control laser on/off beam on/off logic
below.
ref_avg_fraction 0.05 float, weight assigned to next reference shot in
rolling average. 1.0 means replace reference with
next shot (no rolling average). 0.05 means next
shot is 5% of average and all previous is 95%.
sig_roi_y '425 725' str, signal roi in y, or the rows of the Opal camera
that the user may want to adjust based on where signal is.
sig_roi_x '0 1023' str, signal roi in x or columns, default is all.
calib_poly '0 1 0' str, TimeTool.Analyze returns results as both a pixel
location on the Detector, as well as a conversion
to femtoseconds by applying this quadratic polynomial. Typically a
special calibration run is performed to compute the mapping from
position on the detector to femtoseconds and the results of this
analysis are passed through this parameter. If calib_poly is 'a b c'
Then femtosecond_result = a + b*x + c*x^2 wher x is the detector position.
** Uncommon Options**
Option default data type / explanation
------ ------- -----------
projectX True bool, if true, project down to the X axis of the opal.
eventcode_skip 0 int, EVR code for events that should be skipped from
TimeTool processing.
ipm_get_key '' str, in addition to the evr code above, the timetool
will look at the threshold on a specified ipmb to decide
if the beam is present. Default is to not look at a
ipmb. Here one can specify the psana source for the
desired ipimb.
ipm_beam_threshold float, threshold for determining if beam present from
an ipm.
weights '0.00940119 ...' str, this defaults to a long string of the
weights that the TimeTool uses when performing signal
processing on the normalized, reference divided signal,
to turn a sharp drop into a peak. It is unusual for
users to modify this string. The full string can be
found in the code.
weights_file '' str, the weights can be put into a file as well
use_calib_db_ref False bool, get the initial reference from the
calibration database. This reference can be deployed
using calibman by making a pedestal for the appropriate
opal. Example use case is creating references not by
dropping the beam from shots, but rather making
reference runs (dropped shots tend to give better results).
ref_load '' str, filename to load reference file from.
ref_store '' str, filename to store reference into.
controlLogic False, bool, to bypass the normal mechanism of letting
the timetool identify when the beam or laser is on,
the user can control this. If this is set, then for
every event, one must call controlLogic before calling
process with the TimeTool.Analyze python class.
proj_cut -(1<<31), int, after projecting of the opal camera to create the
signal, one can require that at least one value in the signal
be greater than this parameter in order to continue processing.
Default is int_min, which means one always processes when the
laser is on (and a reference is available).
sb_roi_x '', str give something like 1 10 to do sideband analysis and specify
the sideband ROI. This feature, while it exists in the code, is not
maintained. Experts interested in this feature may need to contact
their POC for support.
sb_roi_y '', str, see sb_roi_x, this is for the y region (rows)
sb_avg_fraction 0.05, float, see sb_roi_x and sig_avg_fraction for information on
rolling averages.
analyze_event -1, int, special options for analyzing the first few events with respect
to a reference - feature not maintained.
dump 0, int, setting this option is not reccommended. It instructs TimeTool
to use the psana root based historgramming method - however the root
files interfere with MPI based analysis. See the eventdump option below
to access intermediate stages of TimeTool processing.
eventdump False, bool, setting this option to True will cause the underlying C++
Psana TimeTool.Analyze module to return extra data that the Psana PlotAnalyze
module can use (or users can get directly) but presently, the Python wrapper
does not expose this, or setup the plotter in a convenient way. However One can
get the ndarrays directly from the event following a call to PyAnalyze.Process().
psanaName TimeTool.Analyze, str, the logging name passed to this instance of the
underlying C++ Psana Module called TimeTool.Analyze. There should be no
reason to modify this - unless for some reason you want to configure two
separate instances of the Psana Module.
put_key TTANA, str, should be little reason to modify this. This is the key used to get
results back from the C++ TimeTool.Analyze module.
beam_on_off_key ControlLogicBeam, str, there should be no reason to modify this.
if controlLogic is true, this is the internal key string used to
communicate beam on/off with the TimeTool.Analyze module.
laser_on_off_key ControlLogicLaser, str, as above, should be no reason to modify this.
Users migrating older code will need to discard psana config files, or remove the TimeTool.Analyze configuration from the psana config file AS WELL AS THE TimeTool.Analyze C++ Psana Module from the list of psana modules in the config file. If mixing old style Psana modules with PyAnalyze, be advised that PyAnalyze will run after any modules listed in the psana config file (so the TimeTool results will not be available to Psana modules listed in the config file). It should be easy to move Python Psana modules into the new style (C++ modules are not easy to move).
C++ Psana Module Interface
Next we disuss the older, C++ Psana module interface to the TimeTool. The package includes sample configuration files that describe all the options. From a psana release directory, users are encouraged to add the TimeTool package to obtain the latest source. For instance:
newrel ana-current myrel cd myrel kinit # get ticket to check out package from svn repository addpkg TimeTool sit_setup scons # now examine the files in TimeTool/data: sxr_timetool.cfg sxr_timetool_setup.cfg timetool_setup.py xpp_timetool.cfg xpptut.cfg
timetool_setup.py is a python script to calculate the digital filter weights.
Module Analyze
A module that analyzes the camera image by projecting a region of interest onto an axis and dividing by a reference projection acquired without the FEL. The resulting projection is processed by a digital filter which yields a peak at the location of the change in reflectivity/transmission. The resulting parameters are written into the psana event. The type of the parameter depends on the release. Starting with ana-0.13.10, a TimeTool::DataV2 object in put in the event store. ana-0.13.3 put a TimeTool::DataV1 object in the event store. In ana-0.14.4 and later, this is how one gets the data, a TimeTool::DataV2 object. Older releases would also add the output as doubles or ndarrays, but this is no longer the case with ana-0.14.4 and later.
Accessing Results from Analyze (ana-0.14.4 and later)
ttData = evt.get(TimeTool.DataV2, self.timeToolKey) ttdata.position_pixel() # position of edge, in pixels ttdata.amplitude() # amplitude of edge, in pixels ttdata.nxt_amplitude() # amplitude of second-most-significant edge, in pixels ttdata.position_fwhm() # FullWidthHalfMax of the differentiated signal (corresponds to slope of edge) in pixels ttdata.position_time() # position of the most significant edge (see note below)
Note that the position_time() results depend on have appropriate calibration constants deployed to the TimeTool configuration in a variable like this (in this case done as an argument to psana.setOptions()):
'TimeTool.Analyze.calib_poly':'0 1 0'
The three variables are coefficients of a quadratic polynomial that convert pixel number into time. These constants are typically determined by the hutch scientists.
Controlling Laser/Beam Logic
TimeTool.Analyze is often used on experiments where both the laser and beam fire at different times. TimeTool.Analyze does the following based on what it determines about the laser and beam:
laser on, beam off: builds a reference/background based on just the laser. The user may configure TimeTool.Analyze to load the reference from a file, in case no "beam off" data was acquired in the run.
- laser on, beam on: when it has a reference of just the laser background, computes its results and puts them in the Event.
- laser off: nothing
The laser on/off beam on/off logic is typically determined based on evr codes, and looking at energy in the beam monitors (ipmb data) - which evr codes and ipmb's is configurable. However for some experiments, users need to override this logic and make their own decision. Starting in ana-0.13.17, this can be done as follows
- configure TimeTool.Analyze to get laser and/or beam logic from strings in the Event
- Write a Psana Module that puts "on" or "off" in the Event for the laser and/or beam based on your own logic
- Load this Psana Module before loading TimeTool.Analyze
The parameters to tell TimeTool.Analyze to get laser/beam logic are "beam_on_off" and "laser_on_off". For example, if you do
# in a config file [TimeTool.Analyze] beam_on_off_key=beam_on_off laser_on_off_key=laser_on_off
then TimeTool.Analyze will bypass it's own logic for determining if the laser as well as the beam is on or off, and get if from variables in the event that are strings, with the keys "beam_on_off" and "laser_on_off" (you can set those to whatever you like, and you need not specify both if you only want to control the beam logic, or laser logic, respectively).
Next one needs to write a Psana Module (not a standard Python script) that adds these variables into the event. A good reference for Psana Modules is psana - User Manual. Note - this link is different then the links that discuss writing Python scripts, such as psana - Python Script Analysis Manual. The Psana module will have to add the variables for every event - once you specify a value for beam_on_off_key, or laser_on_off_key, those keys need to be present for all events. An example Psana Module written in Python might be
class MyMod(object):
def event(self, evt, env):
evt.put("on","beam_on_off")
evt.put("off","laser_on_off")
Now, assuming this Psana Module called MyMod was in a Package called MyPkg (so it resied in a file in your test release, MyPkg/src/MyMod.py) if one were to set the psana modules option like so
[psana] modules=MyPkg.MyMod,TimeTool.Analyze
then TimeTool.Analyze would treat the beam as on and the laser as off for every event.
Plotting and Details about Analyze
A general feature of psana is to control the level of output that differnent modules cary out. To see the trace and debug messages of TimeTool.Analyze, set the following environment variable before running your code
MSGLOGCONFIG=TimeTool.Analyze=debug
Starting with ana-0.14.4, you can also set the following configuration options:
[TimeTool.Analyze]
eventdump=True
This adds a number of intermediate calculations into the event store. The TimeTool package includes a Python Psana module that will look for these results and plot them. To use this module, include it in the Psana module chain after TimeTool.Analyze, i.e:
[psana]
modules = TimeTool.Analyze TimeTool.PlotAnalyze
and then configure PlotAnalyze as follows:
[TimeTool.PlotAnalyze]
tt_get_key = TSS_OPAL # or whatever the input key is for TimeTool.Analyze to find the camera frame
tt_put_key = TTANA # or whatever the output key, put_key is for TimeTool.Analyze
fignumber = 11 # starting matplotlib figure number, change to not interfere with other figure numbers you may be using
pause = 1 # set to 0 to go through events without pause, otherwise module stops each time the laser is on
Examples
The TimeTool package contains two examples (see /reg/g/psdm/sw/releases/ana-current/TimeTool/examples/)
EVR BYKICK signals no beam
A common way to run the TimeTool is to have the laser always on, but the beam is not present when the evr bykick code is present. The TimeTool.Analyze module will build up a reference during the BYKICK shots, and attempt to compute results for all other shots. There are a few reasons why it may fail and return no results - usually related to a very poor signal during that event. To run this example, do
python TimeTool/examples/plot_analyze.py -h
You see that it is a script with a help message. If you then run it as
python TimeTool/examples/plot_analyze.py -d sxri0214 -r 158 -n 100
The -n option is a testing option that limits the number of events to the first 100. The script loads TimeTool.Analyze to get results, but it also configures TimeTool.Analyze to put extra information in the event store. This is done with the generated a log plot of the image and plots the time tool pixel position result over the plot.
Manage References using Calibration Manager
Some experiments use certain runs to form the reference for the TimeTool. They take a run where the laser is ON, but the beam is blocked. TimeTool.Analyze contains two parameters, ref_load and ref_store. These could be used to have TimeTool save a reference from such a run, and then load it when processing another run. Another option is to form a pedestal file for the camera in question using calibman. Once this pedestal file has been deployed, you can set the option
use_calib_db_ref=1
In the TimeTool.Analyze configuration. It will then detect which run the events are coming from and attempt to load the pedestal file from the calibration manager. Look at
python TimeTool/examples/refCalibExample.py -h
for details.
Module Check
a module that retrieves results from the event for either the above module or from data recorded online.
Module Setup
a module that calculates the reference autocorrelation function from events without FEL for use in the digital filter construction.
Overview
Content Tools