Page History
...
Here we cover topics specific to the offline TimeTool module. The TimeTool results can be obtained in one of two ways depending on the experimental setup. The first is directly during data acquisition. For data recorded prior to Oct 13, 2014, the timetool results were always recorded as EPICS PV's. After Oct 13, 2014, they are still recorded as EPICS PV's, but also recorded in their own data type: TimeTool::DataV1. The latter is the preferred way to access the data, EPICS PV's are provided for backward compatibility. The second is during offline analysis using the psana module TimeTool.Analyze. Similarly, this module puts a TimeTool::DataV1 object in the event store. Previously, it put a collection of floats, or ndarrays in the event store (for backward compatibility, it still puts the floats in the event store). Documentation on the psana TimeTool modules can be found in the psana - Module Catalog.
...
The case
...
studies below reference code in the TimeToolExamples package from the svn users repository at SLAC. The simplest way to work through these case studies is to add this package to a test release and build/run the code - potentially modifiying it to run on the experimental data that you are have access to. If you are not at SLAC, downloads of the code are provides below.
Assuming you are at SLAC, start by making a test release:
newrel ana-current myrel
cd myrel
addpkg -u TimeToolExamples V00-00-05
scons
If you have any trouble with the addpkg -u command, you may need to do
kinitto get a kerberos ticket to access the users svn repository. This tag (V00-00-05) has been tested with ana-0.13.4 and should work with later releases. This is the release when the TimeTool.Analyze package started to use the TimeTool::DataV1 object.
The package includes scripts in the TimeToolExamples/app subdirectory. It is intended that users will modify these scripts for their purposes.
*IMPORTANT* after modifying a script in the app sub-directory of a package, you *MUST* run scons. This installs a copy of the script (with a modification to the #! line) in the myrel/arch directory. The file in myrel/arch is what is actually run.
SXR case study: EVR BYKICK signals no beam
This is the most straightforward way to run the TimeTool and, I believe, the expected way it is run in SXR. The laser is always on, but the beam is not present when the evr bykick code is present. The TimeTool.Analyze module will build up a reference during the BYKICK shots, and compute results for all other shots. An example script which executes this scenario can be found in
TimeToolExamples/app/tt_sxr_bykickAfter installing the package in a test release as described above and running scons, do
tt_sxr_bykick -hto get help from the script. The script was developed/tested against this data:
tt_sxr_bykick -e sxri0214 -r 158 -n 100The -n option is a testing option that limits the number of events to the first 100. The script generated a log plot of the image and plots the time tool pixel position result over the plot.
One can read the code for the script here:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
#!@PYTHON@
__doc__='''
Runs the TimeTool module on a given run and experiment. Uses BYKICK evr code to
identify reference shots (nobeam, but laser present). Plots results.
'''
import argparse
import sys
import psana
import numpy as np
import matplotlib.pyplot as plt
plt.ion()
###########################
# CONFIGURAITON:
#
# psana options for main() function below. This
# has the same information as a psana config file, however we include it in the
# script so we can be sure to use the same values in our Python code. This is
# basically a transcription of the TimeTool/data/sxr_timetool.cfg file into the
# Python dictionary that we can use to set psana options from a script as opposed to
# a config file.
EVR_BYKICK = 162
psanaOptions = {
########## psana configuration #################
'psana.modules':'TimeTool.Analyze',
########## TimeTool.Analyze configuration #######
# Key for fetching timetool camera image
'TimeTool.Analyze.get_key':'TSS_OPAL',
# Results are written to <put_key>
'TimeTool.Analyze.put_key':'TTANA',
# Indicate absence of beam for updating reference
'TimeTool.Analyze.eventcode_nobeam':EVR_BYKICK,
# Indicate events to skip (no laser, for example)
'TimeTool.Analyze.eventcode_skip':0,
# Polynomial coefficients for position_time calculation
'TimeTool.Analyze.calib_poly':'0 1 0',
# Project onto X axis?
'TimeTool.Analyze.projectX':True,
# Minimum required bin value of projected data
'TimeTool.Analyze.proj_cut':0,
# ROI (x) for signal
'TimeTool.Analyze.sig_roi_x':'0 1023',
# ROI (y) for signal
'TimeTool.Analyze.sig_roi_y':'408 920',
# ROI (x) for sideband
'TimeTool.Analyze.sb_roi_x':'' ,
# ROI (y) for sideband
'TimeTool.Analyze.sb_roi_y':'',
# Rolling average convergence factor (1/Nevents)
'TimeTool.Analyze.sb_avg_fraction':0.05,
# Rolling average convergence factor (1/Nevents)
'TimeTool.Analyze.ref_avg_fraction':1.0,
# Read weights from a text file
'TimeTool.Analyze.weights_file':'',
# Indicate presence of beam from IpmFexV1::sum() [monochromator]
'TimeTool.Analyze.ipm_get_key':'',
# 'TimeTool.Analyze.ipm_beam_threshold':'',
# Load initial reference from file
'TimeTool.Analyze.ref_load':'',
# Save final reference to file
'TimeTool.Analyze.ref_store':'timetool.ref',
# Generate histograms for initial events, dumped to root file
'TimeTool.Analyze.dump':20,
# Filter weights
'TimeTool.Analyze.weights':'0.00940119 -0.00359135 -0.01681714 -0.03046231 -0.04553042 -0.06090473 -0.07645332 -0.09188818 -0.10765874 -0.1158105 -0.10755824 -0.09916765 -0.09032289 -0.08058788 -0.0705904 -0.06022352 -0.05040479 -0.04144206 -0.03426838 -0.02688114 -0.0215419 -0.01685951 -0.01215143 -0.00853327 -0.00563934 -0.00109415 0.00262359 0.00584445 0.00910484 0.01416929 0.0184887 0.02284319 0.02976289 0.03677404 0.04431778 0.05415214 0.06436626 0.07429347 0.08364909 0.09269116 0.10163601 0.10940983 0.10899065 0.10079016 0.08416471 0.06855799 0.05286105 0.03735241 0.02294275 0.00853613',
}
def hatLogTransform(X, preLogOffset=5.0):
'''For plotting, we transform X to be
log(b+X)-log(b) for the non-negative values of X and
-(log(|-b+X|)-log(b)) for the negative values of X
where b is preLogOffset above. Setting b to 1.0 (min) emphasizs values between -1 and 1.
'''
assert preLogOffset >= 1.0
nonNegative = X >= 0.0
negative = np.logical_not(nonNegative)
X[nonNegative] = np.log(preLogOffset+X[nonNegative]) - np.log(preLogOffset)
X[negative] = -(np.log(np.abs(-preLogOffset+X[negative])) - np.log(preLogOffset))
def plotEvent(camera, bkg, position_pixel, sig_roi_x, sig_roi_y, preLogOffset, stopAfterPlot, plot_offset_x, plotTitle=''):
'''Creates and clears figure 1. plots hatLogTransform(camera-bkg, preLogOffset). Overlays with a
vertical line at position_pixel + plot_offset.
Optional parameters sig_roi_x, sig_roi_y are for bounding box (where position_pixel was
computed from) and interactive allows user to hit enter to keep plot on screen.
'''
opalArr = camera - bkg
hatLogTransform(opalArr, preLogOffset)
plt.figure(1)
plt.clf()
plt.imshow(opalArr)
plt.hold(True)
# plot bounding box where signal computed from
x1,x2 = sig_roi_x
y1,y2 = sig_roi_y
plt.plot([x1, x2, x2, x1, x1],
[y1, y1, y2, y2, y1], '-y', linewidth=2, label='roi')
# plot pixel poisition, add offset
plt.plot([position_pixel + plot_offset_x,
position_pixel + plot_offset_x],
[max(1, y1-10),
min(opalArr.shape[1], y2+10)], '-r', linewidth=1, label='timeTool position')
plt.xlim([0,opalArr.shape[1]])
plt.ylim([opalArr.shape[0],0])
plt.legend()
plt.title(plotTitle)
plt.draw()
if stopAfterPlot:
raw_input("hit enter to continue: ")
def main(experiment, run, numEvents, stopAfterPlot, plotBias, preLogOffset, psanaOptions):
global EVR_BYKICK
psana.setOptions(psanaOptions)
ref_avg_fraction = psanaOptions['TimeTool.Analyze.ref_avg_fraction']
sig_roi_x = map(int,psanaOptions['TimeTool.Analyze.sig_roi_x'].split())
sig_roi_y = map(int,psanaOptions['TimeTool.Analyze.sig_roi_y'].split())
put_key = psanaOptions['TimeTool.Analyze.put_key']
ds = psana.DataSource('exp=%s:run=%d'%(experiment, run))
evrSrc = psana.Source('DetInfo(NoDetector.0:Evr.0)')
opalSrc = psana.Source(psanaOptions['TimeTool.Analyze.get_key'])
lastByKick = 0
ourOwnRef = None
for idx,evt in enumerate(ds.events()):
evr = evt.get(psana.EvrData.DataV3, evrSrc)
if evr is None: continue
evrCodes = [fifoEvent.eventCode() for fifoEvent in evr.fifoEvents()]
evrCodes.sort()
frame = evt.get(psana.Camera.FrameV1, opalSrc)
if frame is None: continue
timetool = evt.get(psana.TimeTool.DataV1, put_key)
if EVR_BYKICK in evrCodes:
cameraFrame = np.array(frame.data16(),dtype=np.float)
if ourOwnRef is None:
ourOwnRef = cameraFrame
else:
ourOwnRef = ref_avg_fraction * cameraFrame + (1.0-ref_avg_fraction)*ourOwnRef
print "event %4d has BYKICK. events since last BYKICK=%4d. evr codes: %s" % (idx, idx-lastByKick, ','.join(map(str,evrCodes)))
lastByKick=idx
if timetool is not None:
ratio_ampl_nxt = timetool.amplitude()/(1e-30+timetool.nxt_amplitude())
print "event %4d has TIMETOOL. pos_pixel=%7.1f amplitude=%7.1e amplitude/nxt_amplitude=%.4f" % \
(idx, timetool.position_pixel(), timetool.amplitude(), ratio_ampl_nxt)
cameraFrame = np.array(frame.data16(),dtype=np.float)
plotTitle = 'scaled plot of Opal camera - ref. Both positive and negative values on log scale.'
plotEvent(cameraFrame, ourOwnRef, timetool.position_pixel(), sig_roi_x, sig_roi_y,
preLogOffset, stopAfterPlot, plotBias, plotTitle=plotTitle)
if numEvents > 0 and idx > numEvents:
break
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('-e', '--exp', type=str, help="experiment, for example sxri0214", required=True)
parser.add_argument('-r', '--run', type=int, help="run, for example 158", required=True)
parser.add_argument('-n', '--events', type=int, help="number of events to process, default (0) means all", default=0)
parser.add_argument('-b', '--plotbias', type=float, help="amount to add to pixel_position during plots, correct for fixed bias", default=26.0)
parser.add_argument('-s', '--stop', action='store_true', help="when plotting, stop after each plot and wait for user input", default=False)
parser.add_argument('-l', '--logoffset', type=float, help="for plotting, how to adjust log plot, log(A+x) where x is this parameter", default=1.0)
args = parser.parse_args()
main(experiment = args.exp,
run = args.run,
numEvents = args.events,
stopAfterPlot = args.stop,
plotBias = args.plotbias,
preLogOffset = args.logoffset,
psanaOptions = psanaOptions) |
SXR case study: reference/signal in different runs
Below we go over a script for generated offline TimeTool data for an sxr experiment. For this experiment, run 144 was done with the beam blocked, and run 150 with the beam on. The laser is always on in both runs.
This requires some configuration of the TimeTool. We run the TimeTool.Analyze module on run 144 - telling it that the beam is always off. Analyze builds up a reference. We further configure Analyze to save the reference. We also save a averaged background image for our own interactive plotting later.
We then run TimeTool.Analyze on run 150. We save the resulting timetool values to an hdf5 file using the psana xtc to hdf5 translator, see The XTC-to-HDF5 Translator for details.
Finally, to check the work, we process run 150 in index mode, see psana - Python Script Analysis Manual for details on indexing mode (random access) to psana events. We load the time tool values from the h5 file and plot them against the opal, after subtracting our background image.
The TimeToolExamples package discusses above includes two files for this case study
The example code is part of the package TimeToolExamples. The easiest way to access it is through the users svn repository at SLAC. If you do not have access to the repository, the code is available as attachments (see below). Assuming you are working on the SLAC system, to get started, do
newrel ana-current myrel
cd myrel
addpkg -u TimeToolExamples V00-00-04
scons
If you have any trouble with the addpkg -u command, you may need to do
kinitto get a kerberos ticket to access the users svn repository. This tag (V00-00-04) has been tested with ana-0.13.4 and should work with later releases. This is the release when the TimeTool.Analyze package started to use the TimeTool::DataV1 object. For previous analysis releases, get tag V00-00-01 which uses the put_ndarrays option to get the data.
The package contains two files. The first is the script that designates the sxr experiment data to run on. It is the file TimeToolExamples/app/tt_sxrd5814. You can immediately take a look at the code here: tt_sxrd5814. The second is a library of functions called by the driver script: TimeToolExamples/src/ttlib.py that you can download here: ttlib.py. Instead of the addpkg command above, one can do
...
to build the code. The run the script, from the release directory do
tt_sxrd5814
Note, users of other experiments will not be able to access the data from this experiment. You will want to modify TimeToolExamples/app/tt_sxrd5814 to read your own data.
*IMPORTANT* after modifying tt_sxrd5814, you *MUST* run scons. This installs a copy of the script (with a modification to the #! line) in the myrel/arch directory. The file in myrel/arch is what is actually runNote the warning above about running scons after modifying a script in the app subdirectory.
After running the script as is, it should produce the files
...
Overview
Content Tools