Page History

Introduction

XTCAV is a detector that is used to determine the laser-power vs. time of each LCLS shot.  Alvaro Sanchez-Gonzalez has written psana-python code to do the rather complex analysis of images from the XTCAV camera to determine these quantities.  Some detailed documentation from Tim Maxwell on this device is Here.

These scripts use some XTCAV data that was made public so they should be runnable by all users.  The scripts can be found in /reg/g/psdm/tutorials/examplePython/xtcav/ in the files xtcavDark.py, xtcavLasingOff.py, xtcavLasingOn.py.  They analyze a minimal number of events to make them run fairly quickly.

Analysis Setup

Two things must be done before XTCAV analysis will function: a "dark run" must be analyzed to get the pedestal values for cameras, and a "no lasing" run must be analyzed to generate sets of "no lasing" images (the latter is quite a complex process).  Note that for demonstration these first two scripts write constants to a "local" calibration directory called "calib".  For a real-experiment you won't need these lines because you will have permission to write to your official experiment calibration-constants directory.

An  An example of a dark-run analysis is in /reg/g/psdm/tutorials/examplePython/xtcav/xtcavDark.py:

#!/usr/bin/env python

# these two lines for example purposes only, to allow user to write
# calibration information to local directory called "calib".
# should be deleted for real analysis.
import psana
psana.setOption('psana.calib-dir','calib')

from xtcav.GenerateDarkBackground import *
GDB=GenerateDarkBackground();
GDB.experiment='xpptut15'
GDB.runs='102300'
GDB.maxshots=10  #small number for this example, people often use 1000 shots for this.
GDB.SetValidityRange(101300,125302) # delete second run number argument to have the validity range be open-ended 
("end")
GDB.Generate();

An example of a non-lasing run is in /reg/g/psdm/tutorials/examplePython/xtcav/xtcavLasingOff.py:

#!/usr/bin/env python

# these two lines for example purposes only, to allow user to write
# calibration information to local directory called "calib".
# should be deleted for real analysis.
import psana
psana.setOption('psana.calib-dir','calib')
 
from xtcav.GenerateLasingOffReference import *
GLOC=GenerateLasingOffReference();
GLOC.experiment='xpptut15'
GLOC.runs='101301'
GLOC.maxshots=2  #small number for this example, people often use 1400 shots for this.
GLOC.nb=21
GLOC.islandsplitmethod = 'scipyLabel'       # see confluence documentation for how to set this parameter
GLOC.groupsize=540             # see confluence documentation for how to set this parameter
GLOC.SetValidityRange(101300,125302) # delete second run number argument to have the validity range be open-ended
 ("end")
GLOC.Generate();

This script can be quite slow.  It can be easily run in parallel by submitting a parallel MPI job to the batch system as described here, however you should change the above script to increase the "maxshots" parameter (so that each core has at least 1 shot to process).  People often use ~1400 shots for this.  Note that if you analyze multiple runs, you must call GLOC=GenerateLasingOffReference() once per run to get the correct constants.

Once the dark/lasing-off analysis has been completed, users can analyze the lasing-on events using a standard psana-python script similar to the one below.

Example Analysis Script

This script assumes that dark/lasing-off data has been analyzed (see above).   This Unlike the previous two scripts it reads dark/lasing-off constants from the official calibration-directory.  This script can be found in /reg/g/psdm/tutorials/examplePython/xtcav/xtcavLasingOn.py:

importfrom psana

# this line is for example purposes only, to allow user to read
# calibration information from local directory called "calib".
# should be deleted for real analysis.  It also sets a flag
# ("allow-corrupt-epics") that allows the old example data to be analyzed.
psana.setOptions({'psana.calib-dir':'calib',
                  'psana.allow-corrupt-epics':True})

from xtcav.ShotToShotCharacterization import *
experiment='xpptut15'  #Experiment label
runs='124'             #Runs
#Loading the dataset from the "dark" run, this way of working should be compatible with both xtc and hdf5 f
iles
dataSource=psana.DataSource("exp=%s:run=%s:idx" % (experiment,runs))
#XTCAV Retrieval (setting the data source is useful to get information such as experiment name)
XTCAVRetrieval=ShotToShotCharacterization();
XTCAVRetrieval.SetEnv(dataSource.env())
for r,run in enumerate(dataSource.runs()):
    times = run.times()
    for t in times:
        evt = run.event(t)
        if not XTCAVRetrieval.SetCurrentEvent(evt):
            continue
        time,power,ok=XTCAVRetrieval.XRayPower()  
        agreement,ok=XTCAVRetrieval.ReconstructionAgreement()import *
from xtcav.ShotToShotCharacterization import *
ds=DataSource('exp=xpptut15:run=302:smd')
XTCAVRetrieval=ShotToShotCharacterization();
XTCAVRetrieval.SetEnv(ds.env())
import matplotlib.pyplot as plt
ngood = 0
for evt in ds.events():
    if not XTCAVRetrieval.SetCurrentEvent(evt): continue
    time,power,ok=XTCAVRetrieval.XRayPower()  
    if not ok: continue
    agreement,ok=XTCAVRetrieval.ReconstructionAgreement()
    if not ok: continue
    ngood += 1
    # time and power are lists, with each entry corresponding to
    # a bunch number.  The number of bunches is set by the GLOC.nb
    # parameter in the lasing-off analysis.  In general, one should
    # also cut on the "agreement" value, which measures the agreement
    # between the first and second moment analysis (larger is better).
    plt.plot(time[0],power[0])
    plt.xlabel('Time (fs)')
    plt.ylabel('Lasing Power (GW)')
    plt.title('Agreement %4.2f'%agreement)
    plt.show()
    if ngood>1: break

This script runs on one core, but it can be MPI-parallelized in the standard psana-python manner described here.

One caveat: this data shows "horns" at the beginning and the end of the bunch which confuse the algorithm (the accelerator often behaves in this manner).  Only the middle peak in the plotted spectrum is the relevant lasing peak.

Some tips for lasing-on analysis:

• Look at the distribution of the "agreement" parameter that is returned by the ReconstructionAgreement() method.  This value represents the "dot product" of the power-spectrum from the first-moment-analysis of the XTCAV image with the power-spectrum from the second-moment-analysis of the XTCAV image.   Only believe the data where the agreement is good (in the past >0.5 has been useful for some analyses)
• Ignore shots where the X-ray intensity is low, but cutting on the FEEGasDetector value to select stronger shots

Examining The "Ingredients" of the XTCAV Analysis

This is a utility that can help users understand why XTCAV results look the way they do.   Run it like this:

 xtcavDisp exp=xpptut15:run=302

It produces plots that look like this:

Image Added

These plots show the following quantities for both lasing-on shot that is being analyzed and the lasing-off shot that it is being compared to (which is selected as the one having the closest current-profile):

• Current
• Energy computed using the first-moment ("Delta") method
• Energy computed using the second-moment ("Sigma") method
• Power

Close the existing plot window to show the plots for the next event.  If you want to modify the 105-line "xtcavDisp" python script (e.g. to change which events are displayed) you can find it in  /reg/g/psdm/sw/releases/ana-current/xtcav/src/xtcavDisp.

How Often to Take a Lasing Off Run

(courtesy of Tim Maxwell)

That's a very good question. For very stable accelerator conditions, you might not don't really need to update but every hour or two. ButHowever, for example with that AMO experiment it drifted with some special configurations it can drift measurably over as short as twenty minutes as the beam was a trick setup and some feedbacks needed disabling. That's typically for cases where we have to leave feedbacks open (two bucket and some twin bunch configurations).

There's not really a hard, fast rule here. When necessary or when time allows has been Definitely when necessary due to known changes in beam conditions and otherwise as often as time allows is the practical answer so far.

Lasing-off Analysis Parameters

(courtesy of Alvaro Sanchez-Gonzalez)

-NB: Number of bunches. This one is obvious. Just : typically 1, but sometimes 2 for some LCLS experiments.  Just as a note for the future, if at some point you want to do two bunch partial reconstruction (i.e. without lasing off reference), you will need to specify explicitly the number of bunches in an undocumented way (i.e. changing an attribute without a get/set method. Example:

...

-groupsize: 5 is a good number. Increasing this number will make better references, but they may also make them less accurate, as you would be averaging a larger number of profiles. If you decide to increase this number by a factor, then I would also increase n by the same factor, so in the end the total number of reference is the same. (see also thoughts from Tim Maxwell below).

-islandsplitmethod: (default value: 'scipylabel') Several image processing algorithms have been created to separate bunches that appear on the same image. Each algorithm is advantageous in certain types of conditions and thus it might be worth trying all of them and see what achieves the best results.  The defaults parameter is 'scipylabel'. If one does not set islandsplitmethod to anything, the parameter will be set to 'scipylabel'. If one would like to change from the default , one can use GLOC.islandsplitmethod = 'contourLabel'.  'scipylabel' calls the scipy label function to label contiguous regions. 'contourlabel See here for some algorithm details.

• 'scipyLabel' calls the scipy label function to label contiguous regions.  this is fast, but requires a region of no signal (after de-noising) between the regions.
• 'autothreshold' tries to separate the islands by automatically finding thresholds that separate them.  this is the recommended mode for multi-bunch operation.
• 'contourLabel' tries to adjust a threshold until two large groups are found, grouping together pixels using an opencv contour method.  there are two parameters ('ratio1/ratio2') that are settable for the contour method

...

• that determine

-roexpand and roiwaistthres: Call the method 'ProcessedXTCAVImage'. If the image does not look clipped, the parameters are fine.

...

Thoughts from Tim Maxwell on Number of Groups

Two casesFor various beam conditions (note that Tim describes number-of-groups, but our settable parameter is "groupsize"=number-of-events/number-of-groups):

• normal sase (not many beam-related fluctuations): take sqrt of number shots and use that many groups (beat down background noise with lots of averaging).   Tim suggests perhaps a maximum of 100 groups.
• two-bunch slotted foil (many beam-related of fluctuations): restrict it to 5-10 images per group, make as many groups as possible using typically 30 seconds to a minute of data at 60Hz.

120Hz Operation Issues

(Thoughts from Tim Maxwell on 2/2/2015, discouraging this mode of operation)

...

• split undulator, single bunch, head and tail lase in separate sections of the split undulator: Tim expects this to be somewhat more chaotic than SASE, but not as chaotic as slotted-foil, so some number of groups in between.

• seeded beam: For any results, I would use the SASE settings. However, note that reconstruction for seeding is a little ambiguous. The beam first seeds, so is partly spoiled before seeding itself in the second stage. Therefore it isn't clear exactly with one lasing off reference which part lased more for the seeded part. It is similar to the case when one bunch is used to make two pulses with the split undulator. However, if seeding amplification was strong and intense, this may be enough.

Detector Resolution

(From Tim Maxwell)

Time resolution is around 1.1 fs RMS for soft x-rays, 2.5 fs fwhm (in quadrature, of course). So actually pulse length is probably 4.3 - 9.7 fs FWHM.

...