You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Introduction

 

XTCAV is a detector that is used to determine the laser-power vs. time of each LCLS shot. Some detailed documentation from Tim Maxwell on this device is Here. Alvaro Sanchez-Gonzalez authored the original psana-python code to do the rather complex analysis of images from the XTCAV camera to determine these quantities. This current version has been updated to run more quickly and to fix some analysis errors.

These scripts use some XTCAV data that was made public so they should be runnable by all users. The scripts can be found in <TJ HELP PLEASE> in the files xtcavDark.py, xtcavLasingOff.py, xtcavLasingOn.py.

Quickstart

XTCAV analysis is a three step process involving "dark", "lasing off", and "lasing on" runs. First, we process dark runs, which are runs without any electron bunches. We take the average of a series of dark run images to get the general pixel values of the camera background. We can then subtract this "average image" from the lasing on and off runs to get a closer approximation of the actual energy of the electron bunches. We then process the lasing off runs, which are images of electron bunches without any lasing. We take a series of these images, subtract the average background (from previous step), and zero-out all noise around the region of interest. From this cleaned up image, we can calculate the electron current, the energy dispersion and center of mass, and other physical properties of the electron bunch. We then cluster together these lasing off images into an (optimally or manually chosen) number of groups based of of the electron current. The properties of these profiles are then averaged to create an aggregate lasing off profile. This is done to help reduce noise in the image as well as to speed up the process of choosing a lasing off reference in the lasing on analysis (discussed next). For the lasing on runs, we process each image individually in a similar manner to the lasing off images. Once we calculate the physical properties, we choose an aggregated lasing off profile with the most similar electron current. Then, using either the energy loss or energy spread methods, we can calculate the power of the electron bunch.  

Here are some examples of command line tools and output using the scripts located in xtcav/bin:

Dark

xtcavDark amox23616 104 --max_shots 400

Output

dark background reference
  Experiment: amox23616
  Run: 104
  Valid shots to process: 400
100.0 % done, 400 / 400
Maximum number of images processed


Lasing Off

xtcavLasingOff amox23616 131 --max_shots 200


Output

Lasing off reference
	 Experiment: amox23616
	 Runs: 131
	 Number of bunches: 1
	 Valid shots to process: 200
	 Dark reference run: None
100.0 % done, 200 / 200
Averaging lasing off profiles into  12  groups.

 

Lasing On

xtcavLasingOn amox23616 137 --max_shots 5


Output

Using file 104-end.data for dark reference
Using file 131-end.data for lasing off reference
Agreement: 91.4276%; Maximum power: 6.64857; GW Pulse Delay: 18.512 
Agreement: 91.1052%; Maximum power: 5.82139; GW Pulse Delay: 20.6758 
Agreement: 93.136%; Maximum power: 4.88641; GW Pulse Delay: 18.0389 
Agreement: 94.1173%; Maximum power: 6.76523; GW Pulse Delay: 16.2321 
Agreement: 92.9344%; Maximum power: 7.14828; GW Pulse Delay: 11.2065


Analysis Setup

Two things must be done before XTCAV analysis will function: a "dark run" must be analyzed to get the pedestal values for cameras, and a "no lasing" run must be analyzed to generate sets of "no lasing" images (the latter is quite a complex process). Note that for demonstration these first two scripts write constants to a "local" calibration directory called "calib". For a real-experiment you won't need these lines because you will have permission to write to your official experiment calibration-constants directory.

Sample of dark run analysis:

# these first two lines for example purposes only, to allow user to write
# calibration information to local directory called "calib"
# should be deleted for real analysis.
import psana
psana.setOption('psana.calib-dir','calib')
from xtcav.DarkBackgroundReference import *

DarkBackgroundReference(experiment='xpptut15', 
	run_number='300',   # run number within experiment
	max_shots=1000,		# maximum number of shots to process
	validity_range=(300,302))  # range of runs for which this dark run should be used

Sample of "no-lasing" reference generating script:

# these two lines for example purposes only, to allow user to write
# calibration information to local directory called "calib"
# should be deleted for real analysis.
import psana
psana.setOption('psana.calib-dir','calib')
from xtcav.LasingOffReference import *
LasingOffReference(
	experiment='xpptut15',
	run_number='301',
	max_shots=200,
	num_bunches=1,
	validity_range=(301,)) #only give first run number to have the validity range be open-ended ("end")

This script can be easily run in parallel by submitting a parallel MPI job to the batch system as described here. Once the dark/lasing-off analysis has been completed, users can analyze the lasing-on events using a standard psana-python script similar to the one below.

Example Analysis Script

This script assumes that dark/lasing-off data has been analyzed (see above).   Unlike the previous two scripts it reads dark/lasing-off constants from the official calibration-directory.  

from psana import *
import matplotlib.pyplot as plt
from xtcav.LasingOnCharacterization import *
experiment = 'xpptut15'
run = '302'
mode = 'smd'
ds = psana.DataSource("exp=%s:run=%s:%s" % (experiment, run, mode))
ngood = 0
XTCAVRetrieval=LasingOnCharacterization() 
for evt in ds.events():
    if not XTCAVRetrieval.processEvent(evt):
        continue # continue to next image if analysis fails for some reason
    # time and power are lists, with each entry corresponding to
    # a bunch number.  The number of bunches is set by the GLOC.nb
    # parameter in the lasing-off analysis.  In general, one should
    # also cut on the "agreement" value, which measures the agreement
    # between the first and second moment analysis (larger is better).
    time, power = XTCAVRetrieval.xRayPower(method='RMS')  
    agreement = XTCAVRetrieval.reconstructionAgreement()
    ngood += 1
    print 'Agreement: %g%% ' % (agreement*100)
    plt.plot(time[0],power[0])
    plt.xlabel('Time (fs)')
    plt.ylabel('Lasing Power (GW)')
    plt.title('Agreement %4.2f'%agreement)
    plt.show()
    if ngood > 1: 
        break

Notes: The LasingOnCharacerization module uses the detector interface to find the datasource being used. The program will fail if you try to process events without first setting a datasource. If you are analyzing an older experiment, you may find that psana does not support the 'smd' mode. Instead, use the 'idx' mode.

One caveat: this data shows "horns" at the beginning and the end of the bunch which confuse the algorithm (the accelerator often behaves in this manner).  Only the middle peak in the plotted spectrum is the relevant lasing peak.

Some tips for lasing-on analysis:

  • The second-moment analysis (energy spread method) of the XTCAV image generally provides a better estimate of the true power profile of the XTCAV image. It is recommended that you use method='RMS' when calling xRayPower(), as demonstrated above.
  • Ignore shots where the X-ray intensity is low, but cutting on the FEEGasDetector value to select stronger shots

Examining The "Ingredients" of the XTCAV Analysis

This is a utility that can help users understand why XTCAV results look the way they do.   Run it like this:

 xtcavDisplay exp=xpptut15:run=302

It produces plots that look like this:

These plots show the following quantities for both lasing-on shot that is being analyzed and the lasing-off shot that it is being compared to (which is selected as the one having the closest current-profile):

  • Current
  • Energy computed using the first-moment ("Delta") method
  • Energy computed using the second-moment ("Sigma") method
  • Power

Close the existing plot window to show the plots for the next event.  

XTCAV Analysis Parameters

-experiment: Name of XTCAV experiment (i.e. 'xpptut15')

-run_number: Run number within experiment. Can be in string or integer format.

-num_bunches: Number of bunches: typically 1, but can be 2+ for some LCLS experiments. For analysis of a lasing on run, this must be the same as the number of bunches in the reference lasing off run that you're using.

-max_shots: number of images to process for dark background or lasing off reference generation. In principle the bigger the better, but around 1000 should work fine. If you select a bigger number for dark background generation, then you should get a more representative average of the background noise.  If you select a bigger number for lasing off reference generation, you will get more references. This will take longer to process but you'll have a better chance of finding a well matched profile when processing lasing on shots. 

-num_groups: The number of groups into which lasing off profiles are clustered. Setting this number to 1 would average all of the lasing off profiles in the run together while setting this number to 'maxshots' would not average your profiles at all. Averaging profiles helps to remove some of the noise you might find in an individual profile but may also make comparisons less accurate. Our experiments have found that setting this parameter to maxshots/3 (which should theoretically lead to groups of ~3) works best.

-dark_reference_path/lasingoff_reference_path: File paths for the generated dark and lasing off references you want to use. If none given, the most recent generated reference file (with the run being processed included in the 'validity_range') will be chosen. You can set the `dark_reference_path` when initializing both the LasingOnCharacterization and the LasingOffReference. You can set the `lasingoff_reference_path` when initializing the LasingOnCharacterization.

-roi_expand (default 1): Ratio by which to expand the image from the max and min of the region of interest. This should not affect any analysis but will affect the returned image. Do not set to <1! Call the method 'processedXTCAVImage'. If the image does not look clipped, the parameters are fine.

-roi_fraction (default 0.001): fraction of pixels that must be non-zero in roi(s) of image for analysis.  If you find that all of your images are registering as 'empty', try decreasing this parameter.

-snr_filter (default 10): Number of sigmas for the noise threshold. This is an important parameter. If you find that all of your images are registering as 'empty', try decreasing this parameter. If you find that your images contain a lot of noise, try increasing this parameter. This parameter is particularly important for 2+ bunch data. Since it sets a threshold based on the noise in an image, zeroing out pixels below that threshold, it is the factor that can separate the two island in case of two bunches. For two bunch data, you may need to tweak this parameter until the two bunches are separated for most of the shots. Bottom line:

  • For single bunch: As low as possible while still removing the noise (i.e.) not getting noise current profiles.
  • For double bunch: As low as possible while separating the two bunches. Call the 'processedXTCAVImage' method again, and look at the size of axis 0: this will be the number of bunches detected. You can probably write a program to automatize the determination of this parameter, until you get lets say a 95% of two bunch detection.

-validity_range (default `(current_run_number - end)`): The range of run numbers for which the reference run you are creating is valid. Parameter can be set in initialization of both the DarkBackground and the LasingOffReference. See example above. Note: Hopefully this is obvious, but if you set the validity range to (574,578) then the reference run will not be used for run 579 unless the reference path is explicitly given.

-start_image (default 0): Image within the run to start building the reference from. Can be set for `DarkBackgroundReference` and `LasingOffReference`.

-island_split_method: (default value: 'scipylabel') Only necessary when analyzing data with more than 1 bunch. Several image processing algorithms have been created to separate bunches that appear on the same image.  See here for some algorithm details.

  • 'scipyLabel' calls the scipy label function to label contiguous regions.  this is fast, but requires a region of no signal (after de-noising) between the regions.
  • 'autothreshold' tries to separate the islands by automatically finding thresholds that separate them.  this is the recommended mode for multi-bunch operation.
  • 'contourLabel' tries to adjust a threshold until two large groups are found, grouping together pixels using an opencv contour method.  there are two parameters ('ratio1/ratio2') that are settable for the contour method that determine

Differences from Previous XTCAV code

  • Changes to the user interface:
    • All parameters should be set during the initialization of `DarkBackgroundReference`, `LasingOffReference`, and `LasingOnCharacterization`.
    • There is no longer a `.Generate()` call. A reference will be created upon initialization of the `DarkBackgroundReference` and `LasingOffReference` classes. The reference will be saved to the psana data directory (default location unless the `calibration_path` parameter is set). If you would like to access the generated reference profiles, you can either load them from the saved file or access the member variable `averaged_profiles` within the `LasingOffReference` class.
    • Some parameters, such as roi_waist_threshold, have been deprecated. The current code automatically chooses this value for you based on the `snr_filter` (see documentation above). Other parameter names have changed so that naming convention is consistent across the entire xtcav package (again, see documentation above). 
  • Changes to analysis:
    • Improved profile clustering algorithm
    • Image is no longer median filtered before analysis
    • Power profile for lasing off image is not averaged between RMS and COM methods
  • Other changes include: improved code readability, integration with psana detector interface, and speed up in processing time.

Detector Resolution

(From Tim Maxwell)

Time resolution is around 1.1 fs RMS for soft x-rays, 2.5 fs fwhm (in quadrature, of course). So actually pulse length is probably 4.3 - 9.7 fs FWHM.

This also doesn't include the "slippage resolution." That is, if they're using the full undulator, then by the end the x-rays can have slipped out of the electron slice by ~3 fs for soft x-rays. Obviously not a small number if trying to make 5 fs pulses. They've been advised to not use the full undulator when shorter pulses are more important than number of photons.

  • No labels