Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The most pain-free way to access LCLS XTC data files from python is through LCLS's python framework, pyana. It is a non-interactive framework, but to some extent you can work interactively with the data it produces
All about pyana.

C++ framework: psana

The idea is the same as for pyana. Non-interactive. No interactive support as of yet.
All about psana - Original Documentation.

If you like GUIs:

The XTC Explorer - Old gives you an "interactive" way to configure your analysis.

...

Python/IPython can be used to analyze data after you've saved them, or they can be embedded into a pyana module to give you interactive access to the data at regular intervals throughout your analysis.
'IPython' (http://ipython.org/Image Removed) is an enhanced python shell for interactive use. Many of the examples here would work equally well with a 'regular' python shell.
Plotting is done with 'matplotlib' (http://matplotlib.sourceforge.net/Image Removed)
If you're looking for an IDE to work with, consider 'Spyder' (http://code.google.com/p/spyderlib/Image Removed).

Interactively exploring the XTC file.

...

With interactive python embedded, see: https://confluence.slac.stanford.edu/display/PCDS/XTC+Explorer#XTCExplorer-InteractiveplottingwithIPythonImage Removed

IPython used "like" MATLAB

...

Table of comparison (MATLAB vs MatPlotLib)

See also http://www.scipy.org/NumPy_for_Matlab_UsersImage Removed

MatLab

MatPlotLib

Comments

Loglog plot of one array vs. another

Code Block
%
%
%
a1 = subplot(121);
loglog(channels(:,1),channels(:,2),'o')
xlabel('CH0')
ylabel('CH1')
a2 = subplot(122);
loglog(channels(:,3),channels(:,4),'o')
xlabel('CH2')
ylabel('CH3')

Loglog plot of one array vs. another

Code Block
import matplotlib.pyplot as plt
import numpy as np

a1 = plt.subplot(221)
plt.loglog(channels[:,0],channels[:,1], 'o' )
plt.xlabel('CH0')
plt.ylabel('CH1')
a2 = plt.subplot(222)
plt.loglog(channels[:,2],channels[:,3], 'o' )
plt.xlabel('CH2')
plt.ylabel('CH3')

channels is a 4xN array of floats, where N is the number of events. Each column corresponds to one out of four Ipimb channels.

Note that the arrays are indexed with 1,2,3,4 in MatLab and 0,1,2,3 in MatPlotLib/NumPy/Python.
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="8d94225f-ad04-45c3-9312-66ed7fea74be"><ac:plain-text-body><![CDATA[
Note also the use of paranthesis, array() in MatLab, array[] in MatPlotLib. ]]></ac:plain-text-body></ac:structured-macro>

test

test

Test

array of limits from graphical input

array of limits from graphical input

 

Code Block
axes(a1)
hold on
lims(1:2,:) = ginput(2);

axes(a2)
hold on
lims(3:4,:) = ginput(2);
Code Block
lims = np.zeros((4,2),dtype="float")

plt.axes(a1)
plt.hold(True)
lims[0:2,:] = plt.ginput(2)

plt.axes(a2)
plt.hold(True)
lims[2:4,:] = plt.ginput(2)

In MatLab, lims is an expandable array that holds limits as set by input from mouse click on the plot (ginput).
NumPy arrays cannot be expanded, so I've declared a 4x2 array of zeros to start with, then fill it with ginput().

 

 

 

filter

filter

 

Code Block
fbool1 = (channels(:,1)>min(lims(1:2,1)))&(channels(:,1)<max(lims(1:2,1)))
fbool2 = (channels(:,2)>min(lims(1:2,2)))&(channels(:,2)<max(lims(1:2,2)));
fbool = fbool1&fbool2
loglog(channels(fbool,1),channels(fbool,2),'or')

fbool3 = (channels(:,3)>min(lims(3:4,3)))&(channels(:,3)<max(lims(3:4,3)))
fbool4 = (channels(:,4)>min(lims(3:4,4)))&(channels(:,4)<max(lims(3:4,4)));
fbool = fbool3&fbool4
loglog(channels(fbool,3),channels(fbool,4),'or') 
Code Block
fbools0 = (channels[:,0]>lims[:,0].min())&(channels[:,0]<lims[:,0].max())
fbools1 = (channels[:,1]>lims[:,1].min())&(channels[:,1]<lims[:,1].max())
fbools = fbools0 & fbools1

fbools2 = (channels[:,2]>lims[:,2].min())&(channels[:,2]<lims[:,2].max())
fbools3 = (channels[:,3]>lims[:,3].min())&(channels[:,3]<lims[:,3].max())
fbools = fbools2&fbools3

Comment

 

 

 

Writing Numpy and HDF5 files from python

You can store numpy arrays from a pyana job (reads XTC) and store them in simple numpy files or HDF5 files. Here are some examples:

Simple array to a NumPy file:

Code Block
none
none
titlesaving and loading
import numpy as np

np.save("filename.npy", array)
array = np.load("filename.npy")

np.savetxt("filename.dat", array)
array = loadtxt("filename.dat")

This example shows saving and loading of a binary numpy file (.npy) and an ascii file (.dat).
This only works with single arrays (max 2 dimensions).
If you need to save multiple events/shots in the same file you will need to do some tricks (e.g. flatten the array and stack 1d arrays into 2d arrays where axis2 represent event number). Or you could save as an HDF5 file.

Simple array to an HDF5 file

Code Block
none
none
titlesaving simple arrays to HDF5
import h5py

def beginjob(self,evt,env):
    self.ofile = h5py.File("outputfile.hdf5", 'w') # open for writing (overwrites existing file)
    self.shot_counter = 0

def event(self,evt,env)
    # example: store several arrays from one shot in a group labeled with shot (event) number
    self.shot_counter += 1
    group = self.ofile.create_group("Shot%d" % self.shot_counter)

    image1_source = "CxiSc1-0|TM6740-1"
    image2_source = "CxiSc1-0|TM6740-2"

    frame = evt.getFrameValue(image1_source)
    image1 = frame.data()
    frame = evt.getFrameValue(image2_source)
    image2 = frame.data()

    dataset1 = group.create_dataset("%s"%image1_source,data=image1)
    dataset2 = group.create_dataset("%s"%image2_source,data=image2)

def endjob(self,env)
    self.ofile.close()

This example is shown in a pyana setting. The HDF5 file is declared and opened in beginjob, datasets created for each event, and the file is closed in the endjob method.
Or you can group your datasets any other way you find useful, of course.

Saving complex datasets to HDF5 file

Some more advanced examples (courtesy of Hubertus Bromberger):

...