Versions Compared

Old Version 29

changes.mady.by.user Unknown User (ofte)

Saved on

compared with

New Version 30

changes.mady.by.user Unknown User (ofte)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The most pain-free way to access LCLS XTC data files from python is through LCLS's python framework, pyana. It is a non-interactive framework, but to some extent you can work interactively with the data it produces
All about pyana.

C++ framework: psana

The idea is the same as for pyana. Non-interactive. No interactive support as of yet.
All about psana - Original Documentation.

If you like GUIs:

The XTC Explorer - Old gives you an "interactive" way to configure your analysis.

...

Python/IPython can be used to analyze data after you've saved them, or they can be embedded into a pyana module to give you interactive access to the data at regular intervals throughout your analysis.
'IPython' (http://ipython.org/Image Removed) is an enhanced python shell for interactive use. Many of the examples here would work equally well with a 'regular' python shell.
Plotting is done with 'matplotlib' (http://matplotlib.sourceforge.net/Image Removed)
If you're looking for an IDE to work with, consider 'Spyder' (http://code.google.com/p/spyderlib/Image Removed).

Interactively exploring the XTC file.

...

Code Block
usage: xtcscanner [options] xtc-files ...

options:
  -h, --help            show this help message and exit
  -n NDATAGRAMS, --ndatagrams=NDATAGRAMS
  -v, --verbose
  -l L1_OFFSET, --l1-offset=L1_OFFSET

Example:

none
Code Block
none
titlextcscanner -n 200 /reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s0*
none
Scanning....
Start parsing files:
['/reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s00-c00.xtc', '/reg/d/psdm/AMO/amo01509/xtc/e8-r0094-s01-c00.xtc']
  201 datagrams read in 0.070000 s .   .   .   .   .   .   .
-------------------------------------------------------------
XtcScanner information:
  - 1 calibration cycles.
  - Events per calib cycle:
   [197]

Information from  0  control channels found:
Information from  9  devices found
                      BldInfo:EBeam:             EBeamBld (197)
            BldInfo:FEEGasDetEnergy:             FEEGasDetEnergy (197)
        DetInfo:AmoETof-0|Acqiris-0:  (5 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
      DetInfo:AmoGasdet-0|Acqiris-0:  (2 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
        DetInfo:AmoITof-0|Acqiris-0:  (1 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
        DetInfo:AmoMbes-0|Acqiris-0:  (1 ch)     AcqConfig_V1 (1)   AcqWaveform_V1 (197)
     DetInfo:EpicsArch-0|NoDevice-0:             Epics_V1 (688)
         DetInfo:NoDetector-0|Evr-0:             EvrConfig_V2 (1)
                          ProcInfo::             RunControlConfig_V1 (11)
XtcScanner is done!
-------------------------------------------------------------

...

With interactive python embedded, see: https://confluence.slac.stanford.edu/display/PCDS/XTC+Explorer#XTCExplorer-InteractiveplottingwithIPythonImage Removed

IPython used "like" MATLAB

...

Starting an interactive session

Code Block
nonenone
titleStarting iPython
borderStylesolid
none
[ofte@psana0XXX myrelease]$ ipython -pylab
Python 2.4.3 (#1, Nov  3 2010, 12:52:40)
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.
Code Block
nonenone
titleload the library module
borderStylesolid
none
In [1]: from pymatlab import *

Generally, it is recommended to load library modules with 'import pymatlab' and access all its methods and classes with pyamatlab.function. In an interactive session it may be easier to have access to the contents of pymatlab in your immediate workspace by doing 'from pymatlab import *'.

Code Block
nonenone
titleList the workspace contents ('who' or 'whos')
borderStylesolid
none
In [2]: who
H5getobjnames   ScanInput       ScanOutput      filtvec findmovingmotor
getSTDMEANfrac_from_startpoint  get_filter get_limits       get_limits_automatic
get_limits_channelhist  get_limits_correlation  get_limits_corrfrac
h5py    np      plt     rdXPPdata       runexpNO2fina       scan       scaninput

In [3]: whos
Variable                         Type          Data/Info
--------------------------------------------------------
H5getobjnames                    function      <function H5getobjnames at 0x2b57de8>
ScanInput                        type          <class 'pymatlab.ScanInput'>
ScanOutput                       type          <class 'pymatlab.ScanOutput'>
filtvec                          function      <function filtvec at 0x2b57f50>
findmovingmotor                  function      <function findmovingmotor at 0x2b57d70>
getSTDMEANfrac_from_startpoint   function      <function getSTDMEANfrac_<...>_startpoint at 0x2b581b8>
get_filter                       function      <function get_filter at 0x2b57ed8>
get_limits                       function      <function get_limits at 0x2b58050>
get_limits_automatic             function      <function get_limits_automatic at 0x2b58230>
get_limits_channelhist           function      <function get_limits_channelhist at 0x2b582a8>
get_limits_correlation           function      <function get_limits_correlation at 0x2b580c8>
get_limits_corrfrac              function      <function get_limits_corrfrac at 0x2b58140>
h5py                             module        <module 'h5py' from '/reg<...>ython/h5py/__init__.pyc'>
np                               module        <module 'numpy' from '/re<...>thon/numpy/__init__.pyc'>
plt                              module        <module 'matplotlib.pyplo<...>n/matplotlib/pyplot.pyc'>
rdXPPdata                        function      <function rdXPPdata at 0x2b57c80>
runexpNO2fina                    function      <function runexpNO2fina at 0x2b57e60>
scan                             ScanOutput    <pymatlab.ScanOutput object at 0x2b60536bee90>
scaninput                        ScanInput     <pymatlab.ScanInput object at 0x2b60536b4e90>

...

Table of comparison (MATLAB vs MatPlotLib)

See also http://www.scipy.org/NumPy_for_Matlab_UsersImage Removed

MatLab

MatPlotLib

Comments

Loglog plot of one array vs. another

Code Block
%
%
%
a1 = subplot(121);
loglog(channels(:,1),channels(:,2),'o')
xlabel('CH0')
ylabel('CH1')
a2 = subplot(122);
loglog(channels(:,3),channels(:,4),'o')
xlabel('CH2')
ylabel('CH3')

Loglog plot of one array vs. another

Code Block
import matplotlib.pyplot as plt
import numpy as np

a1 = plt.subplot(221)
plt.loglog(channels[:,0],channels[:,1], 'o' )
plt.xlabel('CH0')
plt.ylabel('CH1')
a2 = plt.subplot(222)
plt.loglog(channels[:,2],channels[:,3], 'o' )
plt.xlabel('CH2')
plt.ylabel('CH3')

channels is a 4xN array of floats, where N is the number of events. Each column corresponds to one out of four Ipimb channels.

Note that the arrays are indexed with 1,2,3,4 in MatLab and 0,1,2,3 in MatPlotLib/NumPy/Python.
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="e6286353-7174-4eff-8099-a8d90a710ad4"><ac:plain-text-body><![CDATA[
Note also the use of paranthesis, array() in MatLab, array[] in MatPlotLib. ]]></ac:plain-text-body></ac:structured-macro>

test

test

Test

array of limits from graphical input

array of limits from graphical input

 

Code Block
axes(a1)
hold on
lims(1:2,:) = ginput(2);

axes(a2)
hold on
lims(3:4,:) = ginput(2);
Code Block
lims = np.zeros((4,2),dtype="float")

plt.axes(a1)
plt.hold(True)
lims[0:2,:] = plt.ginput(2)

plt.axes(a2)
plt.hold(True)
lims[2:4,:] = plt.ginput(2)

In MatLab, lims is an expandable array that holds limits as set by input from mouse click on the plot (ginput).
NumPy arrays cannot be expanded, so I've declared a 4x2 array of zeros to start with, then fill it with ginput().

 

 

 

filter

filter

 

Code Block
fbool1 = (channels(:,1)>min(lims(1:2,1)))&(channels(:,1)<max(lims(1:2,1)))
fbool2 = (channels(:,2)>min(lims(1:2,2)))&(channels(:,2)<max(lims(1:2,2)));
fbool = fbool1&fbool2
loglog(channels(fbool,1),channels(fbool,2),'or')

fbool3 = (channels(:,3)>min(lims(3:4,3)))&(channels(:,3)<max(lims(3:4,3)))
fbool4 = (channels(:,4)>min(lims(3:4,4)))&(channels(:,4)<max(lims(3:4,4)));
fbool = fbool3&fbool4
loglog(channels(fbool,3),channels(fbool,4),'or')
Code Block
fbools0 = (channels[:,0]>lims[:,0].min())&(channels[:,0]<lims[:,0].max())
fbools1 = (channels[:,1]>lims[:,1].min())&(channels[:,1]<lims[:,1].max())
fbools = fbools0 & fbools1

fbools2 = (channels[:,2]>lims[:,2].min())&(channels[:,2]<lims[:,2].max())
fbools3 = (channels[:,3]>lims[:,3].min())&(channels[:,3]<lims[:,3].max())
fbools = fbools2&fbools3

Comment

 

 

 

...

Simple array to a NumPy file:

Code Block
nonenone
titlesaving and loading
none
import numpy as np

np.save("filename.npy", array)
array = np.load("filename.npy")

np.savetxt("filename.dat", array)
array = loadtxt("filename.dat")

...

Simple array to an HDF5 file

none
Code Block
none
titlesaving simple arrays to HDF5
none
import h5py

def beginjob(self,evt,env):
    self.ofile = h5py.File("outputfile.hdf5", 'w') # open for writing (overwrites existing file)
    self.shot_counter = 0

def event(self,evt,env)
    # example: store several arrays from one shot in a group labeled with shot (event) number
    self.shot_counter += 1
    group = self.ofile.create_group("Shot%d" % self.shot_counter)

    image1_source = "CxiSc1-0|TM6740-1"
    image2_source = "CxiSc1-0|TM6740-2"

    frame = evt.getFrameValue(image1_source)
    image1 = frame.data()
    frame = evt.getFrameValue(image2_source)
    image2 = frame.data()

    dataset1 = group.create_dataset("%s"%image1_source,data=image1)
    dataset2 = group.create_dataset("%s"%image2_source,data=image2)

def endjob(self,env)
    self.ofile.close()

...