Page History
Include Page | ||||
---|---|---|---|---|
|
Table of Contents |
---|
Include Page | ||||
---|---|---|---|---|
|
Motivation
Development of this application was stimulated by the discussion with Marcin Sikorski (meeting on
...
2012-08-30),
...
doing
...
xcs
...
experiments.
...
Users
...
need
...
in
...
real-time
...
algorithm
...
for
...
calculation
...
of
...
image
...
vs
...
time
...
auto-correlation
...
function
Code Block |
---|
} g2(tau) = <I(t)*I(t+tau)> / (<I(t)> * <I(t+tau)>), {code} where {{ |
where I(t)
...
is
...
an
...
image
...
intensity
...
at
...
time
...
t
...
,
...
and
...
tau
...
is
...
a
...
delay
...
between
...
two
...
measurements.
...
Typical
...
experimental
...
condition
...
can
...
be
...
described
...
as
...
follows:
...
- Run
...
- duration
...
- is
...
- about
...
- one
...
- hour
...
- at
...
- frequency
...
- up
...
- to
...
- 120
...
- Hz
...
- that
...
- gives
...
- up
...
- to
...
- 10^5-10^6
...
- images.
...
- Currently
...
- typical
...
- imaging
...
- devise
...
- is
...
- a
...
- Princeton
...
- camera
...
- with
...
- 1300x1340
...
- pixels.
...
- Need
...
- to
...
- calculate
...
g2(tau)
...
- for
...
- each
...
- pixel,
...
- averaged
...
- over
...
- all
...
- possible
...
- image
...
- times
...
t
...
- with
...
- time
...
- difference
...
tau
...
- between
...
- images.
...
- A
...
- set
...
- of
...
tau
...
- should
...
- have
...
- about
...
- 30-100
...
- points
...
- in
...
- log
...
- scale
...
- uniformly
...
- covering
...
- the
...
- run
...
- duration.
...
- Use
...
- for
...
- example
...
- xcsi0112-r0015:
...
- 500
...
- images
...
- with
...
- 8
...
- sec
...
- delay
...
- between
...
- images.
...
Desired
...
- time
...
- for
...
- evaluation
...
- of
...
- the
...
- auto-correlation
...
- function
...
- should
...
- be
...
- comparable
...
- with
...
- run
...
- duration
...
- <1
...
- hour.
...
- Currently
...
- this
...
- algorithm
...
- takes
...
- a
...
- few
...
- hours
...
- that
...
- can
...
- not
...
- be
...
- used
...
- for
...
- fast
...
- feedback
...
- in
...
- real
...
- time
...
- experiment.
...
In
...
first
...
approximation
...
this
...
problem
...
was
...
solved,
...
as
...
it
...
is
...
described
...
in
...
the
...
...
...
...
...
...
...
...
.
...
However,
...
in
...
order
...
to
...
be
...
useful,
...
the
...
command
...
line
...
algorithm
...
needs
...
to
...
be
...
integrated
...
in
...
the
...
global
...
analysis
...
system,
...
which
...
de
...
bene
...
esse
...
called
...
as
...
Integrated
...
Analysis
...
Environment
...
for
...
Time
...
Correlation
...
Experiments
...
and
...
is discussed in IDPE for TCE in XCS - Problems and tentative solutions.
Wiki Markup |
---|
{htmlcomment:hidden}
Here is my comment
{htmlcomment} |
GUI Implementation
Updated on 2013-04-01
As an example of GUI style Marcin suggested to use XPCSGUI - earlier implementation of similar application.
Three versions of GUI were implemented for this application. It was decided to use the GUIMainTB
layout, where:
- All input windows are integrated in one with tab-bar for switching and custom tool bar on the top
- Logger and File Browser windows are isolated
- All plots are isolated, although similar plots appear in the same windows.
h3 Plots for intensity in q-static bins
Intensity monitors
FEEGasDetEnergy, XCS-IPM-02, XCS-IPM-mono, XcsBeamline.1:Ipimb.4, and XcsBeamline.1:Ipimb.5
Reflective scattering geometry
Source code from Marcin: q_functions.txt
Code location
Everything resides in CorAna
package, which is included in LCLS offline releases beginning from ana-0.7.16
.
References to the specific algorithms can be found in Algorithms for Time Correlation Experiments.
Original data processing code from Marcin is described in Note on XCS code from Marcin.
Modules
In tables below we list modules from packages CorAna
and ImgAlgos
, which were developed for this project.
Notations:
- is done (at least it is assumed for now...)
- needs more work
- test or deprecated modules
Package CorAna
Updated on 2013-10-14
All module names beginning with letters GUI
implements different Graphical User Interfaces.
List of modules in alphabetic order:
Module | Description |
---|---|
AppDataPath.py | Local version with added path to data from src directory. |
BatchJob.py | Suprclass for other |
BatchJobCorAna.py | Â |
BatchJobData.py | Â |
BatchJobPedestals.py | Class contains methods for batch job submission and monitoring for pedestals. |
BatchLogParser.py | Â |
ConfigFileGenerator.py | Class has methods to generate psana configuration and other scripts from stubs located in |
ConfigParameters.py | Base class for configuration parameters. |
ConfigParametersCorAna.py | Sub-class for |
CorAnaUtils.py | Junk for eamples. |
Drag.py | Â |
DragCenter.py | Â |
DragCircle.py | Â |
DragLine.py | Â |
DragObjectSet.py | Â |
DragPolygon.py | Â |
DragRectangle.py | Â |
DragWedge.py | Â |
EventTimeRecords.py | Â |
FileNameManager.py | Class dynamically generates all file names for current configuration parameters. |
GUIAnaPartitions.py | Â |
GUIAnaSettings.py | Â |
GUIAnaSettingsLeft.py | Â |
GUIAnaSettingsOptions.py | Â |
GUIAnaSettingsRight.py | Â |
GUIBlamish.py | Sub-GUI of GUIFiles.py |
GUICCDSettings.py | Â |
GUIConfigParameters.py | GUI for the configuration parameters file management. |
GUIDark.py | Sub-GUI of GUIFiles.py - manipulations with dark runs |
GUIData.py | Â |
GUIELogPostingDialog.py | Dialog window for submission of messages with attachments to ELog using response ID and Igor's new interface: Python module for posting message into ELog |
GUIELogPostingFields | Widget with necessary fields for GUIELogPostingDialog |
GUIFileBrowser.py | Text file browser for this project |
GUIFiles.py | Central GUI for file settings contains tab-bar for other widget selection |
GUIFlatField.py | Sub-GUI of GUIFiles.py |
GUIHelp.py | WIdget for messages, i.e. help |
GUIImgSizePosition.py | Â |
GUIInstrExpRun.py | depricated GUI |
GUIIntensityMonitors.py | Control GUI for intensity monitors |
GUIKineticMode.py | Â |
GUIListOfTau.py | GUI manipulation with list of tau indexes |
GUILogger.py | GUI for logger |
GUIMain.py | The first GUI at start of this application |
GUIMainSplit.py | The same as GUIMainTB.py with integrated GUILogger.py |
GUIMainTB.py | The first GUI at start of this application with tab bar |
GUINonKineticMode.py | Â |
GUIRun.py | Â |
GUIRunAuto.py | Â |
GUIRunInput.py | Â |
GUIRunMerge.py | Â |
GUIRunProc.py | Â |
GUIRunSplit.py | Â |
GUISetupBeamZero.py | Â |
GUISetupData.py | Â |
GUISetupEnergyAngle.py | Â |
GUISetupInfo.py | Â |
GUISetupInfoLeft.py | Â |
GUISetupInfoRight.py | Â |
GUISetupPars.py | Â |
GUISetupSpecular.py | Â |
GUISystemSettings.py | Â |
GUISystemSettingsLeft.py | Â |
GUISystemSettingsRight.py | Â |
GUIViewControl.py | Â |
GUIViewResults.py | Â |
GUIWorkResDirs.py | Â |
GlobalExternal.py | Â |
GlobalUtils.py | Module contains all possible global methods. |
ImgSpeNavToolBar.py | Re-implemented standard tool-bar (depricated) |
Logger.py | Core class for logger |
MaskEditor.py | Â |
MaskEditorButtons.py | Â |
Overlay.py | Demonstration of how to draw something on the top of GUI |
PlotArray.py | Main class for array presentation plot |
PlotArrayButtons.py | Widget for custom button-bar |
PlotArrayWidget.py | Widget for graphical window |
PlotG2.py | Â |
PlotG2Buttons.py | Â |
PlotG2Widget.py | Â |
PlotGraph.py | Â |
PlotGraphWidget.py | Â |
PlotImgSpe.py | Main class for interactive plot with custom button bar |
PlotImgSpeButtons.py | Widget for custom button-bar |
PlotImgSpeWidget.py | Graphical image for image and spectral histogram |
PlotTime.py | Main class for time record presentation plot |
PlotTimeWidget.py | Widget for graphical window |
RecordsFromFiles.py | Â |
ThreadWorker.py | Â |
ViewResults.py | Â |
data/scripts/psana-*.cfg | Stub-scripts for ConfigFileGenerator.py |
Modules used from package ImgAlgos
Updated on 2013-10-14
List of modules in alphabetic order:
Module | Description |
---|---|
CorAna | Superclass for all CorAna* modules |
CorAnaData | Processing of split images, evaluation of correlators |
CorAnaInputParameters | Input parameters parser |
CorAnaMergeFiles | Merges split files with correlatros in a single file |
CorAnaPars.py | Holds common parameters |
CorAnaProcResults | Example of stand-alone processing of the file with correlators |
CorAnaSubmit.py | Command-line submission procedure for processing |
GlobalMethods | Common global methods for |
ImgAverage | Generic |
ImgCalib | Generic |
ImgIntForBins | Generic |
ImgIntMonCorr | Generic |
ImgMaskEvaluation | Generic |
ImgTimeStampList | Generic |
ImgVsTimeSplitInFiles | Generic |
IntensityMonitorsData | Generic |
PrincetonImageProducer | Generic |
Tahometer | Generic |
Functional description
Updated on 2013-10-14
In this section modules are listed in functional order.
PSANA modules
Dark run pre-processing
Scaner
Runs in batch for the dark file to get preliminary information.
ImgAlgos.ImgTimeStampList
- counts number of events in the file
- makes file with time stamps
- evaluate time intervals between frames (for dark run)
Pedestals
Runs in batch for the dark file to get averaged pedestals.
ImgAlgos.PrincetonImageProducer
gets image from event as an ndarray
object
ImgAlgos.Tahometer
evaluates performance of the batch job
ImgAlgos.ImgAverage
produces files with averaged, rms-spread, and hot-pixel mask for images in a given range of events.
Data pre-processing
Scaner
Runs in batch for the data file to get preliminary information.
ImgAlgos.Tahometer
evaluates performance of the batch job, counts number of events in the data file
ImgAlgos.ImgTimeStampList
makes file with time stamps and time record counters for tau
ImgAlgos.IntensityMonitorsData
makes file with intensity monitor records
Average
Runs in batch for the data file to get averaged image
ImgAlgos.Tahometer
evaluates performance of the batch job
ImgAlgos.PrincetonImageProducer
gets image from event as an ndarray
object
ImgAlgos.ImgAverage
produces file with averaged and rms-spread for raw images in a given range of events
ImgAlgos::ImgMaskEvaluation
- module is configured to evaluate masks:
- saturated mask - pixel is considered as saturated if its amplitude exceeds the threshold at least once per run
- noise mask - pixel is considered as noisy if its amplitude exceeds the MEAN+5*RMS in 5% of events. The "noise" MEAN and RMS are evaluated for 8 (or less on the boarder) surrounding pixels. This mask is not used in current analysis.
Data processing
Split
ImgAlgos.Tahometer
for performance evaluation.
ImgAlgos.PrincetonImageProducer
ImgAlgos::ImgCalib
module is configured to:
- subtract pedestals obtained for dark run
- account for threshold (LLD) constant or in number of RMS
ImgAlgos.ImgIntMonCorr
image normalization on intensity monitors data
ImgAlgos.ImgVsTimeSplitInFiles
split image and save blocks for all events in separate files.
ImgAlgos.ImgIntForBins
evaluate intensity (averaged over bin pixels) in bins per event, bin numbers are defined by the map, results are saved in the file.
ImgAlgos.ImgAverage
produces file with averaged and rms-spread for images with subtracted pedestals and allied LLD in a given range of events
Process
Data processing is implemented in stand alone (non-psana) modules
ImgAlgos.CorAna.cpp
ImgAlgos.CorAnaInputParameters.cpp
ImgAlgos.CorAnaData.cpp
Merge
ImgAlgos.CorAna.cpp
ImgAlgos.CorAnaInputParameters.cpp
CorAnaMergeFiles.cpp
- saves binary file for float(32) with shape (Ntau,3,rows,cols), where 3 stands for <Ip>, <If>, and <Ip*If>
Can be acessed in python as
Code Block |
---|
discussed below. h1. Problems and tentative solutions Based on 2012-10-03 meeting: In order to be useful this application should do correct math, accounts for image mask, discards bad events (noisy and "bright" pixels), apply normalization etc., and have a convenient GUI. Below is a list of requirements, marked as (?), with suggested solutions, marked as (/) if exists or as (+) if needs to be implemented. h2. Pedestals (?) "dark" run number should be provided by user and the imaging camera pedestals should be evaluated and applied for all runs until the "dark" run number has not changed. (/) For pedestals evaluation: use available {{ImgAlgos::ImgAverage}} psana module for "dark" run, which produces file with averaged over events pedestals and the file with their rms values. (/) For pedestals subtraction: use {{ImgAlgos::ImgCalib}} psana module in the same job which evaluates correlators; the pedestals will be subtracted and corrected image will be retained in the event and used for correlator calculations. h2. Low level threshold (?) Image pixel intensity physically can't be negative. Low amplitude noise should be suppressed by the threshold. The threshold amplitude should be provided by user (along with substituting amplitude). (/) Is implemented in the {{ImgAlgos::ImgCalib}} psana module, right after pedestal subtractuion. h2. Image filtering (?) Usually users use different type of intensity monitor signals in order to retain/discard image for/from further processing. Discarded images should not contribute into the correlation analysis. The spectra of intensity monitors should be available for browsing. User should be able to select the intensity monitor(s) from the list and set low and high thresholds. (+) The filtering module may be implemented in psana. Based on selected intensity monitor(s) and thresholds it will decide to retain or discard event and accumulate spectral histograms. The histograms will be saved in file at the end of run. (+) Control GUI should be able to browse the intensity monitor histograms and set the thresholds. h2. Selection of intensity monitors (?) It would be nice to have an algorithm like in XTC explorer (+) Possible options: * run application as a plug-in for XTC Explorer, * pyana module performing similar to XTC Explorer algorithm, * stand-alone C\+\+ module reading XTC datagrams, * hardwired list of intensity monitors. * psana module, which needs to be developed h2. Dynamic mask (?) Imaging camera may have permanently hot pixels or some pixels may be saturated during the run. User need to set a thresholds on hot pixels and high intensity. If the pixel amplitude crosses the high intensity threshold at least once during the run, then this pixel should be excluded from further analysis. The same is valid for hot pixels, which shows above threshold intensity in large fraction of events. (/) This is implemented in the psana module {{ImgAlgos::ImgMaskEvaluation}}, which works before event selection algorithm. Two files of image size may be produced 1) for saturated and 2) for hot pixels. h2. Static mask (?) The beam-stopper region and some areas with fringes should be masked. It could be useful to have a graphical editor for mask. (+) See section for [GUI|Command Line Interface For Time Correlation Analysis#GUI]. h2. Graphical editor for selected regions (?) Sometimes it is useful to select good region of the image. It could be convenient to use a graphical editor, as for mask. (+) See section for [GUI|Command Line Interface For Time Correlation Analysis#GUI]. h2. Center of the image (?) User should have an option to set a center of the rings for histograms. (+) See section for [GUI|Command Line Interface For Time Correlation Analysis#GUI]. h2. Correct normalization of g2 (?) Evaluation of {{g2}} for image *regions* is not that simple as presented by the formula for a single pixel: {code} g2(tau) = <I(t)*I(t+tau)> / (<I(t)> * <I(t+tau)>), {code} In order to get physically meaningful results for g2, the correlators <I(t)> and <I(t+tau)> should be averaged in the fine rings around center with number of bins N2, which is order of 100, with dR down to 1-2 pixels. Then the <I(t)*I(t+tau)> (?) correlator should be averaged over bold rings intended for G2 evaluation. The number of these rings N1 should be order of 10. The N2 and N1 should be defined by user. It might be useful to define the histogram region by the sector in the user-defined angular range. (+) In order to have required normalization of correlators, it is not enough to save the {{g2}} value only. So, the format of the resulting file has changed. Now for each value of {{tau}} the output file contains the <I(t)>, <I(t+tau)>, and <I(t)*I(t+tau)>, each for entire image written in binary for float format. Not all masks, selection regions, normalization etc. are available during correlators calculation, so correlators are evaluated for all pixels. Which pixels should be included in the G2 for each region can be decided at the final stage of processing. This approach allows to perform the most time consuming procedure - the correlators calculation once and do the analysis after that. (+) See section for [GUI|Command Line Interface For Time Correlation Analysis#GUI]. h2. GUI (?) In order to get an easy interface to all sub-processes, it seems to be useful to have a GUI with configuration of everything through the GUI. (!) Well, presumably users will want different specific features in their analyses which can not be foreseen in implementation of GUI. It is pretty unlikely that everything in analysis can be done clicking on buttons in GUI. Then, it could be nice if user understand what he is doing step by step and have a monitoring at the end of each stage. We are doing science, not a standard pre-defined things... Most generic way to process data is to have a separate procedures with command line interface. (+) Anyway, the browser/presenter of data stored in the files after pre-processing could be provided for a set of common plots. All features listed in previous sections, such as static and dynamic mask, restriction of the region(s) of interest, selection of the image center, the binning scheme etc., can be done in the browser at the final stage of the analysis. (+) Generic GUI as a shell for entire analysis can be implemented as an interface to the command line procedures: * Each command line procedure may have a dedicated GUI for procedure configuration and monitoring. * All procedures may be listed in the main GUI with a status sign. * The main issue of this approach is a cross-configuration between separate procedures. This can be achieved if all procedures will have a common list of configuration parameters. ---- ---- ---- ---- ---- ---- {HTMLcomment:hidden} Here is my comment {HTMLcomment} h1. GUI Implementation Updated on 2012-12-02 Three [versions of GUI|PCDS:Versions of GUI for Integrated Data Processing Environment for Time Correlation Experiments] were implemented for this application. It was decided to use the {{GUIMainTB}} layout, where: * All input windows are integrated in one with tool bar * Logger and File browser windows are isolated * Plots are isolated !cora-xcsi0112-r0015-gui-files-data.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-files-dark.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-files-blamish.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-files-flat.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-files-confpars.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-files-work.png|thumbnail,border=1! !p-data-img.png|thumbnail,border=1!!p-data-peds.png|thumbnail,border=1!!p-data-time.png|thumbnail,border=1! !GUILogger.png|thumbnail,border=1!!GUIFileBrowser.png|thumbnail,border=1! !cora-xcsi0112-r0015-gui-setup-info.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-setupinfo-data.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-analysissettings-dynamic.png|thumbnail,border=1! !cora-xcsi0112-r0015-gui-system.png|thumbnail,border=1! !cora-xcsi0112-r0015-gui-intensitymonitors.png|thumbnail,border=1!!p-intmon.png|thumbnail,border=1! !cora-xcsi0112-r0015-gui-run-input.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-run-split.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-run-process.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-run-merge.png|thumbnail,border=1!!cora-xcsi0112-r0015-gui-run-auto.png|thumbnail,border=1! !cora-xcsi0112-r0015-gui-view-results.png|thumbnail,border=1! !p-X.png|thumbnail,border=1!!p-Y.png|thumbnail,border=1!!p-R.png|thumbnail,border=1! !p-PHI.png|thumbnail,border=1!!p-Q.png|thumbnail,border=1! !p-mask.png|thumbnail,border=1! !p-phi-stat.png|thumbnail,border=1!!p-q-stat.png|thumbnail,border=1!!p-q-phi-stat.png|thumbnail,border=1! !p-phi-dyna.png|thumbnail,border=1!!p-q-dyna.png|thumbnail,border=1!!p-q-phi-dyna.png|thumbnail,border=1! !p-Ip.png|thumbnail,border=1!!p-IpxIf.png|thumbnail,border=1!!p-g2-map.png|thumbnail,border=1! !p-g2-raw.png|thumbnail,border=1!!p-1oIp.png|thumbnail,border=1!!p-g2-dyna.png|thumbnail,border=1! !p-g2-vs-tau.png|thumbnail,border=1!!p-g2-tau.png|thumbnail,border=1! h1. Code location Everything resides in {{CorAna}} package, which is included in LCLS offline releases beginning from {{ana-0.7.16}}. h1. Modules Updated on 2012-12-10 *Notations:* (/) - is done (at least it is assumed for now...) (+) - needs more work (i) - test or deprecated modules All module names beginning with letters {{GUI}} implements different Graphical User Interfaces. || Module || Description || | (/) AppDataPath.py | Local version with added path to data from src directory. | | (+) BatchJobCorAna.py | | | (+) BatchJobData.py | | | (/) BatchJobPedestals.py | Class contains methods for batch job submission and monitoring for pedestals. | | (+) BatchLogParser.py | | | (+) ConfigFileGenerator.py | Class has methods to generate psana configuration and other scripts from stubs located in {{CorAna/data/scripts/}}. | | (/) ConfigParameters.py | Base class for configuration parameters. | | (+) ConfigParametersCorAna.py | Sub-class for {{CorAna}} specific configuration parameters. | | (+) CorAnaUtils.py | Junk for eamples. | | (+) FileNameManager.py | Class dynamically generates all file names for current configuration parameters. | | (+) GUIAnaPartitions.py | | | (+) GUIAnaSettings.py | | | (+) GUIAnaSettingsLeft.py | | | (+) GUIAnaSettingsOptions.py | | | (+) GUIAnaSettingsRight.py | | | (+) GUIBeamZeroPars.py | | | (/) GUIBlamish.py | Sub-GUI of GUIFiles.py | | (+) GUICCDSettings.py | | | (+) GUIConfigParameters.py | GUI for the configuration parameters file management. | | (/) GUIDark.py | Sub-GUI of GUIFiles.py - manipulations with dark runs | | (/) GUIData.py | | | (/) GUIELogPostingDialog.py | Dialog window for submission of messages with attachments to ELog | | (/) GUIELogPostingFields | Widget with necessary fields for GUIELogPostingDialog | | (/) GUIFileBrowser.py | Text file browser for this project | | (/) GUIFiles.py | Central GUI for file settings contains tab-bar for other widget selection | | (/) GUIFlatField.py | Sub-GUI of GUIFiles.py | | (/) GUIHelp.py | WIdget for messages, i.e. help | | (+) GUIImgSizePosition.py | | | (+) GUIInstrExpRun.py | depricated GUI | | (/) GUIIntensityMonitors.py | Control GUI for intensity monitors | | (+) GUIKineticMode.py | | | (+) GUILoadResults.py | | | (/) GUILogger.py | GUI for logger | | (/) GUIMain.py | The first GUI at start of this application | | (/) GUIMainSplit.py | The same as GUIMainTB.py with integrated GUILogger.py | | (/) GUIMainTB.py | The first GUI at start of this application with tab bar | | (+) GUINonKineticMode.py | | | (+) GUIRun.py | | | (+) GUISetupEnergyAngle.py | | | (+) GUISetupInfo.py | | | (+) GUISetupInfoLeft.py | | | (+) GUISetupInfoRight.py | | | (+) GUISetupPars.py | | | (+) GUISpecularPars.py | | | (+) GUISystemSettings.py | | | (+) GUISystemSettingsLeft.py | | | (+) GUISystemSettingsRight.py | | | (+) GUITransmissionPars.py | | | (+) GUIViewResults.py | | | (/) GUIWorkResDirs.py | | | (+) GlobalUtils.py | Module contains all possible global methods. | | (i) ImgSpeNavToolBar.py | Re-implemented standard tool-bar (depricated) | | (/) Logger.py | Core class for logger | | (i) Overlay.py | Demonstration of how to draw something on the top of GUI | | (/) PlotArray.py | Main class for array presentation plot | | (/) PlotArrayButtons.py | Widget for custom button-bar | | (/) PlotArrayWidget.py | Widget for graphical window | | (/) PlotImgSpe.py | Main class for interactive plot with custom button bar | | (/) PlotImgSpeButtons.py | Widget for custom button-bar | | (/) PlotImgSpeWidget.py | Graphical image for image and spectral histogram | | (/) PlotTime.py | Main class for time record presentation plot | | (/) PlotTimeWidget.py | Widget for graphical window | | (+) data/scripts/psana-*.cfg | Stub-scripts for ConfigFileGenerator.py | h1. Status of the project *Updated on 2013-02-07* h2. PSANA modules h3. For dark run pre-processing *Scaner* Runs in batch for the dark file to get preliminary information. (/) {{ImgAlgos.ImgTimeStampList}} * counts number of events in the file * makes file with time stamps * evaluate time intervals between frames (for dark run) *Pedestals* Runs in batch for the dark file to get averaged pedestals. (/) {{ImgAlgos.PrincetonImageProducer}} gets image from event as an {{ndarray}} object (/) {{ImgAlgos.Tahometer}} evaluates performance of the batch job (/) {{ImgAlgos.ImgAverage}} produces file with averaged image for given range of events. h3. For data pre-processing *Scaner* Runs in batch for the data file to get preliminary information. (/) {{ImgAlgos.Tahometer}} evaluates performance of the batch job, counts number of events in the data file (/) {{ImgAlgos.ImgTimeStampList}} makes file with time stamps and time record counters for tau (/) {{ImgAlgos.IntensityMonitorsData}} makes file with intensity monitor records *Average* Runs in batch for the data file to get averaged image (/) {{ImgAlgos.Tahometer}} evaluates performance of the batch job (/) {{ImgAlgos.PrincetonImageProducer}} gets image from event as an {{ndarray}} object (/) {{ImgAlgos.ImgAverage}} produces file with averaged image for given range of events (-) {{ImgAlgos::ImgMaskEvaluation}} - module is available, but needs to be included in psana.cfg file for evaluation of: * saturated mask * noise mask * combined mask h3. For data processing *Split* (/) {{ImgAlgos.Tahometer}} (/) {{ImgAlgos.PrincetonImageProducer}} (/) {{ImgAlgos.ImgVsTimeSplitInFiles}} *Process* Data processing is implemented in stand alone (non-psana) modules (/) {{ImgAlgos.CorAna.cpp}} (/) {{ImgAlgos.CorAnaInputParameters.cpp}} (/) {{ImgAlgos.CorAnaData.cpp}} *Merge* (/) {{CorAnaMergeFiles.cpp}} - saves binary file for float(32) with shape (Ntau,3,rows,cols), where 3 stands for <Ip>, <If>, and <Ip*If> Can be acessed in python as {code} sp.cor_arr = np.fromfile(sp.fname, dtype=np.float32) <image-size> = rows * cols nptau = <file-size>/<image-size>/3 sp.cor_arr.shape = (nptau, 3, rows, cols) {code} h2. GUI The system of GUIs, consisting of dozens of {{ |
GUI
The system of GUIs, consisting of dozens of CorAna.GUI...
...
modules
...
is
...
implemented
...
in
...
the
...
draft
...
approximation.
...
Roughly
...
it
...
reproduces
...
all
...
features
...
of
...
the
...
old
...
program.
...
Input
...
parameters
...
GUIs
CorAna.GUI...
...
most
...
of
...
them
...
are
...
available.
...
Will
...
be
...
added
...
or
...
extended
...
if
...
necessary.
...
Files - define input files and do pre-processing
Setup Info
Analysis Info
System
Intensity Monitor
Run GUI
Input - short summary of input info for data processing
Split - control and monitoring for the 1st stage of processing
Process - ... 2nd stage ...
Merge - ... 3d stage ...
Auto - ... for all 3 stages ...
View Results GUI
Contains a set of control fields for presentation of results
Direct and reflected beam geometry is implemented in ViewResults.py
. Currently the switch between the direct and reflected beam geometry is used from tab status variable: cp.exp_setup_geom.value()
.
If the PV variable will be used, then the switch should be changed in ViewResults.get_q_map(sp)
module.
Graphics
PlotArray*.py
- for intensity monitors
PlotImgSpe*.py
- for images, partition maps, masks etc.
PlotTime*.py
- for time stamp monitoring
PlotG2*.py
- G2 plot
PlotGraph*.py
- I(q) and I(q,t) plot
MaskEditor*.py
, Drag*.py
- Mask Editor for
- region of interest (ROI-mask)
- blemish mask
ViewResults.pyQ
- all evaluations for resulting array of correlators - maps for pixel x,y coordinates, r, phi
- q maps for transmission and reflective geometry
- etc.
Infrastructural modules
Infrastructural modules provide basic infrastructure of the project.
ConfigParameters.py
, ConfigParametersCorAna.py
, and GUIConfigParameters.py
provides convenient approach for maintenance of all configuration parameters.
Infrastructural modules Logger.py
and GUILogger.py
provides a generic approach to logging system.
Module ConfigFileGenerator.py
use current settings of configuration parameters and stub-file scripts from CorAna/data/scripts/
and generates the psana configuration files.
Module FileNameManager.py
is a single place which provides a dynamic file names for current version of the configuration parameters.
GlobalUtils.py
- global utilities for common operations.
BatchJob.py
- superclass for batch job submission.
BatchJobPedestals.py
- pre-processing for dark run files.
BatchJobData.py
- re-processing for data files.
BatchJobCorAna.py
- main data processing - calculation algorithm.
RecordsFromFiles.py
- class helps to access data in files.
Status of the project
- This project is in the stage of comparison of results between old Marcin's scripts and this application.
Comparison of results
Conditions
- dark run:
/reg/d/ana12/xcs/xcsi0112/xtc/e167-r0020-s00-c00.xtc
use all 75 events - data run:
/reg/d/ana12/xcs/xcsi0112/xtc/e167-r0015-s00-c00.xtc
use all 500 events - do not use any intensity monitor selection or correction
- do not use any mask including hot pixel, saturation, blemish, ROI, and restriction on the image size.
- use LLD as a constant ADU threshold = 20
- use a single q-phi static bin
- use a single q-phi dynamic bin
- q value is not an issue for current comparison, so geometry does not matter.
Compare g2 at 14 tau values with indexes:
Code Block |
---|
1 2 3 4 5 6 7 8 9 10 12 14 16 20
|
See Progress section for more details on comparison.
Progress
Weekly progress of the IDPE project for TCE
To-do list
Processing (at psana Split level)
Get cp.photon_energy
, cp.nominal_angle
and other PV variables from data scan and setup confiruration
Comparison of results
- Search for the reason of difference in results for single-q-bin g2.
View Results
more plots for results
g2(tau) for q-dynamic
Average intensity and Intensity(q-static)
Intensity(q-static, t)
Histogram for intensity monitor
Fit results for function g2(tau|pars) = C*exp[-(2t/tau0)**beta] + B
- plot for all fits with beta=1 and float,
- tau0 vs q-dynamic
- C vs q-dynamic
- B vs q-dynamic
- beta vs q-dynamic
References
Photon Correlation Spectroscopy, article in wikipedia
Algorithms for Time Correlation Experiments
Note on XCS code from Marcin