You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Weekly progress of the Integrated Data Processing Environment (IDPE) for Time Correlation Experiments (TCE)

2012-11-05 – 2012-11-11 Week progress

A src/PsanaConfigFileGenerator.py
A src/FileNameManager.py
A src/BatchJobPedestals.py
A src/AppDataPath.py
A src/ConfigFileGenerator.py
A src/GUIDark.py
A src/ImgSpeButtons.py
A src/ImgSpeNavToolBar.py
A src/ImgSpeWidget.py
A src/ImgSpeWithGUI.py
A data/scripts/psana-pedestals.cfg
A data/scripts/psana-tahometer.cfg

2012-11-12 – 2012-11-18 Week progress

A src/GUIFileBrowser.py - file browser for text files of this project.
A src/GUIHelp.py - pop-up box with message as a widget.
A src/GUIWorkResDirs.py - separate widget for setting of work and result output directory and common prefix file name.

In src/GUILogger.py and src/Logger.py - now messages are accumulated in 2d tuple. Add filtration by the message type "debug", "info", "warning", etc.

src/GUIFiles.py is significantly re-organized; now it is GUI with tabs.
All significant fields are moved in GUIs like src/GUIDark.py (GUIData, GUIFlat).

src/GUIDark.py - is re-organized.

In the interactive plot:

  • In src/ImgSpeWithGUI.py the src/ImgSpeNavToolBar.py - is depricated
  • src/ImgSpeButtons.py - re-organized: add save, reset, and menu buttons, log check-box.
  • In src/ImgSpeWidget.py - the log functionality needs to be implemented...

2012-11-19 – 2012-11-25 Week progress

Suggestions from Marcin:

  • Time(sec) - should be the averaged time between frame time-stamps.
  • Spectral plot: log-scale for X-axis only.
  • Flat field and Blamish files will be provided from external apps. No need to evaluate them now. Needs to be selected by the path.
  • Substitute the start and end events after the scan.

Progress:

  • Restracturization: tahometer replaced by peds_scan.
  • update the start and end fields after scan.
  • remove psana log files before job submission in batch.
  • the start and end fields are set from cofig.pars, if the scan log-file is availabe, they are updated from this file.
  • reset fields to default if the input xtc file is changed.
  • A src/GUIFlatField.py - file selection, plot, browse file
  • A src/GUIBlamish.py
  • A src/GUIMainTB.py - try different view for composite window
  • A src/GUIMainSplit.py - try different view for composite window
  • Add psana module: ImgAlgos::ImgTimeStampList - creates the time-stamp list and evaluates the mean time between frames and its RMS.
  • in GUIDark.py use variables produced by the ImgAlgos::ImgTimeStampList.

2012-11-26 – 2012-12-02 Week progress

Suggestions from Marcin:

  • Marcin prefers layout with separated GUILogger window, like in GUIMainTB.py
  • Need in a tool reporting problems in time-stamp sequence.
  • In GUIDark.py - untill the total number of events is unknown, use end "-1" and generate a warning if it is submitted in batch for pedestals.
  • Add check-box showing if the Blamish and Flat-field files will be applied
  • Marcin returned the paper-copy of GUIs with comments about titles and parameters.
  • The list of variables for intensity monitor:

    • IPM2 [CH0,...,CH3]

    • IPMMON0 [...]

    • IPM4 [...]

    • IPM5 [...]

    • GASDET [...]

  • Needs in separate GUI for intensity monitors for normalization and veto.

Progress:

  • In GUIBlamish.py and src/GUIFlatField.py add check-boxes.
  • Apply Marcin's comments to GUIs with System, Setup etc. parameters, GUICCDCorrectionSettings.py is removed..
  • In src/GUIMainTB.py optimized the GUI sizes.
  • Changed names from GUIBatch... to GUISystem...
  • New psana module Psana Module Catalog, which gets data for equivalent HDF5 variables: 
    /Configure:0000/Run:0000/CalibCycle:0000/Bld::BldDataFEEGasDetEnergy/FEEGasDetEnergy
/Configure:0000/Run:0000/CalibCycle:0000/Bld::BldDataIpimbV1/XCS-IPM-02
/Configure:0000/Run:0000/CalibCycle:0000/Bld::BldDataIpimbV1/XCS-IPM-mono
/Configure:0000/Run:0000/CalibCycle:0000/Ipimb::DataV2/XcsBeamline.1:Ipimb.4
/Configure:0000/Run:0000/CalibCycle:0000/Ipimb::DataV2/XcsBeamline.1:Ipimb.5
    in psana they can be accessed as: 
      m_srcFEEGasDetE = configStr("feeSource", "BldInfo(FEEGasDetEnergy)");
  m_srcIPM2       = configStr("ipm2",      "BldInfo(XCS-IPM-02)");
  m_srcIPMMono    = configStr("ipmmono",   "BldInfo(XCS-IPM-mono)");
  m_srcIPM4       = configStr("ipm4",      "DetInfo(XcsBeamline.1:Ipimb.4)");
  m_srcIPM5       = configStr("ipm5",      "DetInfo(XcsBeamline.1:Ipimb.5)");
    and share pointers: 
      shared_ptr<Psana::Bld::BldDataFEEGasDetEnergy> fee = evt.get(m_srcFEEGasDetE);
  shared_ptr<Psana::Ipimb::DataV2> data2 = evt.get(m_src); // for m_srcIPM4 and m_srcIPM5
  shared_ptr<Psana::Bld::BldDataIpimbV1> ipimb1 = evt.get(m_src_ipimb); // for m_srcIPM2 and m_srcIPMMono
  • In GUISystemPars: add total number of events and dt+-rms to GUISystemPars.
  • In GUIDark: add check-box and file-browser button with complete functionality.
  • In GUIDark: add protection for pedestal job submission in batch with default number of events.
  • In BatchJobPedestals: add protection against duplicated job submission in batch using timeout (now 100 sec) or until the previous job is DONE.
  • In ImgSpeWidget: implemented the log-scale plot with full functionality and protections...

2012-12-03 – 2012-12-09 Week progress

No meeting this week due to experiments in xcs hatch.

Progress:

  • In GUIFileBrowser: add one more parameters after the list of files, the file to start browser
  • GUIData is implemented with full functionality including
    • check box to activate/protect buttons
    • "File" button and info field - selects the xtc file
    • "Scan" button - submits scanner job in batch which
      1. counts events,
      2. saves in file the time records ind evaluate an average interval between frames
      3. saves in file the intensity monitor data
    • "Average" button - submits job in batch for averaging image for the specified range of events
    • "Check status" button - checks the status of both batch jobs, check if the file with averaged image is available, update the info fields for number of events and averaged time between frames
    • "Check files" button - prints the list of work files for data and check their availability
    • "Browse" button - open the file browser with a list of work files for data
    • "img-Plot" button - plots the averaged image from file
    • "t-Plot" button - shows the plot for time records saved in file
    • "Remove" button - removes all work files from the work directory
  • GUIIntensityMonitors: GUI for Intensity monitors
  • PlotArray.py, PlotArrayButtons.py, PlotArrayWidget.py - plot for array presentation
  • PlotTime.py, PlotTimeWidget.py - plot for time records presentation

2012-12-10 – 2012-12-16 Week progress

Q&A Marcin:

  • Q: XcsBeamline.2 - is it used simultaneously with XcsBeamline.1 ?
    • A: XcsBeamline.1
  • Q: What control parameters should be added for intensity monitors?
    • A: should be added:
      1. radio button for normalization or no-normalization
      2. filter data check box
      3. low-upper amplitude range
  • Q: Layout of GUIMainTB.py with variable height, is it ok?
    • A: Looks fine.
  • Q: Layout variation for check-box - disappearance of buttons, is it ok?
    • A: Change color of button-titles.
  • Q: What image buttons are used as a mask editor etc.
    • A: GUIAnalysisSettings:
      • "MaskPolygon" button - for polygon editor on data image
      • "Show Mask & Partitions" button - shows the mask and partitions on data image

Suggestions from Marcin:

  • (plus) Add button to all plots and gui to submit them in ELog.
  • (tick) Change log presentation for image histogram - log for vertical scale only.

Progress

  • (tick) GUIDark and GUIData - inappropriate change of style for "end" field is fixed
  • (tick) GUIDark, GUIData, GUIBlamish, GUIFlatField - have changed behavior for non-active buttons:
    • for now it is set to default - frame of buttons disappeared for non-active
    • GUIFlatField has an example of how to set a dynamic style-sheet for non-default behavior styles
  • (tick) PlotImgSpeWidget.py - change log scale from x to y
  • (tick) GUIIntensityMonitors.py - add more fields:
    • radio buttons for normalization
    • check boxes for selection
    • edit fields for Imin and Imax values with changing behavior and style depending on radio and check box status. Check them for equity to default and Imin >= Imax
  • (tick) Check move window coordinates for plots... Some of them opens in unexpected places.
  • (tick) GUIELogPostingDialog, GUIELogPostingFields, PlotArrayButtons, PlotImgSpeButtons, and GlobalUtils: submission of message with attached plots in ELog
  • (tick) Add icons to some of frequently used buttons.
  • (tick) Add button for some of GUIs amd plots to submit them in ELog.

2012-12-17 – 2012-12-23 Week progress

Q&A Marcin:

  • Discuss layout of fields in
    • GUIIntensityMonitors - Is it necessary to use Imax/Imin for normalization?
    • GUIELogPostingDialog, GUIELogPostingFields - is it consistent with what you want?
  • Layout of buttons
    • GUIMainTB - it is possible to change the "buttons with icons and text" -> "buttons with icons" -> "icons". Which style is preferred?
    • GUIDark.py, GUIData.py, etc. - style for active/non-active buttons - just default or add more colors, like in GUIFlatField ?

Progress

  • GUIDark and GUIData - add check box for parameters cp.use_dark_xtc_all and cp.use_data_xtc_all with appropriate behavior for check box but_path and edi_path fields. Now, the name of the file for batch job is taken from fnm.path_dark_xtc_cond() and fnm.path_data_xtc_cond(), respectively.
  • FileNameManager - fixed a few typos in the file name methods: "dark"->"data"

2012-12-24 – 2013-01-06 Week progress (winter break)

Progress

  • GUIRun, GUIRunInfo, GUIRunSplit - new style GUI for the splitting procedure.

2013-01-07 – 2013-01-13 Week progress

Progress

  • GUIRunProc, BatchJobCorAna, GlobalUtils

2013-01-14 – 2013-01-20 Week progress

Q&A Marcin:
(tick) Add lock for ELog submission dialog box
(tick) Add lock for nparts field in GUIRunInput
(tick) GUIIntensityMonitors:

  • change response on radio button
  • short intensity monitor names

Progress

  • GUIRunInfo change name to GUIRunInput
  • GUIRunProc - is completed
  • GUIRunMerge
  • GUIRunAuto
  • GUIELogPostingDialog, GUIELogPostingFields - add lock, restructorization for editable fields.
  • GUIIntensityMonitors - (1) remove the selection fields style change for radio button, (2) use a list of short intensity monitor names.

2013-01-21 – 2013-02-03 Week progress

Q&A Marcin:
(tick) Reorganize GUISetupInfoLeft.py for CCD position input in modes: Beam Zero, Specular, and Data

Progress
(tick) Fixed image partitioning, processing, and merging in ImgAlgos/src/ImgVsTimeSplitInFiles.cpp, CorAnaData.cpp, and CorAnaMergeFiles.cpp
(tick) Add CCD orientation in GUICCDSettings

  • CCD is always displayed as a matrix of (row,col) with origin in the top left corner.
  • CCD orientation is assumed as entire world is rotated w.r.t. CCD

Work on

  • GUIViewControl.py
  • GUIViewResults.py
  • ViewResults.py
  • g2 Normalization - need to use parameters from GUISetupInfo etc.
    Map evaluation speed in ViewResults.py for 1300x1340 img
  • x,y maps = 0.02sec
  • r map = 0.05sec
  • theta map = 0.12sec
  • q map = 0.4sec

2013-02-04 – 2013-02-10 Week progress

Progress
Work in progress on

  • GUIViewControl.py
  • ViewResults.py
  • PlotG2*.py

(tick) Currently available plots for:

  • Raw data maps (tau): <Ip>, <If>, <Ip*If>, g2-raw
  • Geometry maps: X, Y, R, Phi, Q for direct beam mode
  • Partitioning maps: Phi, Q, Q-Phi maps for static and dynamic partitions
  • Masks: Image limits, blemish, hot pixels, good regions, total
  • Calculated maps (tau): total mask as applied at normalization and for maps
    • normalization factors 1/<Ip>, 1/<If> for static bins,
    • g2 as a map
    • g2 averaged over dynamic bins
    • g2 for dynamic bins vs tau as a 2D histogram
    • g2(tau) plot for dynamic bins as a set of graphic

Q to Marcin:

  • How to calculate angle and Q in reflective geometry?
  • Presentation of CCD image and detector orientation issues.

2013-02-11 – 2013-02-17 Week progress

Progress
Work on mask editor application:

  • All Drag*.py
  • MaskEditor.py, MaskEditorButtons.py
  • No labels