Versions Compared

Old Version 9

changes.mady.by.user White, Glen

Saved on

compared with

New Version 127

changes.mady.by.user Zach Buschmann

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Accelerator Physics Software Work Tracking

Priorities 1,2,3 are high/medium/low respectively

Major Upgrades & Projects

TaskApplication/areaPerson(s) ResponsibleStatusPriorityNotesGitHub/CATER/etc
  •  Bmad live model
    •  BmadLiveModel 
    •  python PVA service
    •  GUI conversion
GlobalBuschmannWIP1

Lattice files for Bmad already exist. Current high-level goals are:

  • Establish agreement with Lucretia (S2E et al)
  • Implement interface to live machine parameters (Buschmann)
  • Add Bmad model to python PVA server
  • (TBD) Convert GUIs
    • Depends on tao→matlab compatibility
    • alternative: PyTao would let us rejoin the LCLS shared application ecosystem in some areas (emittance GUI, wires)
  • Update 08/06/24:  development of BmadLiveModel and server infrastructure is underway. 
    • long term plan is to migrate apps to python, in-line with AD LFD diagnostics group
    • python model interface makes sharing apps with LCLS easier
  • Update 08/22/2024: first real-time test successful, live model cold-start is ~3-15s, single-device update response is O(10ms)
  • Update 08/26/24: v0.1 non-stable released to production. Added "official" API documentation: https://f2-live-model.readthedocs.io/en/latest/
  • Update 08/29/24: PVA server tested successfully
  • Update 09/06/24: alpha BmadLiveModel & PVA server deployed to production, server running 24/7 for the time being to stress test
  • Update 09/29/24: Added LEM, split server into two processes
  • Update 10/9/24: heartbeats are setup, server is deployed 24/7

design page: Bmad live modeling

Github: https://github.com/slaclab/F2_live_model

  •  LEM server + watcher conversion
    •  server
    •  GUI
LEMBuschmannWIP2

Waiting on model server deployment

  • 3/13/24: current plan is to deploy a second PVA server that writes LEM EREF/EACT/BDES etc to an NTTTable, then the ModelReceiver can be updated in-kind to provide those quantities
    • python PVA will make it straightforward to build a LEM CUD
  • Update 08/06/24: LEM will run using the Bmad model
  • Update 09/10/24: prototyping, will run as a component of the BmadLiveModel
  • Update 09/29/24: server part is done, trim code WIP
  • Update 10/9/24: ready to test GUI with-beam


Full design notes on Bmad live model page

Github: https://github.com/slaclab/F2_LEM_GUI

  •  New longitudinal feedbacks
    •  DL10 energy
    •  BC11 energy
    •  BC11 bunch length
    •  BC14 energy
    •  BC14 bunch length
    •  BC20 energy
FeedbacksBuschmann
1
  • Current feedbacks regulate poorly and are not easily tuned/maintained
    • overall goal: improve overall beam stability, reduce feedback convergence times
  • LCLS-SC energy feedbacks are designed with off-the-shelf PID libraries, and written in python, plan is to implement the same
  • (from a controls perspective) this is straightforward since through BC11 everything is in EPICS
  • SLC device control can be accomplished through python PVA libraries, though this will take some research on RPC usage
  • deployment plan:
    • replace feedbacks one-by-one, i.e. run a new DL10 energy feedback, then use preexisting feedbacks downstream

  •  RTBSA for FACET-II
    •  python class for streaming BSA data
    •  python class for real-time BSA time series & correlation plots
    •  GUI
BSABuschmannWIP2
  • 07/05/2024: Developed proof-of-concept GUI using lcls BSA PVs - works for us as well since our BSA encodes pulse IDs the same way. Corr plot with fitting, time series and FFT are all working.
    • still need to optimize syncing for network performance
    • likely can replace the current RTBSA GUI for lcls as well
  • Update 10/9/24: some sync errors to debug, but basic functionality is OK
  • Update 10/11/24: released to prod, added to physics apps page (labeled as "experimental" for now)

  •  live dispersion GUI



1
  • LCLS dispersion live is, in principle, adaptable to this task. However the complication of needing to read both SLC buffer data and EPICS BSA buffers might make a new implementation simpler
  • Update 10/9/24: SLC buffacq is disruptive, need to strategize... EPICs-only live dispersion can be done simply w/ RTBSA tools

  •  New Phase scan GUI
    •  L0 phase scan automation
Phase ScansBuschmannDONE2

rudimentary functionality, needs with-beam test

update 05/06/24: tested with beam L0, L1 and L2-3 are working, deployment soon...

https://github.com/slaclab/facet-matlabTNG/pull/4
  •  Add quad scan emittance measurement function (for L0)
Optics MatchingPerez
3

last step needed to migrate off of the LCLS-side emittance application. seems to be already partially implemented, so the challenge here is partially archeological in nature

Update 09/29/24: should put this on hiatus pending migration back to LCLS app


  •  Lucretia model server conversion
    •  python PVA service
    •  matlab interface class
    •  GUI conversion
GlobalCesar, Perez, BuschmannPaused3

UPDATE: 08/06/24: Lucretia is being deprecated. The lack of Lucretia resources, both (written and in-person know how), and the poor maintainability of F2_LiveModelApp makes this a suboptimal time investment, especially since the desire was simply for apps to be more performant.

  • The F2_ModelReceiver is still usable as a matlab interface to the model server
  • Existing matlabTNG apps and the Lucretia model will remain in use, but with life-support only
  • physics apps will be ported to python over time (help wanted)
  • improved matlab app performance (from the control room) can be achieved with Glen's matlab server
  • 1/12/24: demo PVA server works, writing live/design Rs and twiss params, but matlab pvaGet not working yet. next step: document lucretia dependencies across matlabTNG to determine what the matlab interface needs to do
  • 1/23/24: matlab PVA access works with a python workaround (good enough for now, likely need a better long-term solution) Cesar prepped draft interface design document. reviewed high-priority GUI model uses. 
  • 2/8/24: PVA server is running but not deployed to prod yet. Drafting implementation of matlab class F2_ModelReceiver - once the lucretia functions GetRmats, RmatAtoB and GetTwiss are replicated we can proceed to test GUIs. Need to check w/ controls deputes to get some watcher PVs set up
  • 3/12/24: draft version of receiver on GitHub, testing for correctness and use by GUI controllers
  • 4/9/24: F2_ModelReceiver 0.1 version is ready for deployment, unfortunately the python models service has some errors in it, but the mechanics of the interface are mostly stable
  • 4/18/24: live model is currently starting from the gun, which produces very unrealistic twiss parameters. Need to start sim downstream at some treaty point
    • live model GUI gets marginally more realistic outputs starting from L0BFBEG, one option...
    • more realistic: take the latest PR10571 emittance measurement, back propagate to initial twiss at L0AFBEG – there are some kind of hooks in place for this already. but I'm not 100% on if they ever worked...
https://github.com/slaclab/facet-matlabTNG/pull/3

Minor Upgrades & Projects

IssueApplication/areaPerson(s) ResponsibleStatusPriorityNotesGitHub/CATER/etc
  •  LAME GUI scan automation
LAMEParkerWIP3

  •  Injector slice emittance tool

ParkerWIP


  •  Full lists of PVs used in HLAs accessible from each help menu
GlobalPerezWIP3F2_Feedbacks does this already with a hardcoded list, something similar could be done in other GUIs than have a lot of anonymous matlab PVs – ideally this could be automated by parsing the GUIs internal list of PVs
  •  Implement real-time rms energy spread PV with SYAG stats

Buschmann/Parker
2Camera is serving stats. Just need a way to measure a conversion factor of MeV/pixel, then write both the calibration and the sigmaE to matlab PVC
  •  update 2/28/24: need to better understand what AD plugin stats actually calculate, the quantity I thought was the Xrms is not
  • simpler solution: measure the MeV/mm calibration and make an MeV axis so the sigE can be eyeballed
  • update 08/06/24: this should go into a watcher process...

  •  Create tool to display corrector strength relative to beam energy vs Z.

Kalsi/ParkerWIP1

  •  python machine state summary script

Buschmann
3I'm imagining a script that gets 10571 + 10711 + SYAG + DTOTR2 images, and logbooks the most recent set of emittance/S20 measurements
  •  Linac bump maker

LoneyWIP1
  • simple version: matlab script that takes a BPM and an offset/angle as arguments
    • works, but depends on lucretia live (slow)
  • fancier version: GUI where one can select a BPM, preview corrector settings and undo

  •  convert wirescan plot Y axis to counts instead of charge
wirescansBuschmann
3

  •  IPWS1 Auto wire finder

LoneyWIP3simple: core plot config, preferred: script
  •  bunch length measurement with double-gaussian fitting

Buschmann
2

  •  orbit fitting script

Kalsi
2

  •  Rebrand EPICS with fancy new orange
GlobalBuschmanndone3nice
  •  CUD reference image/orbits
CUDsBuschmanndone3update 2/28/24: infrastructure exists to set new references remotely and update CUDs remotely, but the callbacks for each type of reference data are not complete – i.e. need to save DTOTR2 to a PNG, so CUDs can display, or save BPM data to a .mat etc etc
  •  Add Logbook feature to BSA GUI
BSAPerezDone2

  •  Add wire scan timestamps + skew + kurtosis to plot/logbook
Wire/Multi-WireBuschmann (+intrepid sidekick?)Done1

Both wire GUIs use the F2_WirescanApp object – save fit data/timestamps etc there, then GUIs can log & display that info as needed

  • update 3/5/2024: got the skew/kurt calculation implemented and have GUI frontends updated in-kind. still need to do some correctness checks and add timestamps

  •  FB CUD running flag (fix or remove)
F2 CUDsBuschmanndone3

update 4/9/24: working on updates to this display

5/24: done, v2 released


  •  Change default screen to 711 for injector TCAV measurements 
TCAV GUIRipmanDone2








Maintenance & Technical Debt

  •  tech debt to-do list goes here ...

Standby Tasks / Wishlist / Brainstorm

Generic/Global

  • Remove remaining dependencies on legacy Matlab2012 & "matlab model" code
  • Remove instances of direct use of LabCA - move everything over to using PV class in readiness for implementation of PVA
  • Beam stay clear plot vs Z (beam aperture vs Z, or acceptance vs delta E)
  • Automation/tools to facilitate 7-hole alignment 
  • laser heater GUI
  • MPS interlocking for EOS
  • Klystron jitter report → should get an automated FTP update script process thing
  • laser cathode auto-alignment software → meeting w/ LCLS injector AP to discuss methodology

LEM

  • ignore CQs/SQs when propagating twins parameters. (BMAG plot)
  • flag "matching quads" and handle separately when "Lem to model"
  • fix BACT=0 causing NaNs in BMAG propagation

Orbit

  • indicate TRIM operation in progress by disabling UI
  • Orbit fitting tools
  • Allow reference orbits to have names changed in Python Orbit Display Reference Orbit list

Optics

  • wire names don't update in multiwire matching tabs after changing linac selection
  • automatically de-select CQ/SQs from matching options
  • native quad scans → ultimate goal of fully migrating off of the LCLS emittance GUI

Feedbacks

  • tie DL10 set point directly to LLRF control (switch off "slow FB" on RF controller when FB on and vice-versa, increase gain)
  • Implement full PID controller coefficients to each feedback, setup and test and include P or PID as option (at least for DL10)
  • Implement use of LI17 fast phase shifters for BC20 feedback through SCP multiknob controls
  • Auto-adjust gains for changes in beam rate
  • Add FFS transverse feedback

MDL Feedforward

Wirescans

  • Wire scanner GUI not set wire speed to such a low speed after a scan that the following scan fails if you change planes for IPWS1. 

S20 Config

Orphaned/Unknown Tasks

These are software tasks that are in an unknown state or not currently being worked

TaskLast Person ResponsibleStatus
Implement interface to SCP corr plotsGeorge, Glen10/22/22: Implemented by George, documentation on AIDA-PVA website, needs testing
Feedback HSTA bit control (be able to change Feedback → Compute)George, Glen9/22/22: Implemented and tested ability to write raw HSTA bit values, matlab interface to toggle OFF, Compute, Feedback states written and tested. George to implement cleaner AIDA-level state change interface.
Fix AIDA service crashes on VAX (or be able to detect and self correct)George, Greg9/22/22: Many potential non catching of errors found by George and changes to server-side code made, cannot induce server failures by testing. Need to operationally observe to see if fixes are good.





...

Software Development Workflow

All production software must be under some form of version control. In general: code with a larger audience of users or that is a significant dependency of downstream software should be managed more carefully.

Master repositories can be found here: /afs/slac/g/cd/swe/git/repos/slac/FACET/

"Production" HLAs live in a number of location, mainly $TOOLS/python and $TOOLS/matlabTNG

These instructions are written assuming some preexisting knowledge of version control & some basic git knowledge. If you have questions ask Zack.

How to create a new repo

  1. navigate to the /afs/ prod directory and mkdir <repo_name>.git
  2. run git init --bare to instantiate a new empty repo
  3. (if using GitHub) make a new GitHub repo with the same name
  4. clone the master repo into a work directory (can be anywhere, /afs/-space, DMZ or prod): git clone ssh:///afs/slac/g/cd/swe/git/repos/slac/FACET/<repo_name>.git
  5. do work, then git commit & git push
  6. (after implementation/deployment) clone the master repo into a production repository

Once you create a new repo or clone one, there are two workflows: one simple command-line only way to use git, or also using the SLAC GitHub to enable some more formal code review processes.

Repo mirroring with GitHub:

GitHub is hosted on the public internet. In order to store production software there, we need to use a "relay" repo that lives on the DMZ network to facilitate pushing/pulling changes between GitHub and the local repo on the SLAC network. 

  1. ssh to centos7, navigate to /u/gu/zack/github_relay/
  2. run python relay_setup.py <prod_repo_name> <github_repo_name> to setup a "relay repository" used to sync /afs/ and GitHub. This script will:
    1. git clone the repo into the github_relay directory
    2. git remote add github git@github.com:slaclab/<github_repo_name>.git
    3. add <repo_name> to /u/gu/zack/github_relay/tracked_repos.txt

Direct push to master (for minor changes or lower-impact software):

  1. from your work directory: git commit, git push origin master
  2. from the production directory, and git pull origin master

Feature-branches with GitHub (for major changes or high-impact software):

This process will be automated in future, but for now relies manually using the script: /u/gu/zack/github_relay/sync_github.py

  1. Setup relay mirroring to GitHub as described above
  2. from your work directory for the repo in question, make a new branch: git branch <branch_name>
  3. make changes, commit them to <branch_name> and push the branch to /afs/
  4. use sync_github.py to push the branch to GitHub
  5. open a pull request, assign reviewers, review code & merge to master
  6. use sync_github.py to pull the updated master branch from GitHub back to /afs/
  7. deploy to production

To deploy software to production

  1. run git pull origin master in the production repository

Using sync_github.py

  • to sync prod → GitHub: $ python sync_github.py push <repo name> <(optional) branch name>
  • to sync GitHub → prod: $ python sync_github.py pull <repo name> <(optional) branch name>

...

Legacy Task Tracking

These are the HLA tasks as they existed as of 01/2024. This content is archival.

Preferred development workflow with git repository:

  • Create new development branch for work in personal clone of repo
  • Do work and test
  • Merge in any recent changes from main branch
  • Push your branch, email Glen to evaluate & merge into main branch

Generic tasks/ideas

TaskStatusAssigned ByActively Worked on by...
Full lists of PVs used in HLAs accessible from each help menu
GlenSharon
Complete documentation for each HLA & generic controls/modeling interface
Glen
Remove remaining dependencies on legacy Matlab2012 & "matlab model" code
Glen
Remove instances of direct use of LabCA - move everything over to using PV class in readiness for implementation of PVANeeds doing by mid-2023?Glen
Faster list-based caget operations in PV class + improve first-time calls (speed up LiveModel initialization)
Glen
Implement asyn operations when move to >Matlab2021
Glen
Extend matlab server model to facet-srv01?
Glen
Change default range of Schottky Scan GUI
LoneySharon
Automation of 10-3/10-4 phase scans (in Schottky GUI or phase scan GUI)To be implemented into new phase scan GUILoneyBuschmann
Add Logbook feature to BSA GUI
LoneySharon

AIDA-PVA

TaskStatusAssigned byActively Worked on by...
Larger # buffered BPM data acquisitions (up to 1000 @ 10Hz?)11/7/22: Tested, deployed in Wirescan app- appears to be working well.GlenCOMPLETE
Asynchronous calls (especially for bufferedacq) from Matlab11/4/22: Tested, declared working.GlenCOMPLETE
Implement interface to SCP corr plots10/22/22: Implemented by George, documentation on AIDA-PVA website, needs testingGlenGeorge, Ability to initiate fast phase dither scans and pull back buffered data + pulse ID's Glen
Multi-device set (and asynchronous status return) on PMDL (and others...?)GlenAutomate ENLD calculation from AIDA call (generic ability to initiate button macros?)11/7/22: Tested, working well in MDLFF app for multi-set of PMDL values.GlenCOMPLETEGlen
Feedback HSTA bit control (be able to change Feedback → Compute)Priority 9/22/22: Implemented and tested ability to write raw HSTA bit values, matlab interface to toggle OFF, Compute, Feedback states written and tested. George to implement cleaner AIDA-level state change interface.GlenGeorge, Glen
Fix AIDA service crashes on VAX (or be able to detect and self correct)Priority 9/22/22: Many potential non catching of errors found by George and changes to server-side code made, cannot induce server failures by testing. Need to operationally observe to see if fixes are good.GlenGeorge & Greg

F2_Orbit

TaskStatusAssigned ByActively Worked on by...
Plotted orbit doesn't respect pre-selected BPM list when loading a Config with pre-measured BPM data
Glen
Indicate TRIM operation in progress by grey-out of "Do correction" button or similar
Glen
Make MIA tab functionality work
Glen
"Plot all" option doesn't work correctly when displaying corrected orbit after performing orbit correction calc
Glen
Re-instate some auto dispersion correct functionality (e.g. DL10 correction with Q10731)
Glen
Add kick and dispersion source fit functionality to orbit and dispersion tabs
Glen
Changing fit location should update plot and fit data- currently doesn't work
Glen
Link reference orbit saved to Python bpm orbit tool reference saves
Glen
Implement TMIT cuts for BPM orbit
Glen
Implement measured Rmat (as opposed to model Rmat) interface (including measurement tools?)
Glen

F2_LEM

TaskStatusAssigned ByActively Worked on by...
Store design L1,L2,L3 operating phases in EPICS PVs
Glen
Watcher version of LEM: info on when LEM needed etc + interface with CUD
Glen
Ignore CQ's & SQ's for purposes of Twiss parameter propagation (BMAG plot)
Glen
Make a way to flag "Matching Quads" and deal with them differently when "LEM to model"?
Glen
Fix BACT=0 causing "NaN"'s in BMAG propogation
Glen
Convert to server + watcher architecture
BuschmannBuschmann

F2_Matching

TaskStatusAssigned ByActively Worked on by...
On Multi-Wire matching tabs, wire names don't change when reading in data from different Linac section
Glen
Keep last vals in MW tab when switch back from other tab
Glen
Auto de-select CQ & SQ magnets from matching options
Glen
Add native quad scan functionality
Glen
Allow Undo button to retain old values after calculating new match
Loney

F2_Feedback

TaskStatusAssigned ByActively Worked on by...
Jitter button ON/OFF status not working properly
Glen
DL10 controller name "IN10"→"LI10" on GUI
Glen
Settings GUI plot to include line showing feedback desired offset value
Glen
Fix freezing of GUI when closing settings panels
Glen
Tie DL10 feedback stpoint directly to LLRF control → switch off "slow fb" on RF controller when feedback on and vice-versa, increase feedback gain
Glen
Implement full PID controller coefficients to each feedback, setup and test and include P or PID as option (at least for DL10)
Glen
Implement use of LI17 fast phase shifters for BC20 feedback through SCP multiknob controls
Glen
Auto-adjust gains for changes in beam rate
Glen
Add FFS transverse feedback
GlenLauren




MDL_FeedForward

TaskStatusAssigned byActively Worked on by...
Data collection: have watcher record each GOLD entry and associated data and save to disk
Glen
Make user-facing configuration GUI to examine training data & evaluate performance & tune NN or linear models or manually tweak
Glen
Implement multi-device write commands in AIDA for PMDL data

10/7/22: AIDA software written to allow this
11/7/22: Implemented in MDLFF app and tested.

GlenCOMPLETE

F2_Wirescanner

TaskStatusAssigned byActively Worked on by...
User access to PMT timing & gate widths
Glen
When selecting motor position units option- propagate changes to plot to see centroid of scan in motor units
Glen
Jitter correction for Linac wirescanners in L2 and L3

9/19/22: Initial tests show buffered bpm acq with ~200 pulses works, initial code implemented, needs testing with beam
10/7/22: AIDA now supports up to 1800 buffered data points
11/7/22: Debugged jitter correction code with AIDA BPM in app, test worked well- all beam size measurements in L2 & L3 reduce when jitter correction selected.

GlenCOMPLETE

S20 Configurator

TaskStatusAssigned byActively Worked on by...
Write current IP waist position to a dedicated PV instead of sharing with desired waist position 
RyanBuschmann

S20 IP Waist from BPM Jitter

TaskStatusAssigned byActively Worked on by...
Create11/6/22: Tested, working. Results are unstable, assuming this is due to dispersion leakage and large energy jitter. Next job is to implement SVD algorithm to simultaneously fit dispersion functions and remove energy jitter component.GlenGlen


SLC Control System Notes

Ken Brobeck's How to restart some VMS processes guide.