Accelerator Physics Software Work Tracking

Priorities 1,2,3 are high/medium/low respectively

Major Upgrades & Projects

TaskApplication/areaPerson(s) ResponsibleStatusPriorityNotesGitHub/CATER/etc
  • Lucretia model server conversion
    • python PVA service
    • matlab interface class
    • GUI conversion
GlobalCesar, Perez, BuschmannWIP1
  • 1/12/24: demo PVA server works, writing live/design Rs and twiss params, but matlab pvaGet not working yet. next step: document lucretia dependencies across matlabTNG to determine what the matlab interface needs to do
  • 1/23/24: matlab PVA access works with a python workaround (good enough for now, likely need a better long-term solution) Cesar prepped draft interface design document. reviewed high-priority GUI model uses. 
  • 2/8/24: PVA server is running but not deployed to prod yet. Drafting implementation of matlab class F2_ModelReceiver - once the lucretia functions GetRmats, RmatAtoB and GetTwiss are replicated we can proceed to test GUIs. Need to check w/ controls deputes to get some watcher PVs set up
  • 3/12/24: draft version of receiver on GitHub, testing for correctness and use by GUI controllers
  • 4/9/24: F2_ModelReceiver 0.1 version is ready for deployment, unfortunately the python models service has some errors in it, but the mechanics of the interface are mostly stable
  • 4/18/24: live model is currently starting from the gun, which produces very unrealistic twiss parameters. Need to start sim downstream at some treaty point
    • live model GUI gets marginally more realistic outputs starting from L0BFBEG, one option...
    • more realistic: take the latest PR10571 emittance measurement, back propagate to initial twiss at L0AFBEG – there are some kind of hooks in place for this already. but I'm not 100% on if they ever worked...
https://github.com/slaclab/facet-matlabTNG/pull/3
  • LEM server + watcher conversion
LEMBuschmann
2

Waiting on model server deployment

  • 3/13/24: current plan is to deploy a second PVA server that writes LEM EREF/EACT/BDES etc to an NTTTable, then the ModelReceiver can be updated in-kind to provide those quantities
    • python PVA will make it straightforward to build a LEM CUD



  • New Phase scan GUI
    • L0 phase scan automation
Phase ScansBuschmannWIP2

rudimentary functionality, needs with-beam test

update 05/06/24: tested with beam L0, L1 and L2-3 are working, deployment soon...

https://github.com/slaclab/facet-matlabTNG/pull/4
  • Add quad scan emittance measurement function (for L0)
Optics MatchingPerez
3last step needed to migrate off of the LCLS-side emittance application. seems to be already partially implemented, so the challenge here is partially archeological in nature.







Maintenance & Technical Debt

IssueApplication/areaPerson(s) ResponsibleStatusPriorityNotesGitHub/CATER/etc
  • BC20 energy feedback no-op
FeedbacksBuschmannWIP1MD plan for debugging is ready to go166749
  • Matlab server revival
GlobalBuschmannWIP2

managed to launch server, but haven't figured out how to launch GUIs yet

update 4/24: done. added a button to launch from the ops dashboard – only a few OPIs supported at the moment, but adding support from new machines isn't hard


  • Save previous magnet settings after loading new emittance msmt
OpticsBuschmann
2
166714
  • Add logbook button for LEM magnet settings
LEMBuschmann
2
166618
  • Write waist position PV after trim, not after selection
S20 ConfigBuschmannDone3

  • Sextupole GUI "fix offsets" button does nothing
SextupoleBuschmann
3ancient CATER - maybe not an issue anymore? needs discussion117540
  • Change default range of Schottky Scan GUI
Schottky ScanPerezDone3changed to  [ -50 10 ]
  • Feedbacks GUI has a typo in DL10 PV
FeedbacksBuschmannDone3trivial fix157790
  • fix FC01 toggle on Schottky GUI
Schottky ScanLoney
3(should be) trivial fix, also change default range and add gun offset167704







Minor Upgrades & Projects

IssueApplication/areaPerson(s) ResponsibleStatusPriorityNotesGitHub/CATER/etc
  • Rebrand EPICS with fancy new orange
GlobalBuschmannWIP3discussing how to implement non-invasively with EED
  • LAME GUI scan automation
LAMEParkerWIP3

  • CUD reference image/orbits
CUDsBuschmann
3update 2/28/24: infrastructure exists to set new references remotely and update CUDs remotely, but the callbacks for each type of reference data are not complete – i.e. need to save DTOTR2 to a PNG, so CUDs can display, or save BPM data to a .mat etc etc
  • Full lists of PVs used in HLAs accessible from each help menu
GlobalPerezWIP3F2_Feedbacks does this already with a hardcoded list, something similar could be done in other GUIs than have a lot of anonymous matlab PVs – ideally this could be automated by parsing the GUIs internal list of PVs
  • Add Logbook feature to BSA GUI
BSAPerezDone2

  • Add wire scan timestamps + skew + kurtosis to plot/logbook
Wire/Multi-WireBuschmann (+intrepid sidekick?)Done1

Both wire GUIs use the F2_WirescanApp object – save fit data/timestamps etc there, then GUIs can log & display that info as needed

  • update 3/5/2024: got the skew/kurt calculation implemented and have GUI frontends updated in-kind. still need to do some correctness checks and add timestamps

  • Implement real-time rms energy spread PV with SYAG stats

Buschmann/ParkerWIP2Camera is serving stats. Just need a way to measure a conversion factor of MeV/pixel, then write both the calibration and the sigmaE to matlab PVC
  •  update 2/28/24: need to better understand what AD plugin stats actually calculate, the quantity I thought was the Xrms is not
  • simpler solution: measure the MeV/mm calibration and make an MeV axis so the sigE can be eyeballed

  • Create tool to display corrector strength relative to beam energy vs Z.

KalsiWIP1

  • FB CUD running flag (fix or remove)
F2 CUDsBuschmann
3

update 4/9/24: working on updates to this display

5/24: done, v2 released


  • python machine state summary script

Buschmann
3I'm imagining a script that gets 10571 + 10711 + SYAG + DTOTR2 images, and logbooks the most recent set of emittance/S20 measurements
  • linac bump maker

LoneyWIP1
  • simple version: matlab script that takes a BPM and an offset/angle as arguments
  • fancier version: GUI where one can select a BPM, preview corrector settings and undo

Standby Tasks / Wishlist / Brainstorm

Generic/Global

  • Remove remaining dependencies on legacy Matlab2012 & "matlab model" code
  • Remove instances of direct use of LabCA - move everything over to using PV class in readiness for implementation of PVA
  • Beam stay clear plot vs Z (beam aperture vs Z, or acceptance vs delta E)

LEM

  • write L1-L3 design phases to PVs
  • ignore CQs/SQs when propagating twins parameters. (BMAG plot)
  • flag "matching quads" and handle separately when "Lem to model"
  • fix BACT=0 causing NaNs in BMAG propagation

Orbit

  • indicate TRIM operation in progress by disabling UI

Optics

  • wire names don't update in multiwire matching tabs after changing linac selection
  • automatically de-select CQ/SQs from matching options
  • native quad scans → ultimate goal of fully migrating off of the LCLS emittance GUI

Feedbacks

  • tie DL10 set point directly to LLRF control (switch off "slow FB" on RF controller when FB on and vice-versa, increase gain)
  • Implement full PID controller coefficients to each feedback, setup and test and include P or PID as option (at least for DL10)
  • Implement use of LI17 fast phase shifters for BC20 feedback through SCP multiknob controls
  • Auto-adjust gains for changes in beam rate
  • Add FFS transverse feedback

MDL Feedforward

Wirescans

  • Wire scanner GUI not set wire speed to such a low speed after a scan that the following scan fails if you change planes for IPWS1. 

S20 Config

Orphaned/Unknown Tasks

These are software tasks that are in an unknown state or not currently being worked

TaskLast Person ResponsibleStatus
Implement interface to SCP corr plotsGeorge, Glen10/22/22: Implemented by George, documentation on AIDA-PVA website, needs testing
Feedback HSTA bit control (be able to change Feedback → Compute)George, Glen9/22/22: Implemented and tested ability to write raw HSTA bit values, matlab interface to toggle OFF, Compute, Feedback states written and tested. George to implement cleaner AIDA-level state change interface.
Fix AIDA service crashes on VAX (or be able to detect and self correct)George, Greg9/22/22: Many potential non catching of errors found by George and changes to server-side code made, cannot induce server failures by testing. Need to operationally observe to see if fixes are good.






Software Development Workflow

All production software must be under some form of version control. In general: code with a larger audience of users or that is a significant dependency of downstream software should be managed more carefully.

Master repositories can be found here: /afs/slac/g/cd/swe/git/repos/slac/FACET/

"Production" HLAs live in a number of location, mainly $TOOLS/python and $TOOLS/matlabTNG

These instructions are written assuming some preexisting knowledge of version control & some basic git knowledge. If you have questions ask Zack.

How to create a new repo

  1. navigate to the /afs/ prod directory and mkdir <repo_name>.git
  2. run git init --bare to instantiate a new empty repo
  3. (if using GitHub) make a new GitHub repo with the same name
  4. clone the master repo into a work directory (can be anywhere, /afs/-space, DMZ or prod): git clone ssh:///afs/slac/g/cd/swe/git/repos/slac/FACET/<repo_name>.git
  5. do work, then git commitgit push
  6. (after implementation/deployment) clone the master repo into a production repository

Once you create a new repo or clone one, there are two workflows: one simple command-line only way to use git, or also using the SLAC GitHub to enable some more formal code review processes.

Repo mirroring with GitHub:

GitHub is hosted on the public internet. In order to store production software there, we need to use a "relay" repo that lives on the DMZ network to facilitate pushing/pulling changes between GitHub and the local repo on the SLAC network. 

  1. ssh to centos7, navigate to /u/gu/zack/github_relay/
  2. run python relay_setup.py <prod_repo_name> <github_repo_name> to setup a "relay repository" used to sync /afs/ and GitHub. This script will:
    1. git clone the repo into the github_relay directory
    2. git remote add github git@github.com:slaclab/<github_repo_name>.git
    3. add <repo_name> to /u/gu/zack/github_relay/tracked_repos.txt

Direct push to master (for minor changes or lower-impact software):

  1. from your work directory: git commit, git push origin master
  2. from the production directory, and git pull origin master

Feature-branches with GitHub (for major changes or high-impact software):

This process will be automated in future, but for now relies manually using the script: /u/gu/zack/github_relay/sync_github.py

  1. Setup relay mirroring to GitHub as described above
  2. from your work directory for the repo in question, make a new branch: git branch <branch_name>
  3. make changes, commit them to <branch_name> and push the branch to /afs/
  4. use sync_github.py to push the branch to GitHub
  5. open a pull request, assign reviewers, review code & merge to master
  6. use sync_github.py to pull the updated master branch from GitHub back to /afs/
  7. deploy to production

To deploy software to production

  1. run git pull origin master in the production repository

Using sync_github.py

  • to sync prod → GitHub: $ python sync_github.py push <repo name> <(optional) branch name>
  • to sync GitHub → prod: $ python sync_github.py pull <repo name> <(optional) branch name>

Legacy Task Tracking

These are the HLA tasks as they existed as of 01/2024. This content is archival.

Preferred development workflow with git repository:

  • Create new development branch for work in personal clone of repo
  • Do work and test
  • Merge in any recent changes from main branch
  • Push your branch, email Glen to evaluate & merge into main branch

Generic tasks/ideas

TaskStatusAssigned ByActively Worked on by...
Full lists of PVs used in HLAs accessible from each help menu
GlenSharon
Complete documentation for each HLA & generic controls/modeling interface
Glen
Remove remaining dependencies on legacy Matlab2012 & "matlab model" code
Glen
Remove instances of direct use of LabCA - move everything over to using PV class in readiness for implementation of PVANeeds doing by mid-2023?Glen
Faster list-based caget operations in PV class + improve first-time calls (speed up LiveModel initialization)
Glen
Implement asyn operations when move to >Matlab2021
Glen
Extend matlab server model to facet-srv01?
Glen
Change default range of Schottky Scan GUI
LoneySharon
Automation of 10-3/10-4 phase scans (in Schottky GUI or phase scan GUI)To be implemented into new phase scan GUILoneyBuschmann
Add Logbook feature to BSA GUI
LoneySharon

AIDA-PVA

TaskStatusAssigned byActively Worked on by...
Larger # buffered BPM data acquisitions (up to 1000 @ 10Hz?)11/7/22: Tested, deployed in Wirescan app- appears to be working well.GlenCOMPLETE
Asynchronous calls (especially for bufferedacq) from Matlab11/4/22: Tested, declared working.GlenCOMPLETE
Implement interface to SCP corr plots10/22/22: Implemented by George, documentation on AIDA-PVA website, needs testingGlenGeorge, Glen
Multi-device set (and asynchronous status return) on PMDL (and others...?)11/7/22: Tested, working well in MDLFF app for multi-set of PMDL values.GlenCOMPLETE
Feedback HSTA bit control (be able to change Feedback → Compute)9/22/22: Implemented and tested ability to write raw HSTA bit values, matlab interface to toggle OFF, Compute, Feedback states written and tested. George to implement cleaner AIDA-level state change interface.GlenGeorge, Glen
Fix AIDA service crashes on VAX (or be able to detect and self correct)9/22/22: Many potential non catching of errors found by George and changes to server-side code made, cannot induce server failures by testing. Need to operationally observe to see if fixes are good.GlenGeorge & Greg

F2_Orbit

TaskStatusAssigned ByActively Worked on by...
Plotted orbit doesn't respect pre-selected BPM list when loading a Config with pre-measured BPM data
Glen
Indicate TRIM operation in progress by grey-out of "Do correction" button or similar
Glen
Make MIA tab functionality work
Glen
"Plot all" option doesn't work correctly when displaying corrected orbit after performing orbit correction calc
Glen
Re-instate some auto dispersion correct functionality (e.g. DL10 correction with Q10731)
Glen
Add kick and dispersion source fit functionality to orbit and dispersion tabs
Glen
Changing fit location should update plot and fit data- currently doesn't work
Glen
Link reference orbit saved to Python bpm orbit tool reference saves
Glen
Implement TMIT cuts for BPM orbit
Glen
Implement measured Rmat (as opposed to model Rmat) interface (including measurement tools?)
Glen

F2_LEM

TaskStatusAssigned ByActively Worked on by...
Store design L1,L2,L3 operating phases in EPICS PVs
Glen
Watcher version of LEM: info on when LEM needed etc + interface with CUD
Glen
Ignore CQ's & SQ's for purposes of Twiss parameter propagation (BMAG plot)
Glen
Make a way to flag "Matching Quads" and deal with them differently when "LEM to model"?
Glen
Fix BACT=0 causing "NaN"'s in BMAG propogation
Glen
Convert to server + watcher architecture
BuschmannBuschmann

F2_Matching

TaskStatusAssigned ByActively Worked on by...
On Multi-Wire matching tabs, wire names don't change when reading in data from different Linac section
Glen
Keep last vals in MW tab when switch back from other tab
Glen
Auto de-select CQ & SQ magnets from matching options
Glen
Add native quad scan functionality
Glen
Allow Undo button to retain old values after calculating new match
Loney

F2_Feedback

TaskStatusAssigned ByActively Worked on by...
Jitter button ON/OFF status not working properly
Glen
DL10 controller name "IN10"→"LI10" on GUI
Glen
Settings GUI plot to include line showing feedback desired offset value
Glen
Fix freezing of GUI when closing settings panels
Glen
Tie DL10 feedback stpoint directly to LLRF control → switch off "slow fb" on RF controller when feedback on and vice-versa, increase feedback gain
Glen
Implement full PID controller coefficients to each feedback, setup and test and include P or PID as option (at least for DL10)
Glen
Implement use of LI17 fast phase shifters for BC20 feedback through SCP multiknob controls
Glen
Auto-adjust gains for changes in beam rate
Glen
Add FFS transverse feedback
GlenLauren




MDL_FeedForward

TaskStatusAssigned byActively Worked on by...
Data collection: have watcher record each GOLD entry and associated data and save to disk
Glen
Make user-facing configuration GUI to examine training data & evaluate performance & tune NN or linear models or manually tweak
Glen
Implement multi-device write commands in AIDA for PMDL data

10/7/22: AIDA software written to allow this
11/7/22: Implemented in MDLFF app and tested.

GlenCOMPLETE

F2_Wirescanner

TaskStatusAssigned byActively Worked on by...
User access to PMT timing & gate widths
Glen
When selecting motor position units option- propagate changes to plot to see centroid of scan in motor units
Glen
Jitter correction for Linac wirescanners in L2 and L3

9/19/22: Initial tests show buffered bpm acq with ~200 pulses works, initial code implemented, needs testing with beam
10/7/22: AIDA now supports up to 1800 buffered data points
11/7/22: Debugged jitter correction code with AIDA BPM in app, test worked well- all beam size measurements in L2 & L3 reduce when jitter correction selected.

GlenCOMPLETE

S20 Configurator

TaskStatusAssigned byActively Worked on by...
Write current IP waist position to a dedicated PV instead of sharing with desired waist position 
RyanBuschmann

S20 IP Waist from BPM Jitter

TaskStatusAssigned byActively Worked on by...
Create11/6/22: Tested, working. Results are unstable, assuming this is due to dispersion leakage and large energy jitter. Next job is to implement SVD algorithm to simultaneously fit dispersion functions and remove energy jitter component.GlenGlen


SLC Control System Notes

Ken Brobeck's How to restart some VMS processes guide.