Press 8/8
Closed for editing, 8/4
PMPS UI Fixes and Updates
We had a few fixes and feature updates for the PMPS UI diagnostic tool in July.
- Fixed a bug where the rate and transmission readbacks on the line beam parameters page were showing the live totals instead of the controlled readback value, causing some confusion.
- Add a default "Beam Permitted: False" filter on the fast faults page. This makes the GUI load slightly faster because it doesn't have to render all the fast fault widgets on load, and it lets us get to the most important view first.
- Disable the grafana web views for now, these are causing crashes on operator consoles.
- Rearrange the "Arbiter Outputs" page that was previously difficult to see which status was connected to which PLC.
Hutch Python and Conda Updates
The following environments were released during May, June, and July:
- PCDS Conda Release Notes: pcds-5.4.0
- PCDS Conda Release Notes: pcds-5.4.1
- PCDS Conda Release Notes: pcds-5.4.2
pcds-5.4.2 is the first environment with "gentler" dependency updates to minimize the potential for picking up unexpected behavior on update.
Note that any applications using pcds-5.3.1 should update if they plan to use hutch-python in an experiment setting. There is a bug in this version that can lead to dangerous results where the history of separate ipython sessions will get mixed during execution, so someone else's "move" command can end up in your history and it is very very easy to accidentally run their command instead of re-running yours.
Python Performance Project (Ongoing)
We've been working on tracking down the reasons why various Python apps are slow to load and trying to minimize the parts we can control. To this end, we've already found a bunch of speed ups and work is ongoing. Some of these are tricky because, while startup speed is important, having a design tradeoff that adds slowness later can be just as bad.
A project page is opened here and work is ongoing: Hutch Python/Lightpath Device Loading Slowdown Findings/Fixes
L2HE Update
Margaret Ghaly Vincent Esposito
Over the past couple of months, the ECS HE team has been making significant progress on the new controls infrastructure designs. The cost estimates for each hutch has been mostly completed and we are approaching PDR-level readiness in many areas. PDR is now planned for September 2022 (exact date TBD).
The team is working on refining the designs and continues to focus on the PDR, collaborating closely with the mech-E leads and Cosylab.
As a new member of ECS, Mitch Cabral has joined the effort and is working on Jama integration, a collaborative web-based tool for reviewing, approving, and exporting system requirements for release. We have begun capturing L2SI Motion and Vacuum System Functional Requirement Specifications (FRS), and although these requirements are still being reviewed and revised, the hope is to make these documents more accessible for projects to follow, and possibly establish these as standards for our control systems. Transferring to Jama will also allow for agile navigation to find conflicts with upstream or downstream alterations in future project requirements between stakeholders and the team.
MEC-U Update
Lightpath Campaign
We've been working to rebuild the lightpath application, a tool that aims to give a high-level summary of the beam, where it's pointing, and which devices are blocking. In order to properly represent the facility, significant changes were made both how lightpath organizes devices and how those devices are represented. As of the writing of this newsletter, we have completed the major infrastructural changes to lightpath, implemented a new device interface, and begun to spot check the app's performance for select end stations.
Design details and FAQ's are being gathered at this page, which (like the lightpath app) is a work in progress.
ATEF
EPICS Codeathon
NALMS
TMO
- Granite 3 (all components are checked) is ready to go for dream mirror check out experiment(starting from 8/7): IP2(no chamber)-Granite 3 Controls I
Mirror: Waters, Nick Vacuum: Jing Yin Tong Ju Motion: Maarten Thomas-Bosum Zachary L Lentz PMPS: Margaret Ghaly Tong Ju Image: Tong Ju Govednik, Janez
- TMO whole beam line PMPS is completed from the FEE to granite 3 : TMO PMPS System Margaret Ghaly Tong Ju
Scientists will run dream mirror check out experiment and test all PMPS and veto groups at the same time
IM4K4 Motor Resonance Crashing a Turbo
TMO had a strange problem back in April: whenever they would move IM4K4 in or out, it would vibrate the stand violently to the point that it would cause a nearby turbo pump to crash, which was extremely disruptive to endstation operations.
Stepper motors are known for having some problems with resonance and high vibrations. Typically in industry you would pick servo motors if extremely smooth motion was a requirement. This isn't a requirement here, but "do not shake the stand when moving" is absolutely a reasonable ask.
All of the PPM imager/power meter combination units were tuned based on a single instance before it was installed onto the beamline. As such, the parameters were optimized for a lab bench in the temporary clean room in B750 rather than for the beamline as-installed. With the rush to close out the installation of the L2SI components, it looks like this had never been revisited.
In general, we expect every instance of a motor to have slightly different tuning needs based on the precise specifications by which the assembly was installed. This won't always be worked on to completion because often there won't be any problems, and because the most important specification for the majority of our motors is position accuracy and precision of the end position, along with motion reliability, which is not covered by this vibration case.
It was observed that the PPM units have higher vibrations than one would expect in the FEE, RIX, and TMO, culminating in this turbo pump incident. So, we took some time to do the following for every installed/active PPM:
- Make the position correction loop more gentle. The PLC software that governs the in-transit motion was very aggressively conforming the stepper's movement profile to the "optimal" profile, leading to overcorrection and some slight increase in the vibrations. They were also tuned for an expected runtime velocity of 65 mm/s, when in practice we were running them at 5 mm/s.
- Pick a runtime speed for the PPM that minimized vibrations from resonances. It turns out that all of the PPMs have strong resonances with their stands in the 5 mm/s - 8 mm/s movement speed ranges. This is why most of the PPMs could perform OK if they were slowed down. What isn't obvious if you aren't aware of the resonances is that you can also make the PPM perform better by speeding it up. Most of the PPMs had a "best-case" (minimum vibration) speed at around 12 mm/s, though some performed best as high as 15 mm/s or as low as 11 mm/s.
After these adjustments, the PPMs are silky smooth, quiet, and reach their destinations reasonably quickly.
MEC SPL Upgrade
Hello Goodbye!
| Antonio Gilardi has joined the ECS Delivery group as the new MFX and UED Point of Contact. |
 | Mitchell Cabral has joined the ECS Platforms Development team as a Control System Integrator, whose focus is supporting developing projects (L2HE and MEC-U primarily). Mitchell is a recent undergraduate from CSU, Chico and worked with SLAC for the past year to develop a prototype computer vision system robot for Dr. Diling Zhu in XPP. Some of his hobbies/interest include cooking, the Sacramento Kings, volleyball, rock climbing, (hiking to/swimming in) large bodies of water (with a nice beverage). |
We said goodbye and farewell to Maarten in July. He will be missed.
Github
It should be noted a huge quantity of our work is done on Github.com, all development is tracked there. Jira issues capture a significant body of work as well, but at least as much work is also captured in the closure of Github tickets (issues) associated with our various codebases. Unlike Jira, getting a consolidated metric of work done in a past period is not possible without a paid subscription to Github. Roughly speaking over 80 projects were touched since April 8th, with multiple changes of various sizes.
Jira
Getting issues...
Overview
Content Tools