Versions Compared

Old Version 1

changes.mady.by.user Alex Wallace

Saved on

compared with

New Version Current

changes.mady.by.user Alex Wallace

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Introduction

This is the first ECS PD newsletter. Hopefully we start putting these out on a monthly basis to take stock of the improvements and work we're doing, and let the rest of LCLS know what new things are available from ECS! Note, even more was done than is listed here. We gave ECS 2 days to fill in highlights, but everyone is super busy and so this is just what we managed to write down. 

Imaging Systems

Fixed server configuration issues that caused the network interface on the camera servers to periodically reset, disrupting data collection and often worse.

PMPS

Squashed a bug related to how the photon energy bitmask is generated to make things more failsafe.

...

After some apparent inconsistencies with veto logic, a deep review of a majority of PLC veto logic was conducted to ensure the PMPS would produce faults only when necessary, and not when an obvious veto device should be keeping things safe. 

Gas and Vacuum Systems

Improperly latching "BPTM timeout" faults have been resolved and will no longer prevent beam seemingly at random.

Hutch Python

Fixed bugs:

  • Fixed lxt_ttc oddities
  • Fixed synchronized axis support in general
  • Fixed issue where DAQ would disconnect itself after a scan in some scenarios
  • Fixed various malformed status printouts
  • Fixed malformed metadata from Newport motors and from derived unit conversion signals

Added new tools:

  • InterfaceDevice, InterfaceComponent, and ObjectComponent can be used to use fully instantiated ophyd devices as components in other ophyd devices, rather than needing to duplicate their constructors.
  • FuncPositioner can be used to similar effect as the legacy python 2 VirtualMotor support
  • SyncAxis class as a replacement to the deprecated SyncAxesBase class with a more solid foundation and some extra features

And also added several many new devices for LCLS-II!

UI/UX

  • Added missing soft limit configuration to typhos expert screens

...

X-ray Optics

March began with some PMPS hacks to protect mirror coatings as the undulators and operators continue to impress us all with increasing beam power. These hacks were formalized in a subsequent upgrade that added state-based control to the mirror vertical axes, and the full PMPS integration (on the L line only). 

XRT Mirrors M1, M2 and  M3 are now connected to the PMPS arbiter and coating protection is now active.

XRT M1 (

...

Monochromator to RIX Commissioning

Grafana, Logging, Alerting

Double3 Robot

FIM Commissioning

MR1L3) Upgrade

The goal was to get the old Elmo drives out and install a new Beckhoff din rail drive panel. These panels drive 6 axis, 6 absolute encoders, and interface with an E-621 Piezo Controller. Also, the rail was supplied with a new 24/48 volt din rail power supply that had to be racked and wired, so even the wire job was considerable; shout out to Du, Danh  who did some fine electrical work. On top of these tasks we had to run the software for MR1L2 on a PLC running software for 2 other flat mirrors in the XRT running older Elmo drive software. Other tasks included getting a new IOC spun up, a new control screen, and re pointing older screens to use the updated MR1L2 screens.

The wiring job took two days, so we stayed late on the second day to get the rail powered up. The PDU was reading negative voltage, shout out to Sameen Yunus  for not letting me blow the panel up. We flipped the PDU wires, got positive voltage; got the rail powered on. The Beckhoff 5041s were throwing some error lights and complaining about absolute positions being off.

As Day 3 started, we had a full controls day ahead of us. As I walked in that morning I discussed the encoder errors with Jackson R Sheppard and it was decided to swap the cables. The encoders were a bit funny, we swapped 1 encoder cable out with another that was properly shielded/grounded, and 5 of the encoder errors went away! The emergency STO was checked, and small mechatronics movements began. All axes were moving! except the pitch =(  Alex Wallace Margaret Ghaly , led the charge and began digging relentlessly into the heart of the E-621 serial commands. Ultimately, it was decided that the E-621 controller had left us for a better place. 

Final day! We moved on from the E-621 and implemented the pitch motion directly with the new actuator Hardin, Corey L. had installed. He was excited to tell us all about the new specs the actuator had, and was hopeful that the actuator was such a high performer that we wouldn't need the E-621. With the IOC spun up by Zachary L Lentz in mere 15 min and Margaret Ghaly checking out M2, M3 I was able to focus on the final full movements of every axis on MR1L2. The Y axis didn't like moving at 200 um/s and would error occasionally, but ran without complaints at 100 um/s. Soft limit positions were set close enough to touch each limit switch without having to activate. 

We had about an hour to spare before the XRT was getting scheduled for search. With that Gabriel Castilloand I went and tidied some cables up. Success! 

SP1K1-g_pi Support in the DAQ

LCLS-II DAQ support has been added for SP1K1's grating pitch! This works up to 120Hz and involves triggering the motion PLC directly to get encoder counts out and sent directly to the DAQ. This can feasibly be re-used to get any values we'd like from the PLC to the DAQ directly. The values are timed into the DAQ using an EVR.

Grafana, Logging, Alerting

  • EPICS Archiver data source now allows for alerting, thanks to an ongoing fruitful collaboration with KEK, the Japanese High Energy Accelerator Research Organization. This has been a promised functionality for years, so we're very excited to finally roll it out.
  • A new PLC logging daemon ("ads-log-daemon") was installed to automatically stream logs from any PLC configured on the network.
    • Due to limitations of Beckhoff hardware, the previous configuration did not catch all messages coming from PLCs.
    • This new daemon hooks into the same system that TwinCAT XAE does, and therefore should be able to catch all messages from PLCs - even without a valid TF6310 TCP/IP license installed.
  • A beta-level data source was added to log full IOC output in Grafana.
    • The current (stable) data source only accepts EPICS errLog messages, which not all IOC authors may be using.
    • Due to the extreme amount of log messages generated (>100 million per week), some pre-filtering is required, which will take time to perfect.
  • EPICS Archiver Appliance statistics are now being tracked and monitored.


Solid Attenuators

The attenuator AT1K4 was installed and checked out. Coordinating with TMO, individual filter positions were determined and a calculator IOC was deployed to support easy normalized transmission-based operation.

Double3 Robot

Image AddedImage AddedImage Added

Kaneda, our trusty ears and eyes in the NEH, now roams the halls freely (with supervision). Let the robot revolution commence. Login and access guidance can be found here. More robots following soon in other LCLS areas. Drive safe, have fun!

AR Headsets

Image Added 

The RealWear HMT-1 AR headset was at last deployed to the field at LCLS. Headsets have been handed out to SEAs in nearly every area of LCLS for initial deployment and training. The headsets are assigned to each hutch, as well as ECS, SED and Technical support. Check in with your local SEA to get instructions on usage!

You can find more information here.

FIM Commissioning

Image Added

The first FIM commissioning shift was conducted successfully this month.     Once timed in, we started getting good beam signals on all the wave8 channels.     Tests were done w/ both LCLS1 and LCLS2 timing and some lessons were learned re which registers contribute to the timing delay for LCLS1 vs LCLS2.    There were also some issues w/ firmware or timing signals freezing where we needed to reset the timing.    Will continue to look into this.    We'll also be looking to do some improvements to integrate the FIM screens into TMO home and to provide some FIM specific aliases for Wave8 PVs which are of special significance for the FIM.

makepeds

The newest version of makepeds lives in R1.8.5 of engineering_tools. Changes to prior versions:

  • support for the FFB: specify with -F or by specifying a queue in the new FFB (ffb[h/l][1/2/3]q). Please ask your ECS POC which queue is appropriate - this will typically be the same as used for data processing.
  • data processing is 'local/no queue' by default. To use the queue, you need to specify '-q <queue>'. We recommend the queue only for epix10k pedestals as this allow for parallel processing, speeding up processing.
  • geometry option: '-g <experiment_to_copy_from>' will copy the geometry file from that experiment to the specified one with correct permissions and updating the 'history' as well. You will need to pass a run still so that detectors that need a geometry can be identified. Please note that the Jungfrau detectors also need the offset-files which are not yet included here.

Jira and Issue Reporting

ECS Platform Dev began exploring the use of Jira as our future ticketing system. We developed an embedded form for submitting issues to our Jira backlog, which can be found in various confluence locations. The goal is to make a very simple and easy process for people to request help from ECS PD. Ultimately we are looking to embed this process directly into our GUI so control system users can effortlessly report an issue, and provide all the helpful context we need to troubleshoot issues. An additional goal is visualizing the ECS PD work queue. Using Jira we can easily embed our backlog in confluence so anyone can see where their ticket is in the queue and lobby to change priority if needed. So far, our backlog is filling up with tasks from the XBD team as well as our own work. The next step will be to really apply Agile processes to organizing and prioritizing our work items.

MEC-U CD-1

MEC-U successfully passed CDR review on March. It is a critical milestone achievement for the MEC-U project.  Next goal for ECS is to get controls WBS, WBS dictionary, and EOB ready for Director's review and Independent cost review. 

L2SI project 

For March we closed these activities. Congratulations to task owners! 

EPICS: Engineering and Design for FEE lockup

EASE: Engineering and Design for Component Assembly

Exit Slit: Engineering and Design for Assembly

We will get back to TMO DREAM starting April.  Near-term goal is to complete final design of 80% of the DREAM control systems. Long-term goal is to get ready for installation by April 2022.