SLAC End Station A (ESA) Test Beam (ESTB)
The SLAC End Station A Test Beam (ESTB) was re-establishing in 2014 to take small fraction of SLAC LCLS beams into End Station test beam line. The operation is largely parasitic to the LCLS operations with electron beams in the 5-13 GeV range depending on the LCLS operation energy. From the FACET/ESTB portal, you can navigate to info on proposals, new user registrations etc. ESTB beam schedule mostly just follow the LCLS Run schedule (which may be not always up to date). The CALADIUM EUDET telescope owned by Carleton University has been installed at the end of ESTB as a precision tracking telescope with 6 planes of CMOS/MIMOSA sensors (1152x576 18.4x18.4um pixels) with ~3 micron spatial resolution. User Device Under Test (DUT) can be mounted on a movable stage in the middle of the telescope. For ATLAS pixel test beam users, a HSIO2/RCE based DUT readout will be provided as a standard readout utility.
ATLAS Test Beam Proposals and Sessions
Current proposals:
- T-539 ATLAS Silicon Tracking Development (welcome shared ATLAS use)
- T-545 LBNL Pixel Devices
- T-578 UC Santa Cruz HGTD
Scheduled sessions:
The list of sessions below may be incomplete (welcome to add missing sessions). Links behind each session dates below navigates to activities on individual sessions. In case of confluence wiki access issue in either viewing or editing the Event log page yourself, if only needs a trivial registration to gain access.
Latest Run 17 LCLS/ESTB (2018 Fall schedule)
- Dec/13-18 T578 HGTD
- Nov/2-7/2018: T539 + T545 ITk Pixel. Event log
- Oct/3-7/2018: T578 HGTD
- May/30-Jun/5/2018 T545 + T539: Joint ITk pixel session with first RD53A module (+T578 parasitic). Event log
- Apr/5-9/2018 T578: UCSC HGTD run with first ESTB beam after kicker magnet upgrade. [Event log] [Report]
- Sep/26-Oct/4/2016 T-545 + T-539: Joint ITk pixel session for new ITk devices and some additional irradiated IBL FE-I4 module runs. Event log
- Jul/7-19/2016 T-539 + T-545: Combined ITk pixel session and continued irradiated FE-I4 module tests. Event log
- May/05-10/2016 T-539: Data with irradiated IBL FE-I4 modules for simulation model. Event log
- Jan/12-25/2016 T-534: Recommissioning of CALADIUM telescope and ATLAS DUT readout.
- Oct/2015 Caladium installation: 151030_caladium_at_slac_report.pdf, 151030_caladium_pictures.pdf
- May/01-15/2014 T-516: Various ATLAS ITk pixel upgrade samples.
Operations Info
Extension | Contact | Extension | Contact |
---|---|---|---|
2151 | Main Control (MCC) Operator | 8556 | Carsten Hast |
2811 | ESA counting house | 2084 | Keith Jobe |
4299 | Radiation Physics (for ESA exit survey) | 5200 | Mike Dunning |
4371 | End Station A floor | 2284 | Su Dong |
2551 | Main Gate security guards (door keys) | 3912 | Matthias Wittgen |
Su Dong cell phone: 650 430 4470 | |||
Timon Heim cell phone: 510 944 9508 | |||
Ben Nachman cell: 402 689 8125 |
ESTB DAQ Configuration
An example of EUDET+DUT setup as of 2016 is shown below when running EUDAQ-1.5. The upgrade in May/2018 to EUDAQ-1.7 has changed the TLU-NI USB connection for the TLUProducer to TLU-DUTserver. There are variations of DUT readout e.g. RD53 test readout for two session in 2016 and 2018 used YARR in place of the HSIO2+SunServer combination for DUT readout.
The Caladium telescope control and DAQ is handled by the National Instrument (NI) crate which collects data via the TLU with data producers from the telescope itself and the DUT. The DUT readout has a baseline arrangement with HSIO-II controlled by the ESADUTDAQ1 server node via a direct private ethernet link. Due to the very short ESTB bunches, there is no need for additional triggers within the bunch and the DAQ is simply driven by the bunch crossing beam trigger. The SWH-B061-NW01 network switch in the middle of the ESA hall support the ESA-RESTRICTED local network 172.27.104.* (port 25-48) where both the NI crate and ESADUTDAQ1 reside on. The EUDAQ console PC AR-EUDAQ in the ESA Control Room and several other nodes e.g. AR-ESAUX2 are connected to the ESATESTFAC public subnet 172.27.100.* (port 1-24) on the same switch, but can also see the local ESA-RESTRICTED subnet. The connection from the NI crate and ESADUTDAQ1 to the switch are routed through the Belgen patch panel inside the tunnel near the optical table.
The current version of EUDAQ on the DESY/CERN telescopes is 1.7 so we have updated the version at SLAC in May 2018.
Telescope operation (EUDAQ v1.7)
0. In ESA:
- Start chiller: 18 Celsius
- Switch on power supply: 8V constant --> check 2.0 A
- Switch on NI crate and TLU
1. Start sensors
- Remote access to NI crate:
-- Open Terminal
-- source access_NI_crate.sh
-- Login: telescope PWD: telescope+
- Start MI26.exe (upper icon on the NI crate Desktop)
-- Open .mcf threshold file, typically 6 (CALADIUM config files are in C:\opt\mimosas\config_files)
-- Press "Reset" --> 2.0 Amps
-- Press "All" --> 2.8 Amps
-- Press "Read" --> 2.8 Amps
-- Press "Start" --> check 3.6 Amps
2. Open Labview FPGA interface on NI crate, if not already open
- Double click MimosaDAQ (second upper icon on the Desktop)
3. Start EUDAQ on AR-EUDAQ:
- Open a second Terminal (login as tfuser with default dir /home/tfuser/)
- cd eudaq
- source setup_eudaq.sh
- ./STARTRUN
- wait approx. 20 sec. until all Connections ping (uninitialized)
4. Start TLUproducer on esadutdaq1
- (if not done already: start a terminal on ar-eudaq and ssh -Y beam@esadutdaq1 (ask Su Dong or others for password)
- cd eudaq/bin
- ./TLUProducer.exe -r ar-eudaq:44000
--> check in eudaq Run Control, that Produce TLU is connected
5. Start DUT data producer(s) if not running telescope by itself.
- For standard RCE based DUT readout, this should be just starting rceOfflineProducer on esadutdaq1.
- For YARR based readout (for beam rigger runs), login to pixel@172.27.104.52 as in the TLUProducer case, then
cd YarrEudaqProducer
bin/YarrProducer -h configs/controller/specCfgExtTrigger.json -c configs/connectivity/example_rd53a_setup.json -s configs/scans/rd53a/std_exttrigger.json -r 172.20.100.8
6. Configure EUDAQ in eudaq Run Control
- select init file (should just use the default example) and init; then select Config file
(Caladium only exercise config: no-beam: ni_autotrig;
beam: ni_coins, June test beam 2018, ni_coins_YARR)
- Press Config
- wait till all Connections are green (OK)
7. Start Run
- Press Start (wait till all green)
--> check, that Trigger and Event Built numbers are increasing
--> check OnlineMonitor, and reset (upper left x-button)
8. Stop
- Re-configure (5.) and Re-Run (6.)
OR
- Terminate and press "Reset" in MI26.exe at NI crate to stop sensors --> check 2.0 Amps
EUDAQ Online Framework
The EUDAQ software operation involves 3 types of main entities:
Entity | Location | Node name | IP | OPS | Processes |
---|---|---|---|---|---|
Run Control PC | Control room | ar-eudaq | 172.27.100.8 | RHEL 6 | euRun,LogCollector,DataCollector,NIProducer,OnlineMonitor |
NI Crate | Caladium Rack | ni-eudaq | 172.27.104.9 | Win 7 | EUDAQ, MimosaDAQ, LV FPGA |
DUT/RCE | Caladium Rack | esadutdaq1 | 172.27.104.45 | Centos 7 | TLUProducer, rceOfflineProducer |
DUT/YARR | near Caladium | esadutlbnl1 | 172.27.104.52 | SLC 6 | YarrProducer |
Typically, there is only one type of DUT server in operation at any time and that DUT server is also serving the TLUProducer (new arrangement as of Jun/2018).
The operation directory structure on ar-eudaq: user always login as tfuser for EUDAQ operation.
- EUDAQ release and operation base: /opt/install/eudaq/eudaq-1.7
- EUDAQ configuration: /opt/install/eudaq/eudaq-1.7/conf
- Raw data directory: /opt/install/eudaq/eudaq-1.7/data/run***.rawdata
- Online monitoring histograms: /opt/install/eudaq/eudaq-1.7/bin/run*.root (needs manual save to store)
- Online monitoring archive: /home/tfuser/Desktop/yyyymon_onlinemon/ (recommend to copy online mon history root files here)
Pixel DUT RCE Readout
While the DUT readout can be of variety of ways, the RCE readout setup is provided for at least ATLAS pixel users as a standard utility. As of July/2018, the DUT RCE readout is upgrading to HSIO-II setup with EUDAQ-1.7, with the dedicated ESADUTDAQ1 server also upgraded to a fanless Kingdel PC residing the beam tunnel in the EUDET rack next to the NI crate as the DUT RCE readout host. HSIO-II has the DTM mezzanine with the Gen-3 ZYNQ based RCE operating Centos-7 on the ARMS CPU that communicates with external world via its native 1Gb/s Ethernet. More general information on HSIO-II/RCE can be found on the RCE Development Lab Twiki. If the RCE readout setup with HSIO-II is dissembled and reinstalled, it is important to remember two important grounding connections to ensure LVDS communications have proper DC reference levels:
- For HSIO2-module communication: Ground wire between HSIO2 LV supply Molex pigtail and pixel module LV PSU ground
- For HSIO2-TLU communication: Lemo cable between TLU trig0 TTL and any of the HSIO2 main board LEMO bank socket
If you are not running the system from the ESA console next to the EUDET, but from some public nodes at SLAC, you need to first hop onto an ESA public machine on the 172.27.100 subnet and login as yourself e.g.
ssh -Y ar-esaux2
At least calibration actions will need X window displays so that the -Y qualifier through out is necessary. Then continue to hop onto the DUT readout node as dedicated user beam:
ssh -Y esadutdaq1 -l beam cd ~/daq/rce source ./scripts/setup-env.sh
For normal EUDAQ data taking, you need to start and announcing RCE DAQ as a data producer to EUDAQ run control node ar-eudaq with just one command:
rceOfflineProducer -r 0 -i 192.168.1.1 -d ar-eudaq
If everything is run smoothly, once you brought up the overall DAQ with EUDET through EUDAQ run control node ar-eudaq, you should see the RCE data producer show up in the producer list.
Pixel Calibrations
Before taking an serious beam runs, it is advisable check the modules through calibration. Some sensors are known to need different threshold tunes at different temperatures, so that some calibration runs are expected to be a frequent activity between beam runs. The HSIO reboot will automatically bring up the calibserver on the RCE side to be always ready for it. If you suspect the calibserver ran into trouble or something messed on the RCE, doing a rebootHSIO may be the simplest way to recover. After the regular login process to ESADUTDAQ1, you can launch calibration GUI from ~/daq/rce after executing the ./scripts/setup-gen3.sh once for the session:
calibGui
The operations steps of calibrations are explained briefly in the RCE pixel calibration instruction Twiki , while the functionality details of the calibrations are described in the ATLAS pixel calibration manual. The most commonly used are DIGITAL_TEST and THRESHOLD_SCAN. The new Gen-3 HSIO-II based readout uses a simplified process communication scheme which limits the calibration to one instance of calibserver on RCE and one instance of calibGUI on UNIX host ESADUTDAQ1. It will reject duplicate starters and issue an error message. In case you wanted to run caliGUI from the ESA pit but another session is open in the counting house with noone attending, you can just kill it after checking the process list with ps. The default calibration repository is under /home/tbslac/pixelrce/rce/moduleconfigs/.
In case you want to debug some calibration related problems that can benefit e.g. looking at the calibserver console printout, you can logon to the rce from an esadutdaq1 window:
ssh root@rce0 (pw=root) rce> export PATH=/root/pixelrce/bin:$PATH; export LD_LIBRARY_PATH=/root/pixelrce/lib:$LD_LIBRARY_PATH rce> calibserver
DUT DAQ Configuration
The DUT integration is controlled through setup of EUDAQ configuration files with names like ni_coins.cfg etc. residing in /home/tfuser/eudaq/conf folder (also Desktop folder) on the EUDAQ console node ar-eudaq. The data logging also goes to the data directory in the same folder. These configuration info are not only passed on to the RCE producer to configure the DUT, but also interpreted by the EUDAQ to derive online monitoring for the DUT together with the telescope. A general example of telescope + DUT configuration can be found here (not SLAC ESTB), while some explanations of the various configuration items related to the RCE producer can be found here.
Pixel DUT YARR Readout
In addition to RCE, we also have integrated the YARR (Yet Another Rapid Readout) system into the EUDAQ system in ESA. Here are some instructions.
ssh -XY pixel@esadutlbnl1 (pw=Testbeam)
cd YarrEudaqProducer
./start_1.sh
In addition to starting the producer, one needs to modify the DUTMask in the .conf. Add +4 to include YARR. There is no need to add anything else because the configuration is currently hard-coded into the producer (and the Latency = 43 and Trig Count = 3 seem good so there should not be a need to touch this).
To fully kill this producer if it gets stuck or when terminating the run control, do a ctrl+Z to move it to the background and then kill in the usual way with ps ux | grep yarr. Then, kill -9 <pid> where <pid> is the id of the producer from the ps aux.
After stopping a run, there will be some offline plots in ~/yarr-eudet/eudaq/data. There is also a simple script for offline analysis:
cd ~/yarr-eudet/eudet
../src/bin/read_rawData data<file>.raw #<file> is <Dutname>_<runnumber>
On occasion, you the trigger counter will increase, but no events will be built. This could be because the chip is in a funny state. We can reconfigure the chip and power cycle:
to run a digital scan
cd
cd Yarr/src
./test_digi_1.sh
qpdfview data/last_scan/*.p*
check that they look okay (note that syncrhonous is off). To power cycle
minicom -D /dev/ttyUSB1
ctrl-a z
ctrl-a e
++addr 3
to check what you are talking to:
*IDN?
should be Agilent Technologies ... To turn it off:
OUTPUT OFF
OUTPUT ON
ctrl-a z x
Infrastructure Utilities
EPICS Control Interface
The ESTB EPICS control/monitoring interface that provides some important general utilities, such as the XY stage remote control to align the telescope and DUT to the beam, and remote controlled individual power strip socket on/off that can come handy for remotely powering off detectors and rebooting DAQ components and computers inside ESA, as well as web cams. To access the ESTB EPIC panel:
(if you are using the ar-esaux3 console in the ESA control room locally, it's likely that it was already logged in as tfuser with this setup already done)
(this last command should bring up the ESTB EPICS panel on the right) For ATLAS test beam operations with Caladium, the dedicated T-539 panel contains some of the most relevant sub panels. |
---|
Beam Trigger
The "Triggers" panel on the ESTB EPICS GUI brings up the Experimental Triggers panel where channel 10 (Si Tracker Caladium) controls the beam trigger. The trigger NIM signal is carried to Caladium with a green BNC cable to a LEMO connection into the TLU Chan 0 PM in trigger port. The Delay value is the critical control of timing that DUT is very sensitive to and vary significantly between different types of DUTs, while EUDET seems to work with a wide range of delays automatically. The Trigger width parameters didn't appear to have much effect on anything.
Beam Dump Monitor
The beam dump monitors, Lead Glass (LGdump) dump, and Front/Read dumps are a key monitoring tool to give direct look at the beam intensity. ESTB GUI "Scope ESA" panels is particulary useful as a standard monitor for instantaneous beam status as scope pulses. "ADC Chanls"/"Histograms" panel displays the instantaneous values and distributions of energy measurements of various dumps. "HV SY403" can be used to adjust HV values for different intensity measurements.
Time History Chart
The "StripTool" panel is a very handy general utility that allows you to keep track of the time history of any variables in EPICS. For example the DUT region temperature sensors are particularly useful in cooled sensor operations From the StripTool Control panel, you can pick from any EPICS display panel an variable by clicking the middle button of the mouse over the displayed value of that variable which will bring up a signal name string which can be dragged into the StripTool "Plot New Signal" selection field to add the signal to the time history strip chart.
The T-539 panel
The T-539 panel contains most of the relevant displays for Caladium telescope and ATLAS DUT. | |
---|---|
Caladium BeckhoffXY stage motors: The telescope + DUT assembly are mounted on a remote controllable base "coarse" XY control (X=horizontal transverse to beam; Y=vertical) to align the whole assembly wrt beam. During new test beam periods, this is typically done initially using a pair of alignment lasers in conjunction with fiducial markers to center to expected beamline within a few mm. The vertical position is gauged by a laser mounted on the tunnel side wall. The horizontal position is gauged by another laser hanging the ceiling near the side entrance. Once positioned close enough, the refined alignment is done with the beam data in telescope. For both laser alignment and beam alignment, the position adjustments for the whole telescope (including DUT mounted within it) is done with the dedicated Caladium XY stage Beckhoff26 subpanel shown on the right. The base XY stage for the whole telescope + DUT is controlled by the "coarse" X,Y adjustments. +X is pointing away from the ESA entrance gate; +Y is pointing up. The Spring/2016 upgrade added a "fine" XY stage for the DUT to allow additional adjustments of the DUT with respect to the telescope. There is an additional optional DUT rotational stage control. Temperature The Caladium upgrade also added some temperature sensors that can be plugged into desired parts of the DUT setup. The displays for them are on the "KL3314 temp" tab of the Beckhoff panel. | |
Remote Power Strip ControlThe new T-539 panel has a sub panel contains the individual socket controls for the dedicated power strip on the Caladium+DUT electronics/DAQ rack. Clicking on individual channel channel's "Reboot" button will recycle power on that channel. Some channels may automatically flip the state to green after a brief moment, others may need deliberate push on the state button to reenable. The assignments of the channels for the May/2016 T-539 run is shown on the right as an example. Using the terminal in the ESA counting house logged in as tfuser anyone update the channel titles themselves. |
Useful links
SLAC FACET/ESTB portal | ESTB portal (including beam schedule) | ESTB new user registration |
EUDET Telescope User Manual | DESY EUDET support page | EUDAQ-Producer-Software.docx |
Photo Galleries: Caladium installation Oct/2015-Jan/2016 | May/2016 T-539 |
ESTB web cams (only within SLAC network): Ceiling beam view | DUT overhead | Rack power supply | DUT side view | `
ATLAS test beam / pixel documentations: ATLAS pixel test beam Twiki | Pixel calibration manual
Test Periods: 2016-May-July | 2016-Jan | ITk 2015-Sep5 | ITk 2014-SLAC |