...

The Caladium telescope control and DAQ is handled by the National Instrument (NI) crate which collects data via the TLU with data producers from the telescope itself and the DUT. The DUT readout is upgraded to a simpler arrangement with just the HSIO-II controlled by the ESADUTDAQ1 server node (replacing ESA-USER02 server node ) via a direct private ethernet link. Due to the very short ESTB bunches, there is no need for additional triggers within the bunch and the DAQ is simply driven by the bunch crossing beam trigger. The SWH-B061-NW01 network switch in the middle of the ESA hall support the ESA-RESTRICTED local network 127172.27.104.* (port 25+-48) where both the NI crate and ESA-USR02 ESADUTDAQ1 reside on. The EUDAQ console PC AR-EUDAQ in the ESA Control Room and several other nodes e.g. AR-ESAUX2 are connected to the ESATESTFAC public subnet 127172.27.100.* (port 1-24) on the same switch, but can also see the local ESA-RESTRICTED subnet 127.27. 104.*. The connection from the NI crate and ESA_USER02 ESADUTDAQ1 to the switch are routed through the Belgen patch panel inside the tunnel near the optical table.        

...

While the DUT readout can be of variety of ways, the RCE readout setup is provided for at least ATLAS pixel users as a standard utility. As of Apr/2016, the DUT RCE readout is upgrading from old the Gen-1 RCE + HSIO setup to a simpler HSIO-II setup, with the dedicated esa-user02 ESADUTDAQ1 Sun server residing at the end of the beam tunnel near the EUDET area as the DUT RCE readout host. 

DUT Readout Computer Login

. HSIO-II has the DTM mezzanine with the Gen-3 ZYNQ based RCE operating an Arch-Linux based ARMS CPU that communicates with external world via native 1Gb/s Ethernet. More general information on HSIO-II/RCE can be found on the RCE Development Lab Twiki.

If you ar not running the system from the ESA console next to the EUDET, but from some public nodes at SLAC, you need to first hop onto an ESA public machine on the 172.27.100 subnet and login as yourself e.g.

...

At least calibration actions will need X window displays so that the -Y qualifier through out is necessary. Then continue to hop onto the DUT readout node as dedicated user tbslac:

  ssh -Y esa-user02esadutdaq1 -l tbslac         

RCE DAQ Cold Start (to be updated for HSIO-II)

  tcsh

    cd ~/daq/rce 

    source ./scripts/setup_rce-04-00-01.csh 

    unsetenv LC_CTYPE             [ mainly relevant for MAC - see explanation ]

    rce_killall                                 [ wipe clean any existing partitions ]  

    ipc_server &                            [ inter-process-server on Linux server side for host-RCE communication ] 

    rce_ipc_server &                     [ IPC server on the RCE side ]

    rce_is_server &                       [ IS (Information Server) on RCE for gathering histograms ] 

-gen3.sh 

For normal EUDAQ data taking, you need to start and announcing RCE DAQ as a data producer to EUDAQ run control node ar-eudaq with just one command:     rce_load rce20                        [ Get DAQ process to run RCE - current Gen-1 RCE used has a node name rce20 ]  

    rceOfflineProducer -r 0 -d 172.27.100.8     [ Announcing RCE DAQ as a data producer to EUDAQ run control node ar-eudaq ]     

If everything is run smoothly, once you brought up the overall DAQ with EUDET through EUDAQ run control node ar-eudaq, you should see the race data producing show up in the producer list. If the something seems RCE producer seems to be stuck at the rce_load step (can also verify by trying to ping rce20), the simplest remedy for the moment is just to do a remote controlled power cycle on the RCE ATCA shelf (see below for remote power control through EPICS: "AC power 7", outlet 5).  

Resets

Soft reset:  Just use the command "rebootHSIO" followed by "rce load rce20" without relaunching the various servers can sometime work.

not behaving correctly even upon restarting, there is some possibility that some other program (e.g. calibration server) is still running on the RCE . You can logon to the RCE to reboot it for a clean start. The HSIO-II RCE ethernet port is known to the ESADUTDAQ1 server simply with a hostname of rce0:

  ssh root@rce0    (pw=root) 

    reboot

which will kick yourself off the RCE and reboot itself. After ~ half a minute, you can restart the rceOfflineProducer as instructed above. In the very rare event that this still doesn't get you out of trouble due to the problem with the FPGA side of HSIO-II, you can power cycle the HSIO-II remotely from the EPICS controlled power strip which has a dedicated socket for the HSIO-II. Hard resets: Most often it just needs to power cycle the RCE ATCA shelf as described above would be sufficient to recover, although all servers need to be restarted. Very occasionally, you need to power cycle the HSIO also.

Calibrations      

Before taking an serious beam runs, it is advisable check the modules through calibration. The most commonly used are DIGITAL_TEST and THRESHOLD_SCAN. Following the same steps as DAQ until after the rce_load step, one can start the calibration GUI with the command on esa-user02to logon to esadutdaq1 with two windows.

In window-1, logon to HSIO-II RCE to start the calibration server:

  ssh root@rce0    (pw=root)

  rce> export PATH=/root/pixelrce/bin:$PATH; export LD_LIBRARY_PATH=/root/pixelrce/lib:$LD_LIBRARY_PATH
  rce> calibserver 

In window-2,  start the calibration GUI on esadutdaq1:

  calibGui & 

The operations steps of calibrations are explained briefly in the RCE manual page of 2014/Oct test Beam pixel calibration instruction Twiki, while the functionality details of the calibrations are described in the ATLAS pixel calibration manual. The default calibration/config directory on esa-user02 (as of Jan/2016) is /home/tbslac/testbeam20160112 where DUT modules have their calibration constants under the folders with module names.    

DUT DAQ Configuration

The DUT integration is controlled through setup of EUDAQ configuration files with names like ni_coins.cfg etc. residing in a /home/tfuser/eudaq/conf folder (also Desktop folder) on the EUDAQ console node ar-eudaq. The data logging also goes to the datadir in the same folder. These configuration info are not only passed on to the RCE producer to configure the DUT, but also interpreted by the EUDAQ to derive online monitoring for the DUT together with the telescope. A general example of telescope + DUT configration configuration can be found here (not SLAC ESTB), while some explanations of the various configuration items related to the RCE produce producer can be found here.  

Infrastructure Utilities

...

The telescope + DUT assembly are mounted on a remote controllable base "coarse" XY stage (X=horizontal transverse to beam; Y=vertical) to align to the whole assembly wrt beam. During new test beam periods, this is typically done initially using a pair of alignment lasers in conjunction with fiducial markers to center to expected beamline within ~5mma few mm. The vertical position is easier while horizontal centering is still somewhat awkwardgauged by a laser mounted on the tunnel side wall. The horizontal position is gauged by another laser hanging the ceiling near the side entrance. Once positioned close enough, the refined alignment is done with the beam data in telescope. For both laser alignment and beam alignment, the position adjustments for the whole telescope (including DUT mounted within it) is done with the "XY Stages" dedicated Calladium XY Stage subplane under the Beckhoff panel EPICS panel that is accessible from the ESTB main panel shown above. The vertical adjustments are somewhat slow to complete while horizontal adjustments are faster.   base XY stage for the whole telescope + DUT is controlled by the "coarse" X,Y adjustments. The Spring/2016 upgrade added a "fine" XY stage for the DUT to allow additional adjustments for the DUT wrt the telescope. There is not only a fine XY DUT adjustments, the DUT can also sit on a rotational stage with a dedicated rotational control.   

Beam Trigger

The "Triggers" panel on the ESTB EPICS GUI brings up the Experimental Triggers panel where channel 10 (Silicon Tracker Calladium) controls the beam trigger LEMO signal going into the TLU Chan 0 PM in trigger port. The Delay value is the critical control of timing that DUT is very sensitive to, while EUDET seems to work with a wide range of delays automatically. The Trigger width parameters didn't appear to have much effect on anything.

...

The beam dump monitors, Lead Glass (LGdump) dump, and Front/Read dumps are a key monitoring tool to give direct look at the beam intensity. ESTB GUI "Scope ESA" panels is particulary useful as a standard monitor for instantaneous beam status as scope pulses. "ADC Chanls"/"Histograms" panel displays the instantaneous values and distributions of energy measurements of various dumps. "HV SY403" can be used to adjust HV values for different intensity measurements.   

Remote Power Strip Control

...

The new T-539 panel has a sub panel The "AC Power 7" sub panel accessible from the main panel above contains the individual socket controls for the dedicated power strip sitting on the large tall bench at the entrance to the test beam tunnel next to the telescope. Most ATLAS pixel DUT setup powering are using the sockets on this stripCalladium+DUT electronics/DAQ rack. Clicking on individual channel channel's "Reboot" button will recycle power on that channel. Some channels may automatically flip the state to green after a brief moment, others may need deliberate push on the state button to reenable. Some relevant outlet numbers on this AC Power 7" strip (IP: sioc-esa-acsw7) for the ATLAS pixel FE-I3 test setup (Jan/2016): 

...

 

Useful links

SLAC FACET/ESTB portal  |  ESTB portal (including beam schedule) | ESTB new user registration | ESTB photos

...