T-618: A project of E320 and LUXE (DESY) probing the LUXE Cherenkov detector concept at FACET-II

Table of Contents

Description of the Experiment

...

• x = 194.0 mm→ Rotational axis in beam
• y = 114.0 mm → First straw center height (ch8) in gamma beam (according to mark on dump table → 6.35 mm under global 0 mm)
• r = 27.0° → Straws at 0° 
• For FACET coordinates (0, 0, 2017.2525) m (z is global position of rotational axis, which is in between the two straw rows) → (x,y,r) = (194.0, 120.35, 27.0)

Electron 10 GeV beam 61.3 mm below gamma axis at LFOV.

Fibers in parking position x = -66mm, diag 140mm from photon axis

...

Lower Al rod to first straw → ~7 cm

Electronics Setup

Electronics Details

The setup makes use of several custom and self-designed electronics components including

1. Silicon-Photomultiplier electronics
2. Darkening shutter
3. Microcontroller
4. Readout electronics
5. Scintillating screen

1) Silicon-Photomultiplier Electronics

• Hamamatsu S14160-3015PS s14160-1315ps.pdf
  • ~40k pixel with 15 µm pixel size
  • 3x3 mm² sensor size
  • ~5*10⁵ gain
  • Breakdown voltage ~38 V
• SiPMs sit on a self-designed PCB (PCBSchematics.pdf)
  • Temperature sensor of type PT1000 (FST08-B-1K0E.pdf)
  • Contains passive bias and signal filters 
  • The PCB is mounted inside the electronics box ("Box0" & "Box1")
  • All signal and bias channels are fed out via LEMO00 connectors
  • Temperature is connected to a 4-pole LEMO connector

Image RemovedImage Removed

2) Darkening Shutter

In order to move the darkening shutter (see Mechanical Setup) a magnetic switch is attached to the carbon plate. When +12 V_DC are applied the shutter is being pulled and the straws are optically disconnected from the SiPMs. The datasheet can be found here DatasheetMagneticSwitch.pdf.

3) Microcontroller

The microcrontroller box contains a self-made PCB holding an Arduino Micro and various analog components (MicrocontrollerSchematics.pdf). It has two purposes:

1. SiPM calibration system
   1. Generating a pulse to control the 4 LEDs inside the 2 LED boxes
   2. LEDs are blue LEDs (DatasheetBlueLED.pdf) which are mechanically modified to have a flat lens
2. Temperature sensor
   1. Reading the voltage variation over the temperature sensor
   2. 50 times every 5 s → Mean and standard deviation returned

In order to communicate with the microcontroller over long distances a half-duplex serial communication using TTL is being used. Furthermore, a TTL trigger signal is being generated for a timed readout of pulsed LED signals.

A second microcontroller board is sitting close to a PC to establish a Serial-to-USB interface. It is configured as the following:

• Baudrate: 9600
• Bytesize: 8
• Parity: None
• stopbits: 1
• Commands:
  • "T" : Reads back temperature value
    • Returns: "T=0.00 0.00 0.00 0.00 "
    • With temperature1 std1 temperature2 std2 (left to right)
    • If no temperature sensor connected → "T=NaN NaN NaN NaN " returned
  • "R=0" : Sets pulser frequency (int value)
  • "L=1111" : Binary mask for LED channels → "L=0001" enables LED0 only 
  • "U=13000" : Sets LED voltage (int value in mV) of LED0 and LED1
  • "V=13000" : Sets LED voltage (int value in mV) of LED2 and LED3

Image RemovedImage Removed

Electron 10 GeV beam 61.3 mm below gamma axis at LFOV.

Image Added

→ At EDS rotational axis → 61.3 mm * ( (2017.2525 - z_Dipole) / (2015.59 - z_Dipole) ) = 71.86 mm

→ Default y position is 14 mm → when gamma beam above AL rods and electron 10 GeV below:

• y = 114 mm → gamma beam on first straw
• y -= 70 mm → e- beam on first straw
• y -= 30 mm → y = 14 mm default position!

Electronics Setup

Electronics Details

The setup makes use of several custom and self-designed electronics components including

1. Silicon-Photomultiplier electronics
2. Darkening shutter
3. Microcontroller
4. Readout electronics
5. Scintillating screen

1) Silicon-Photomultiplier Electronics

• Hamamatsu S14160-3015PS s14160-1315ps.pdf
  • ~40k pixel with 15 µm pixel size
  • 3x3 mm² sensor size
  • ~5*10⁵ gain
  • Breakdown voltage ~38 V
• SiPMs sit on a self-designed PCB (PCBSchematics.pdf)
  • Temperature sensor of type PT1000 (FST08-B-1K0E.pdf)
  • Contains passive bias and signal filters 
  • The PCB is mounted inside the electronics box ("Box0" & "Box1")
  • All signal and bias channels are fed out via LEMO00 connectors
  • Temperature is connected to a 4-pole LEMO connector

Image AddedImage Added

2) Darkening Shutter

In order to move the darkening shutter (see Mechanical Setup) a magnetic switch is attached to the carbon plate. When +12 V_DC are applied the shutter is being pulled and the straws are optically disconnected from the SiPMs. The datasheet can be found here DatasheetMagneticSwitch.pdf.

3) Microcontroller

The microcrontroller box contains a self-made PCB holding an Arduino Micro and various analog components (MicrocontrollerSchematics.pdf). It has two purposes:

1. SiPM calibration system
   1. Generating a pulse to control the 4 LEDs inside the 2 LED boxes
   2. LEDs are blue LEDs (DatasheetBlueLED.pdf) which are mechanically modified to have a flat lens
2. Temperature sensor
   1. Reading the voltage variation over the temperature sensor
   2. 50 times every 5 s → Mean and standard deviation returned

In order to communicate with the microcontroller over long distances a half-duplex serial communication using TTL is being used. Furthermore, a TTL trigger signal is being generated for a timed readout of pulsed LED signals.

A second microcontroller board is sitting close to a PC to establish a Serial-to-USB interface. It is configured as the following:

• Baudrate: 9600
• Bytesize: 8
• Parity: None
• stopbits: 1
• Commands:
  • "T" : Reads back temperature value
    • Returns: "T=0.00 0.00 0.00 0.00 "
    • With temperature1 std1 temperature2 std2 (left to right)
    • If no temperature sensor connected → "T=NaN NaN NaN NaN " returned
  • "R=0" : Sets pulser frequency (int value)
  • "L=1111" : Binary mask for LED channels → "L=0001" enables LED0 only 
  • "U=13000" : Sets LED voltage (int value in mV) of LED0 and LED1
  • "V=13000" : Sets LED voltage (int value in mV) of LED2 and LED3

Image AddedImage Added

4) Readout Electronics

For the SiPM signal readout the CAEN VX1742 sampling digitizer (located in FKG20-24) is used. 

...

5) Scintillating screen

The camera setup is either the existing LFOV system or LUXE's scintillating screen includes a scintillating screen (Type ) and camera (Basler ) which can be positioned is placed before the Cherenkov detector.

SiPM Power Supply

The SiPMs are powered via the Keysight E36234 from the electronics rack (FKG20-24) via a long BNC cable to the dump table.

• 2 channels
• 0 to 60V_DC
• ~5mV_PP ripple noise
• Usage here: 1 channel with 36V_DC to 45V_DC

Signal Cables

• Placement:
  • (x,y,z) = (-0.02, 0.669, 2016.6595) m (mounting screw location in facet coordinates, uncertainty of +/- 0.02 m) 
  • tilting angle of 36°
  • Camera focused at center of screen at motor position y = 55 mm
• Networking
  • POE adapter in AC power strip channel 16 (ACSW:LI20:NW06:16XXX
  • IP address 192.168.100.5
  • Ethernet at dump table to FKG20_2434 back channel 1
  • LAN2 port of DAQ PC configured with 192.168.100.2
• Trigger splitted from camera trigger pm03-01

SiPM Power Supply

The SiPMs are powered via the Keysight E36234 from the electronics rack (FKG20-24) via a long BNC cable to the dump table.

• 2 channels
• 0 to 60V_DC
• ~5mV_PP ripple noise
• Usage here: 1 channel with 36V_DC to 45V_DC

Signal Cables

...

***Cables ch07&15 are 48cm longer!

Trigger Logic

Image RemovedImage Added

The NIM pulse width of discriminator outputs is set to 10 20 ns by default.

By tuning the first discriminator one can change the pulse timing at the OR input.

The final NIM trigger pulse is 50 60 ns long. With default settings the NIM trigger arrives at the digitizer ~60 ns after the actual trigger input.

The beam trigger is received at the VME crate in FKG20-24:

This trigger can be controlled in facethome:

• Experiment LI20
• FKG20-24 level 35
• Triggers
• Front panel → No 2

Readout and DAQ

EPICS

The delays are tuned such that pulse trigger input and signal are ~300 ns apart (see picture below: pulse trigger in (yellow), dig. trigger in (blue), LED signal ch0 (magenta).

Image Added

For the beam trigger the delays are tuned such that trigger input and signal are ~ ns apart (see picture below: pulse trigger in (yellow), dig. trigger in (blue), LED signal ch0 (magenta).

The beam trigger is delayed via the gate and 5*20 m BNC cable (~5*100 ns) by ~1700 ns. The trigger signals for TR0/1 is delayed by aditional ~5 ns and is 10 ns long.

The beam trigger is received at the VME crate in FKG20-24:

This trigger can be controlled in facethome:

• Experiment LI20
• FKG20-24 level 35
• Triggers
• Front panel → No 2

Readout and DAQ

EPICS

• toto find process variables: $ eget -ts ds -a name ACSW:LI20:NW06:13%
  • eget -ts ds -a name is the command to search
  • ACSW:LI20:NW06:13 is the known part
  • % is the wildcard
• the other (slower) option is $ findpv ACSW:LI20:NW06:13%

•  the data directory is facet-srv20:/nas/nas-li20-pm00/T618/epicsData/ before transferring the EPICS data to our machine

FACET DAQ

• can be started via: facethome → Physics Apps... → FACET DAQ
• scan functions are saved in: /usr/local/facet/tools/matlabTNG/F2_DAQ/scanFunc_XYZ.m
• an example scan for the vertical position scan is: scanFunc_luxe320_Y.m
• darkening of the SiPM can probably not be scanned since it uses two EPICS addresses (POWERON and POWEROFF)

Computer

The network of our machines looks as follows:

Image Added

• gateway computer flaci@cpugateway computer flaci@cpu-li20-sp01, accessible from mcclogin, facet-srv*, or thinsrv01
• digitizer computer luxelab@192.168.0.2 (in FKG20-24), accessible only from cpu-li20-sp01 (in FKG20-01)
  
  • ssh flaci@cpu-li-sp01
  • ssh luxelab@192.168.0.2

Setup the Tunnel between luxelab and facet-srv20

An example configuration file for the permanent tunnel is here:

It also includes an example of a proxy jump. This can be applied the same way to go directly into facet-control. There are a few things to keep in mind. In principle the proxyjumps can be chained together through all possible machines. The identity files to get into any machine has to be the one from the start point, i.e., the public key of the used key from facet-control has to be in the known-hosts of the DAQ machine. Another important point is that when login into the SLAC gateway (centos7 or fastx3), a few things happen in the background (e.g. Kerberos, home setup). Therefore, login from the local machine into facet-control (or further) may be a problem. Our solution is to ssh into the SLAC gateway and from there anything is possible directly with the proper setup.

Setup the Tunnel between luxelab and facet-srv20

• set up the SSH tunnel between the luxelab computer and the facet-srv20 we use the following command:
  
• set up the SSH tunnel between the luxelab computer and the facet-srv20 we use the following command:
  
  • [fphysics@facet-srv20 ~/ivoschul ]$ ssh -F .ssh/config -N -f luxelab
  • -F defines the config file which is not at the default location, it defines
    • the users, key-pair, IPs, and proxyjump
    • RemoteForward 62882 localhost:62882
    • LocalForward 62883 localhost:62883
  • -N makes the session non-interactive
  • -f puts the session in the background
• to kill the session we can use the following:

  • [fphysics@facet-srv20 ~/ivoschul ]$ ps aux | grep 'ssh -F .ssh/config'

  • this will output three processes, one is the grep command and two are the ssh connections to the gate and to the luxelab
  • get the process ID of the connection to the gate and kill it with [fphysics@facet-srv20 ~/ivoschul ]$ kill processID

• port usage:
  • 62884 → Ivo's jupyter server on facet-srv20
  • 62885 → daq server on luxelab
  • 62886 → Antonio's jupyter server on luxelab

Machine/Beam Trigger

1. open facethome (from the control network)
2. navigate to the Experiment - LI20 section
3. select the Triggers... menu in the FKG20-24 section
4. our trigger is channel 2 in the front panel section, called trigger FP2
5. we use the following settings
   
   1. beam trigger (coming from trigger RP2)
   2. normal polarity
   3. 1000 ns width
   4. 10 ns delay

AC Power Switch

• ACSW:LI20:NW06:13XXX
• ACSW:LI20:NW06:14XXX
• ACSW:LI20:NW06:15XXX

Time schedule and planning

Pre-PAMM

•  Fix EPICS motor control
  
•  Create power switch (darkening) control
•  Ivo Schulthess Create temperature logger control
•  Test detector movements
•  Test setup with LED
•  Remove box1 (and cabling of it)
•  Antonios Athanassiadis Documentation of commissioning
•  Straw alignment with respect to rotation axis
•  Count signal cables 

PAMM 06/04

•  Turn of XPS controller
•  Bring detector into tunnel
•  Align detector on optical table (make notes!!!)
•  Connect and test motor controller
•  Define detectors parking position
•  Connect and test power switches for darkening
•  Place, connect, and test the arduino (above table?)
•  Connect trigger, communication, and signal cables
•  E320 patch panel has common ground?
•  Measure motor position for being in the beam
  
•  Measure distance beam window to straws/rotational axis 
•  Set up trigger delays
•  Test setup with LED
  
•  Selfie with detector
•  Foto Arduino front panel
•  One fiber calibration
•  LED Voltage/SiPM gain scan
•  fight for PP-06 J16 and Spencers channel

Post-PAMM (Pre-Beamtime)

•  Write EPICS / CAEN communication
•  Create runlist planning
•  Test CAEN saving on stop
•  Test CAEN saving every N event
•  Create main control script
•  Extend quickDraw script
•  Tidy up computer, put everything in one directory
•  ctrl-C proper disconnect
•  Update trigger scheme
•   

...

• • ssh -F .ssh/config -N -f luxelab
  • -F defines the config file which is not at the default location, it defines
    • the users, key-pair, IPs, and proxyjump
    • RemoteForward 62882 localhost:62882
    • RemoteForward 62883 localhost:62883
  • -N makes the session non-interactive
  • -f puts the session in the background
• to kill the session we can use the following:

  • [fphysics@facet-srv20 ~/ivoschul ]$ ps aux | grep 'ssh -F .ssh/config'

  • this will output three processes, one is the grep command and two are the ssh connections to the gate and to the luxelab
  • get the process ID of the connection to the gate and kill it with [fphysics@facet-srv20 ~/ivoschul ]$ kill processID

Run the Time Server

• create the tunnel between luxelab and facet-srv20 with the proper port forwarding (see above)
• log in to facet-srv20
• create or reattach the screen session (usually called ivoschul_timeSrv)
• run $ python /home/fphysics/ivoschul/timeServer.py
• log in to luxelab
• run $ /home/luxelab/LUXE320/control/timeClient/build/timeClient
• it should print the server and client times

Port Usage

• 62882 / 62883 → ports between luxelab and facet-srv20
• 62884 → Ivo's jupyter server on facet-srv20
• 62885 → daq server on luxelab
• 62886 → Antonio's jupyter server on luxelab

Data Backup

• data backup is triggered via crontab
  • $ crontab -l → lists all active crontabs
  • $ crontab -e → allows to change the crontabs
  • our crontab entry is
    • 5 * * * * /home/luxelab/LUXE320/dataBackup.sh
    • 5 * * * *  → run every 5 minutes
    • /home/luxelab/LUXE320/dataBackup.sh → shell script that does get executed
• the backup script is dataBackup.sh contains

  • rsync -av --log-file=/home/luxelab/LUXE320/rsync.log /home/luxelab/LUXE320/data/ /media/luxelab/data1/backup_LUXE320/

  • -a → archive (fundamental backup option)
  • -v → verbose
  • --log-file → the verbose output is sent to rsync.log
  • source:  /home/luxelab/LUXE320/data/
  • destination: /media/luxelab/data1/backup_LUXE320/

Synchronization

One issue with the setup is that the digitizer DAQ is not in the FACET network. Therefore, it has no synchronization of the events. Various options exist to synchronize the data. One option is using timestamps on both systems and synchronization via some protocol. This was not successful so far. The other option is to use the data itself.

Timestamp: Poll EPICS Timestamp into DAQ Data)

One idea is to implement a socket client into the local DAQ, and whenever it gets an event, it requests an EPICS timestamp from the server running on facet-control. This can be saved to the data and later be used for synchronization. In the socketCOM.zip is an example of such code. First, a tunnel with the open port 62882 has to be set up between facet-srv and luxelab (in our case). Then, the server.py can be run on facet-srv (in a screen shell, for example). On luxelab (the client), the client script can be run. One example in python and one in C++ is available. When executed they connect to the server, request some data from EPICS, and disconnect again. This code can be included in the DAQ software.

Timestamp: Poll DAQ Timestamp into DAQ Data)

TBW

Data: Charge/Position Correlation

One idea is to use the charge and/or position dependence of a signal in both systems. This could be a BPM, toroid, or camera data from the EPICS side and signal amplitude in a straw on the LUXE EDS side. The events can be shifted such that the correlation between the two test variables is maximized.

Data: Beam Background

The FACET DAQ provides the option, to perform a beam background measurement before a run/scan. In that case, the beam is blocked by the MPS shutter (Brendan, slack, 06/12) for some events. This gives a clear start signal.

DAta: Pockel Cell

The Pockel cell can control the electron beam on an event basis. The corresponding EPICS variable is TRIG:LT10:LS04:TCTL (Spencer, email, 06/16). The idea is that at the beginning of a run, the cell is activated/deactivated in a specific pattern that allows the events to be matched based on that pattern.

Machine/Beam Trigger

1. open facethome (from the control network)
2. navigate to the Experiment - LI20 section
3. select the Triggers... menu in the FKG20-24 section
4. our trigger is channel 2 in the front panel section, called trigger FP2
5. we use the following settings
   
   1. beam trigger (coming from trigger RP2)
   2. normal polarity
   3. 1000 ns width
   4. 10 ns delay

AC Power Switch

• ACSW:LI20:NW06:13XXX
• ACSW:LI20:NW06:14XXX
• ACSW:LI20:NW06:15XXX

DAQ settings

Time schedule and planning

Pre-PAMM

•  Fix EPICS motor control
  
•  Create power switch (darkening) control
•  Ivo Schulthess Create temperature logger control
•  Test detector movements
•  Test setup with LED
•  Remove box1 (and cabling of it)
•  Antonios Athanassiadis Documentation of commissioning
•  Straw alignment with respect to rotation axis
•  Count signal cables 

PAMM 06/04

•  Turn of XPS controller
•  Bring detector into tunnel
•  Align detector on optical table (make notes!!!)
•  Connect and test motor controller
•  Define detectors parking position
•  Connect and test power switches for darkening
•  Place, connect, and test the arduino (above table?)
•  Connect trigger, communication, and signal cables
•  E320 patch panel has common ground?
•  Measure motor position for being in the beam
  
•  Measure distance beam window to straws/rotational axis 
•  Set up trigger delays
•  Test setup with LED
  
•  Selfie with detector
•  Foto Arduino front panel
•  One fiber calibration
•  LED Voltage/SiPM gain scan
•  fight for PP-06 J16 and Spencers channel

Post-PAMM (Pre-Beamtime)

•  Write EPICS / CAEN communication
•  Create runlist planning
•  Test CAEN saving on stop
•  Test CAEN saving every N event
•  Create main control script
•  Extend quickDraw script
•  Tidy up computer, put everything in one directory
•  ctrl-C proper disconnect
•  Update trigger scheme
•   

Measurements

Move-in procedure

1. SiPM bias at 0 V
2. Motor homing (0, 0, 0), out of beam
3. Set dipole to 10 GeV 'LI20:LGPS:3330:BACT'
4. Park beam at TD11
5. Move:
   1. x = 194 → beam in the center of straws
   2. y = 14 → beam axis 30 mm above Al profile, electrons between Al profile and straws
   3. r = 27 → perpendicular to beam
6. De-park beam
7. SiPM bias at 42 V

Move-out procedure

1. SiPM bias at 0 V
2. Park beam at TD11
3. Move home before end switches:
   1. r = 2
   2. y = 5
   3. x = 5
4. De-park beam

Measurements

Move-in procedure

1. SiPM bias at 0 V
2. Motor homing (0, 0, 0), out of beam
3. Check the dipole spectrometer setting
4. Park beam at TD11
5. Move:
6. x = 194
7. y = 114
r = 27