T-618: A project of E320 and LUXE (DESY) probing the LUXE Cherenkov detector concept at FACET-II
Table of Contents
| Table of Contents | ||||||
|---|---|---|---|---|---|---|
|
Description of the Experiment
...
- gateway computer flaci@cpu-li20-sp01, accessible from mcclogin, facet-srv*, or thinsrv01
- digitizer computer luxelab@192.168.0.2 (in FKG20-24), accessible only from cpu-li20-sp01 (in FKG20-01)
- ssh flaci@cpu-li-sp01
- ssh luxelab@192.168.0.2
...
An example configuration file for the permanent tunnel is here:
| view |
|---|
| -file |
|---|
- set up the SSH tunnel between the luxelab computer and the facet-srv20 we use the following command:
- [fphysics@facet-srv20 ~/ivoschul ]$ ssh -F .ssh/config -N -f luxelab
- -F defines the config file which is not at the default location, it defines
- the users, key-pair, IPs, and proxyjump
- RemoteForward 62882 localhost:62882
- RemoteForward 62883 localhost:62883
- -N makes the session non-interactive
- -f puts the session in the background
- to kill the session we can use the following:
[fphysics@facet-srv20 ~/ivoschul ]$ ps aux | grep 'ssh -F .ssh/config'
- this will output three processes, one is the grep command and two are the ssh connections to the gate and to the luxelab
- get the process ID of the connection to the gate and kill it with [fphysics@facet-srv20 ~/ivoschul ]$ kill processID
Run the Time Server
- create the tunnel between luxelab and facet-srv20 with the proper port forwarding (see above)
- log in to facet-srv20
- create or reattach the screen session (usually called ivoschul_timeSrv)
- run
$ python /home/fphysics/ivoschul/timeServer.py - log in to luxelab
- run
$ /home/luxelab/LUXE320/control/timeClient/build/timeClient - it should print the server and client times
Port Usage
- 62882 / 62883 → ports between luxelab and facet-srv20
- 62884 → Ivo's jupyter server on facet-srv20
- 62885 → daq server on luxelab
- 62886 → Antonio's jupyter server on luxelab
Data Backup
- data backup is triggered via crontab
- $ crontab -l → lists all active crontabs
- $ crontab -e → allows to change the crontabs
- our crontab entry is
- 5 * * * * /home/luxelab/LUXE320/dataBackup.sh
- 5 * * * * → run every 5 minutes
- /home/luxelab/LUXE320/dataBackup.sh → shell script that does get executed
- the backup script is dataBackup.sh contains
rsync -av --log-file=/home/luxelab/LUXE320/rsync.log /home/luxelab/LUXE320/data/ /media/luxelab/data1/backup_LUXE320/
- -a → archive (fundamental backup option)
- -v → verbose
- --log-file → the verbose output is sent to rsync.log
- source: /home/luxelab/LUXE320/data/
- destination: /media/luxelab/data1/backup_LUXE320/
Synchronization
One issue with the setup is, that the digitizer DAQ is not in the FACET network. Therefore, it has no synchronization of the events. Various options exist to synchronize the data. One option is using timestamps on both systems and synchronization via some protocol. This was not successful so far. The other option is to use the data itself.
|
It also includes an example of a proxy jump. This can be applied the same way to go directly into facet-control. There are a few things to keep in mind. In principle the proxyjumps can be chained together through all possible machines. The identity files to get into any machine has to be the one from the start point, i.e., the public key of the used key from facet-control has to be in the known-hosts of the DAQ machine. Another important point is that when login into the SLAC gateway (centos7 or fastx3), a few things happen in the background (e.g. Kerberos, home setup). Therefore, login from the local machine into facet-control (or further) may be a problem. Our solution is to ssh into the SLAC gateway and from there anything is possible directly with the proper setup.
Setup the Tunnel between luxelab and facet-srv20
- set up the SSH tunnel between the luxelab computer and the facet-srv20 we use the following command:
- [fphysics@facet-srv20 ~/ivoschul ]$ ssh -F .ssh/config -N -f luxelab
- -F defines the config file which is not at the default location, it defines
- the users, key-pair, IPs, and proxyjump
- RemoteForward 62882 localhost:62882
- RemoteForward 62883 localhost:62883
- -N makes the session non-interactive
- -f puts the session in the background
- to kill the session we can use the following:
[fphysics@facet-srv20 ~/ivoschul ]$ ps aux | grep 'ssh -F .ssh/config'
- this will output three processes, one is the grep command and two are the ssh connections to the gate and to the luxelab
- get the process ID of the connection to the gate and kill it with [fphysics@facet-srv20 ~/ivoschul ]$ kill processID
Run the Time Server
- create the tunnel between luxelab and facet-srv20 with the proper port forwarding (see above)
- log in to facet-srv20
- create or reattach the screen session (usually called ivoschul_timeSrv)
- run
$ python /home/fphysics/ivoschul/timeServer.py - log in to luxelab
- run
$ /home/luxelab/LUXE320/control/timeClient/build/timeClient - it should print the server and client times
Port Usage
- 62882 / 62883 → ports between luxelab and facet-srv20
- 62884 → Ivo's jupyter server on facet-srv20
- 62885 → daq server on luxelab
- 62886 → Antonio's jupyter server on luxelab
Data Backup
- data backup is triggered via crontab
- $ crontab -l → lists all active crontabs
- $ crontab -e → allows to change the crontabs
- our crontab entry is
- 5 * * * * /home/luxelab/LUXE320/dataBackup.sh
- 5 * * * * → run every 5 minutes
- /home/luxelab/LUXE320/dataBackup.sh → shell script that does get executed
- the backup script is dataBackup.sh contains
rsync -av --log-file=/home/luxelab/LUXE320/rsync.log /home/luxelab/LUXE320/data/ /media/luxelab/data1/backup_LUXE320/
- -a → archive (fundamental backup option)
- -v → verbose
- --log-file → the verbose output is sent to rsync.log
- source: /home/luxelab/LUXE320/data/
- destination: /media/luxelab/data1/backup_LUXE320/
Synchronization
One issue with the setup is that the digitizer DAQ is not in the FACET network. Therefore, it has no synchronization of the events. Various options exist to synchronize the data. One option is using timestamps on both systems and synchronization via some protocol. This was not successful so far. The other option is to use the data itself.
Timestamp: Poll EPICS Timestamp into DAQ Data)
One idea is to implement a socket client into the local DAQ, and whenever it gets an event, it requests an EPICS timestamp from the server running on facet-control. This can be saved to the data and later be used for synchronization. In the socketCOM.zip is an example of such code. First, a tunnel with the open port 62882 has to be set up between facet-srv and luxelab (in our case). Then, the server.py can be run on facet-srv (in a screen shell, for example). On luxelab (the client), the client script can be run. One example in python and one in C++ is available. When executed they connect to the server, request some data from EPICS, and disconnect again. This code can be included in the DAQ software.
| View file | ||||
|---|---|---|---|---|
|
Timestamp: Poll DAQ Timestamp into DAQ Data)
TBW
Data: Charge/Position Correlation
One idea is to use the charge and/or position dependence of a signal in both systems. This could be a BPM, toroid, or camera data from the EPICS side and signal amplitude in a straw on the LUXE EDS side. The events can be shifted such that the correlation between the two test variables is maximized.
Data: Beam Background
The FACET DAQ provides the option, to perform a beam background measurement before a run/scan. In that case, the beam is blocked by the MPS shutter (Brendan, slack, 06/12) for some events. This gives a clear start signal.
DAta: Pockel Cell
The Pockel cell can control the electron beam on an event basis. The corresponding EPICS variable is TRIG:LT10:LS04:TCTL (Spencer, email, 06/16). The idea is that at the beginning of a run, the cell is activated/deactivated in a specific pattern that allows the events to be matched based on that pattern.
...