...
Specific for SLAC's test-Optopanel will be marked in red .
Mechanics:
- Up to 8x Optoboards are housed in one Optobox. One Optobox consists also of:
- 1 Powerboard: hosts 5 bPOL12V ASICs (conversion 9 V to 2.5 V) and 1 MOPS chip
- 1 Connectorboard: handles power distribution form the 5 bPOL12V to the 8 Optoboards according the SP chain powering/detector layout.
- Up to 28x Optoboxes (14x normal, 14x mirrored) are housed in one Optopanel.
- Each side of the ITk has 4 Optopanels.
Currently specific for SLAC IS demonstrator:
2x Optoboard V2.1 are housed in one Optobox. We have 1 v4 optoboard. This Optobox is housed in a Test-Optopanel.
...
Bern collects support requests and general questions on our Mattermost channel Bern-Optoboard (invite link). Your contacts as of Aug. 2022 are Aaron O'Neill and Daniele del Santo.
FELIX, optoboard software, and YARR setup:
FELIX firmware (see atlas-project-felix.web.cern.ch for more information):
...
- felix:
- felix2: software git commit 73d2d72a of master (compatible with the latest firmware). Note that for local compilation you need to use
source cmake_tdaq/bin/setup.sh x86_64-el9-gcc13-opt.
Microservices software for optoboard, developed by Bern: https://gitlab.cern.ch/bat/optoboard_felix. A lot is documented in the readme and wiki of the repository.
...
During the startup of the digital calibration, the module configuration would increase the ITkPixV1.0 SCC current to 4.38A (from, 3.07A). The calibration results are kept at /home/itkpix/YARR-FELIX/data/. Successful calibration should have an output Root file.
Test-Optopanel at SLAC:
The test-Optopanel houses one Optobox with 4 Optoboards. It has a twinax inlet (round) and a fibre outlet (rectangle). The Optobox is mounted on a cooling plate with two 8 mm pipes.
...
Current firmware allows only 2 lpGBTs to be readout, such that the link alignment string is fixed to x0000000300000000 for channels 00 and 01 in unaligned state. Alignment can be checked using data from module RX 0 (L0 on miniDP-SMA) after module configuration, giving the following link alignment strings independently of which e-links are subscribed on felixcore startup (for optoboard 2400006):
| miniDP-SMA Channel | Alignment String Name | Value |
|---|---|---|
| L1 UPLINK 00 | DECODING_LINK_ALIGNED_00 | 0x000000030000001 |
| L1 UPLINK 01 | DECODING_LINK_ALIGNED_00 | 0x000000030000010 |
| L1 UPLINK 02 | DECODING_LINK_ALIGNED_00 | 0x000000030000000 |
| L1 UPLINK 03 | DECODING_LINK_ALIGNED_00 | 0x000000030001000 |
| L1 UPLINK 04 | DECODING_LINK_ALIGNED_00 | 0x000000030010000 |
| L1 UPLINK 05 | DECODING_LINK_ALIGNED_00 | 0x000000030100000 |
| L2 UPLINK 06 | DECODING_LINK_ALIGNED_01 | 0x000000030000001 |
| L2 UPLINK 07 | DECODING_LINK_ALIGNED_01 | 0x000000030000010 |
| L2 UPLINK 08 | DECODING_LINK_ALIGNED_01 | 0x000000030000000 |
| L2 UPLINK 09 | DECODING_LINK_ALIGNED_01 | 0x000000030001000 |
| L2 UPLINK 10 | DECODING_LINK_ALIGNED_01 | 0x000000030010000 |
| L2 UPLINK 11 | DECODING_LINK_ALIGNED_01 | 0x000000030100000 |
There is an unstable or broken lane on the optoboard corresponding to L1 UPLINK 02, which is not present in optoboard 2400011 (currently installed). Note that alignment is sensitive to polarity of TX connection (i.e. NP↔NP vs NP↔PN) but not to the RX connections.
ITkPixV1.1 Digital Quad
The ITkPixV1.1 digital quad can be read out in two different configurations, either interfaced through the PP0+Type 0 ring (more details of the hardware setup can be found on the ITkPixV1 quad module readout page).
Interface with ring + PP0
At present the full chain including PP0 and Type 0 ring cannot be read out using FELIX. The following channel mapping is used in the connectivity file. Chips 2 and 4 send data on power-up. Chip 1 has poor data transmission quality on the negative polarity side, but is not observed using the LBL adapter card indicating that this is a feature of the ring+PP0.
| Mini-DP SMA Channel | Rx Lane | Chip ID | Chip Number |
|---|---|---|---|
| L0 | 12 | 0x15468 | 3 |
| L1 | 8 | 0x15448 | 4 |
| L2 | 4 | 0x15429 | 2 |
| L3 | 0 | 0x15428 | 1 |
Interface with LBL adapter board
Testing using the following configuration files (set in the alias mconf_quad):
connectivity: /home/itkpix/YARRdev/configs_common/connectivity/20UPGR91101015/20UPGR91101015_L2_warm.json (same as used for YARR readout)
controller: /home/itkpix/YARRdev/configs_common/controller/felix_rd53b.json
The downlink differential signal polarity should not be inverted (i.e. P↔P and N↔N SMA connections between DP and Optoboard BOB), as this is only required due to the ring channel mapping. L3,L2,L1,L0 are connected to EFF-SMA board UPLINK 00,01,02,04. Before configuration, alignment bit is unstable on UPLINK 01 (chip 2) and always 0 for other channels, but after configuration all alignment bits are zero. This is similar behavior as observed with the PP0+Ring.
| Mini-DP SMA Channel | Rx Lane | Chip ID | Chip Number |
|---|---|---|---|
| L0 | 0 | 0x15448 | 4 |
| L1 | 4 | 0x15428 | 1 |
| L2 | 8 | 0x15429 | 2 |
| L3 | 12 | 0x15468 | 3 |
Running Vivado ILA on FELIX:
The Integrated Logic Analyzer (ILA) is a feature of vivado that allows registers in ILA cores of an FPGA to be read out while in use. It is a powerful tool for debugging DAQ setups with Felix. The Felix card now has /afs/ mounted and can run release v2019.1 of vivado with a license from TID. To bring up the vivado GUI, do:
...
- LinkData: 224 bits (need to check what is contained here)
- EgroupGB1Data_dbg[i]: 36-bits. Bits 33-35 are added by the gearbox: bit 35 indicates of the header is valid and bits 33-34 are the header (should be 01 if all is well). Bit 32 indicates if the data is valid.
- LinkAligned: a single bit that indicates the alignment of the optical signal between Felix and optoboard
- DecoderAligned_ila: should be 01 if uplink is aligned
- EgroupUnscrData_dbg[0]: if you see values like 87*fffff then the UPLINK polarity is inverted (it should be d78100000)
Running Remotely with FastX
FastX can be used to connect into the SLAC network for low-latency communication with nodes in the DAQ network. Information installing FastX can be found on the FastX Confluence documentation.
After installation, you can start Terminal sessions to steer the FELIX/optoboard/YARR instances. From the bastion host, run:
| Code Block |
|---|
ssh -L8081:raspi_b84_lab_felix2:8081 rddev111 |
And connect to a browser in FastX desktop to localhost:8081. From another tab, connect to http://atlascr.slac.stanford.edu:3000 and login to Grafana with
user: admin
pwd: ##########
You can also open terminal windows in the FastX browser so that they stay alive when you close FastX. You will need to tunnel into itkpix@felix via <user>@rddev111.slac.stanford.edu.
GBCR Documentations
- GBCR PDR SPR and PDR (2019): Indico (Sep/12/2019), Documentation collection and report (Nov/3/2019)
- GBCR2 Specification document (register details) Nov/2019
- GBCR Testing Github
- GBCR status Jan/2023 (Jingo Ye)
...
| Register | Value | Comments |
|---|---|---|
| dllEnable | True | Enable DLL clock for for retiming |
| dllCapReset | False | |
| dllForceDown | False | DLL can have harmonic lock problem which would need dllForceDown=T to release it |
| dllChargePumpCurrent | 0x0 | Set charge injection current to min value. Higher current enables faster locking but may cause larger jitter. We care less about locking time but care more about jitter. |
| dllClockDelay | scan | Scan this delay for the active channel for good BER window bathtub. |
Vakhtang's readout software
Vakhtang Tsiskaridze (VT) wrote a standalone python package to perform digital scans outside of YARR. Use the following steps to run on felix.
...
- Optoboard System Documentation
- Current Opto Panel map (Oct/2024)
- FELIX JIRA (Oct/22) on optoboard + ITkPix readout setup
- CERN mattermost Bern-Optoboard channel
- Talk (Dec/9/2022) by Angira Rastogi on Optoboard-FELIX setup at LBNL
...