The GT Readout Platform aims to provide a readout for use with new pixel detectors developed at SLAC (≥ 2023), which use high-speed (> 1 Gbit/s) gigabit transceivers for their data output. See for example the SparkPix-IO prototype.
It consists of three main parts (shown in the block diagram below):
The sections below describe the platform in more detail. Use the table of contents below to quickly find a specific section you might be looking for or use one of the useful resources on the right for quick access.
Table of contents
Useful resources
Jira issues in progress
The tasks and questions below should be looked at as soon as possible since they might be blocking
Requirement | ePixUHR | SparkPix-S | SparkPix-ED |
---|---|---|---|
Frame rate | 100kfps | 1Mfps | 1Mfps |
Power supplies | 2.5V Analog 1.3V (AS/DS/IO) | 2.5V Analog 1.3V (AS/DS/IO) 0.6V (Current sink!) | 1.3V (AS/DS/IO) |
Power for each supply | ePixUHR - 35 kHz | SparkPix-S: supply/ground and power consumption | t.b.d. |
Number of GT IOs per ASIC | 8 (outputs) | 8 (outputs) 1 clock in | t.b.d (The current agreement is to have 8 outputs) |
Expected I/O speed | 5.25 Gbit/s | 5.25 Gbit/s | 10 Gbit/s |
Total data bandwidth | 42 Gbit/s | 42 Gbit/s | 80 Gbit/s |
There are three targeted cameras for this project:
2x2 ePix/SparkPix | 1M ePix | 2M SparkPix S | |||
Parameter (estimated) | Small Camera | Small Camera | Super tile | Super tile | Quad Camera |
Pixels | 129,024 px | 540,672 px | 967,680 | 1,161,216 px | 2,162,688 px |
Rate | 35kHz / 100kHz | 1MHz | 35kHz / 100kHz | 35kHz / 100kHz | 1MHz |
Focal Plane Area | 4cm x 4cm | 4cm x 4cm | 12cm x 10cm | 12cm x 12cm | 8cm x 8cm |
Front side footprint (window) | 5cm x 5cm | 5cm x 5cm | 14cm x 12cm | 14cm x 14cm | 10cm x 10cm |
Power (only ASIC) | 0.016 kW/??? | 0.021kW | 0.130 kW/??? | 0.144 kW/??? | 0.084 kW |
Weight | 1.5kg | 1.5kg | 9Kg | 10kg | 6kg |
Data volume | 56 Gbit/s/ 160 Gbit/s | 160 Gbit/s | 420 Gbit/s / 1190 Gbit/s | 504 Gbit/s / 1440 Gbit/s | 640 Gbit/s |
ASIC Power Requirement | Analog Section | Digital Section | I/O Section | 0.6V Sink | Analog TPS | |||||
---|---|---|---|---|---|---|---|---|---|---|
ePixUHR 140k 2x2 Detector | SparkPix-S 500k 2x2 Detector | ePixUHR 140k 2x2 Detector | SparkPix-S 500k 2x2 Detector | ePixUHR 140k 2x2 Detector | SparkPix-S 500k 2x2 Detector | ePixUHR 140k 2x2 Detector | SparkPix-S 500k 2x2 Detector | ePixUHR 140k 2x2 Detector | SparkPix-S 500k 2x2 Detector | |
Voltage | 1.3 V | 1.3V | 1.3V | 1.3V | 1.3V | 1.3V | ??? Maybe | 0.6 V | 2.5 V | 2.5V |
Required current | 10A (= 2.5 A* 4 ASIC) | 13.4 A (= 3.35A * 4 ASIC) | - Old digital design: ???? | - Old digital design ???? | 1.6 A [1RX, 8TX, 8serializer and 2cm clkspine : ~ 317mA] | ??? (If existing lower or equal than SparkPixS) | -8 A | 0.4 A (=0.1 * 4 ASIC) | ||
System Requirement | +1.3 V @ +17.5 A (Adding +30% current for PVT variation) | +1.3 V @ +3 A (Adding +30% current for PVT variation) [waiting for the new digital design] | +1.3 V @ +2.5 A (Adding +30% current for PVT variation) | +0.6 V @ -11 A This current is not provided by the LDO. But it passes through it. (Adding +30% current for PVT variation) | +2.5 V @ +0.5 A (Adding +30% current for PVT variation) |
ePixUHR 140k 2x2 Detector Specs | ePixUHR 1.1M 6x6 Detector Specs | ePixUHR 1M 6x5 Detector Specs | SparkPix-S 500k 2x2 Detector Specs | SparkPix-S 2M 4x4 Detector Specs | KU15P (-A1760) Kintex Ultrascale+ | KU15P (-A1156) Kintex Ultrascale+ FPGA USED IN ePixHR250M | KU15P (-E1517) Kintex Ultrascale+ | XCVU160 (-C2104) Virtex Ultrascale | XCVU190 (-A2577) Virtex Ultrascale | VU13P (-A2577) Virtex Ultrascale+ | |
---|---|---|---|---|---|---|---|---|---|---|---|
General IO (HD, HP) | 96 HD, 416 HP | 48 HD, 486 HP | 96 HD, 416 HP | 52 HD, 364 HP | 0 HD, 448 HP | 0 HD, 448 HP | |||||
High Speed GTs (GTH/GTY) | - ASIC data: 32 = 8 lanes * 4 ASIC - Spare outputs : 4 - PGP communication: 12 = 12* 160 Gbit/s / 275Gbit/s (1 Amphenol Transceiver) Total: 48 High Speed GTs | - ASIC data: 288 = 8 lanes * 36 ASIC - Spare outputs : 0 - PGP communication: 72 = 12* 1.44 Tbit/s / 275Gbit/s (6 Amphenol Transceivers) Total: 360 High Speed GTs | - ASIC data: 240 = 8 lanes * 30 ASIC - Spare outputs : 0 - PGP communication: 72 = 12 lanes * 1.19 Tbit/s / 275Gbit/s (6 Amphenol Transceivers) Suggested 3 transceivers 1.4x compression in the detector Total: 312 High Speed GTs (If considering 5x2 Modules, 104 GTs each) | - ASIC data: 32 = 8 lanes * 4 ASIC - Spare outputs : 4 - PGP communication: 12 = 12* 160 Gbit/s / 275Gbit/s (1 Amphenol Transceivers) Total: 48 High Speed GTs | - ASIC data: 128 = 8 lanes * 16 ASIC - Spare outputs : 0 - PGP communication: 24* = 12* 495 Gbit/s / 275Gbit/s (2 Amphenol Transceivers) Total: 152 High Speed GTs | 76 (44 GTH/32 GTY) | 28 | 56 (32 GTH/24 GTY) | 104 (52 GTH/52 GTY) | 120 (60 GTH/60 GTY) | 128 (0 GTH/128 GTY) |
Total Block RAM | 34.6 Mb | 34.6 Mb | 34.6 Mb | 115.2 Mb | 132.9 Mb | 94.5 Mb | |||||
UltraRam, HBM | 36 Mb, None | 36 Mb, None | 36 Mb, None | None, None | None, None | 360 Mb, None | |||||
Transceiver Speed (GTH, GTY) | > 10 Gbit/s | > 10 Gbit/s | > 10 Gbit/s | > 10 Gbit/s | > 10 Gbit/s | GTH 16.3 Gbit/s GTY 32.75 Gbit/s Transceivers | GTH 16.3 Gbit/s GTY 16.3 Gbit/s Transceivers | GTH 16.3 Gbit/s GTY 32.75 Gbit/s Transceivers | GTH 16.3 Gbit/s GTY 30.5 Gbit/s Transceivers | GTH 16.3 Gbit/s GTY 30.5 Gbit/s Transceivers | GTY 32.75 Gbit/s Transceivers |
Size | The PCB width is (preferably) 65 mm (2.56’’) | 42.5 x 42.5 mm | 35 x 35 mm | 40 x 40 mm | 47.5x47.5 mm | 52.5 x 52.5 mm | 52.5 x 52.5 mm | ||||
Cost | 6-10 k$ | 5-9 k$ | 6-10k$ | 40 k$ | 50-70 k$ | 60-110 k$ | |||||
Comments | This is fine for the 2x2 Systems. This is fine for the SparkPix-S 4x4 | The number of GTs in this FPGA does not fit any of the cameras we are targetting | This is fine for the 2x2 Systems. For the larger systems we need more than 3 FPGAs | This is fine for the 2x2 Systems. | This is fine for the 2x2 Systems (assuming we can fit the real estate). | This is fine for the 2x2 Systems.(assuming we can fit the real estate) |
*Done considering 1% Occupancy instead of maxing out the transceivers
UHR 2x2 | SparkPix S 2x2 | SparkPix S 4x4 | UHR 5x6 | UHR 6x6 | ||
---|---|---|---|---|---|---|
Requirements Characteristics | 48 GTs | 48 GTs | 152 GTs | 312 GTs | 360 GTs | |
KU15P (-A1156) Kintex U+ | 28 GTs / 352 mm2 / 10k$ | ❌ | ❌ | ❌ | ❌ | ❌ |
KU15P (-E1517) Kintex U+ | 56 GTs / 402 mm2 / 10k$ | ✅ | ✅ | ❌ | ❌ | ❌ |
KU15P (-A1760) Kintex U+ | 76 GTs / 42.52 mm2 / 10k$ | ✅ | ✅ | ✅ (2 FPGA) | ❌ | ❌ |
XCVU160 (-C2104) Virtex U | 104 GTs / 47.52 mm2 / 40k$ | ✅ | ✅ | ✅ (2 FPGA) | ✅ (1 FPGA/module) | ❌ |
XCVU190 (-A2577) Virtex U | 120 GTs / 47.52 mm2 / 70k$ | ✅ | ✅ | ✅(2 FPGA) | ✅ (1 FPGA/module) | ✅ (1 FPGA/module) |
VU13P (-A2577) Virtex U+ | 128 GTs / 52.52 mm2 / 110k$ | ✅ | ✅ | ✅(2 FPGA) | ✅ (1 FPGA/module) | ✅ (1 FPGA/module) |
From Kintex UltraScale+ FPGAs Data Sheet: DC and AC Switching Characteristics (DS922)
Name | Voltage |
---|---|
VCCINT | 0.85V |
VCCINT_IO | 0.85V |
VCCBRAM | 0.85V |
VCCAUX | 1.8V |
VCCAUX_IO | 1.8V |
MGTVCCAUX_LN/LS/RN/RS | 1.8V |
VCCADC | 1.8V |
MGTAVCC_LN/LS/RN/RS | 0.9V |
MGTAVTT_LN/LS/RN/RS | 1.2V |
Requirement | Parameters | Notes |
---|---|---|
Power supply | 24V consistent with the HR detector | |
Mechanical size | We would like to match the ePixHRM board dimensions to reuse cooling Side entrance detector
| |
Digital board | 2.56x5.265" | |
Power and communication | 2.56x5.240" | |
Carrier | 2.56x1.95" |
|
Board | Domain | Part | Final Voltages | |||||||
---|---|---|---|---|---|---|---|---|---|---|
Analog | Analog | ASIC | A0VDD_P1V3 1.3V @4.4A | ← +1.3V | LT3086 x3 (LDO) Max 6.3A | ← +1.8V Max 1.8V*6.3A*4 ≈ 45W | LT8638S x2 42V, 10A Synchronous Step-Down Silent Switcher 2 Max Current = 10*2 = 20A (Around 93% efficiency for 24 to 1.8V at max load) Max 36 W_out, 40 W_in | ← +24 V The current drawn by the 0.6V current sink should not be counted twice since it's sourced by the G_AS. The power drawn by the ASIC analog part is 1.3V*4.4A*4= 23W. With 1.8V as the LDO input, we are also burning: (1.8-1.3)V*4.4A*4 = 9W. The power drawn by the TPS should be less than 2W. So the power that the DC/DC converters have to provide is around 23 + 9 + 2 = 34W. Considering the efficiency curves of the DC/DC converters: 34/0.85/0.93 = 43W Total Analog Power (same calculation for ePixUHR would be 32W) | ||
A1VDD_P1V3 1.3V @4.4A | ← +1.3V | LT3086 x3 (LDO) Max 6.3A | ||||||||
A2VDD_P1V3 1.3V @4.4A | ← +1.3V | LT3086 x3 (LDO) Max 6.3A | ||||||||
A3VDD_P1V3 1.3V @4.4A | ← +1.3V | LT3086 x3 (LDO) Max 6.3A | ||||||||
A0SINK_P0V6 0.6V @ -2.75A | ← +0.6V | LT3091 x2 (LDO) Max 3.0A | ← +2.5V | LT3086 (LDO) Max 2.1A | ← +5.0V Max 5.0V*2.1+5.0V*2.1+5.0V*2.1A ≈ 32W | LT8638S 42V, 10A Synchronous Step-Down Silent Switcher 2 Max Current = 10A (Around 93% efficiency for 24 to 5V at max load) Max 50 W_out, 54 W_in | HS ADC: ~350mA Slow ADC: few mA DACs: ~250 mA Total 0.7A Worst case scenario starting from 5V, 3.5W/0.85/0.93 = 4.4W | |||
A1SINK_P0V6 0.6V @ -2.75A | ← +0.6V | LT3091 x2 (LDO) Max 3.0A | ||||||||
A2SINK_P0V6 0.6V @ -2.75A | ← +0.6V | LT3091 x2 (LDO) Max 3.0A | ||||||||
A3SINK_P0V6 0.6V @ -2.75A | ← +0.6V | LT3091 x2 (LDO) Max 3.0A | ||||||||
G_AS_2V5 2.5V @ <0.5 A | ← +2.5V | LT3086 (LDO) Max 2.1A | ← +5.0V | |||||||
DAC/ADC/Misc. | P3V3A +3.3V @ <1A? 3.3 W | ← +3.3V | LT3086 (LDO) Max 2.1A | |||||||
P1V8A +1.8V @ <1A? 1.8 W | ← +1.8V | LT3086 (LDO) Max 2.1A | ||||||||
Digital | ASIC | DVDD_P1V3 1.3V @3A | ← +1.3V | LT3086 x2 (LDO) Max 4.2A | ← +1.8V Max 1.8V*4.2A+1.8V*4.2A+1.8V*2.1A ≈ 20W | LT8638S 42V, 10A Synchronous Step-Down Silent Switcher 2 Max Current = 10A (Around 93% efficiency for 24 to 1.8V at max load) Max 18 W_out, 19 W_in | ← +24 V For the digital consumption of the ASIC we do not have precise numbers regarding the new logic. Let's use the old numbers: 1.3V*3A = 4W While for the I/O voltage we have an expected: 1.3V*2.5A = 3.25 W. LDO losses: 0.5*5.5 = 2.75 W 10W/0.85/0.93 = 13W (ASIC Digital and I/O) (same calculation for ePixUHR would be 9.5W) | |||
IOVDD_P1V3 1.3V @3A | ← +1.3V | LT3086 x2 (LDO) Max 4.2A | ||||||||
ASIC_TX_DC_P0V8 0.8V @??A | ← +0.8V | LT3086 (LDO) Max 2.1A | ||||||||
DAC/ADC/Misc. | P3V3D +3.3V @ <1A? 3.3 W | ← +3.3V | LT3086 (LDO) Max 2.1A | ← +5.0V Max 5.0V*2.1+5.0V*2.1 ≈ 21W | LT8638S 42V, 10A Synchronous Step-Down Silent Switcher 2 Max Current = 10A (Around 93% efficiency for 24 to 5V at max load) Max 50 W_out, 54 W_in | |||||
P1V8D +1.8V @ <1A? 1.8 W | ← +1.8V | LT3086 (LDO) Max 2.1A | ||||||||
Board | Domain | Part | Final Voltages | |||||||
Digital | Digital | FPGA | VCCINT 0.85V @7.05 A 6 W | ← +0.85V | LMZ31520 DC/DC Buck converter 3V to 14.5V input 20A (Around 90% efficiency) Max 17 W_out, 19 W_in | ← +5 V Max 19*3+42 ≈ 99 W | LT8638S x2 42V, 10A Synchronous Step-Down Silent Switcher 2 Max Current = 10*2 = 20A (Around 93% efficiency for 24 to 5V at max load) Max 100 W_out, 110 W_in | ← +24 V Total power for FPGA is around: 6+1.3+3.3+6.6≈17W Considering a worst case efficiency of the DC/DC: | ||
VCCAUX + VCC_1.8V +VCCADC + MGTVCCAUX+ MGTYVCCAUX 1.8V @0.7A 1.3 W | ← +1.8V | TPSM5D1806 (DC/DC) 4.5V to 15V input Dual 6A output (85% efficiency for max load at Vout = 1.8V) Max 16 W_out, 19 W_in | ||||||||
MGTAVCC +MGTYAVCC 0.9V @3.7A 3.3 W | ← +0.9V | |||||||||
VCC_1.2V + MGTAVTT + MGTYAVTT 1.2V @5.5A 6.6 W | ← +1.2V | TPSM5D1806 (DC/DC) 4.5V to 15V input Parallel 12A output (Between 80% and 90% efficiency) Max 15 W_out, 19 W_in | ||||||||
Bank IO, clocks, buffers, etc. LEAP Transceiver | P3V3 +3.3V @ <4A 13 W | ← +3.3V | LT3086 x2 (LDO) Max 4.2A | ← +5V Max 5V*4.2+5V*2.1A*2 ≈ 42W | LEAP transceiver with everything enable ~8W → 2.4A Other parts assume < 1A 3.4A * 5V = 17W 17/0.93 ≈ 19W | |||||
Bank IO, clocks, buffers, etc. | P2V5 +2.5V @ <1A 2.5 W | ← +2.5V | LT3086 (LDO) Max 2.1A | |||||||
ASIC IO level translators | P1V3 +1.3V @ <10mA <0.1 W | ← +1.3V | LT3086 (LDO) Max 2.1A |
Product number | Type | Input Voltage | Output Voltage | Max Current | Comment |
LT8638S | Buck | 2.8V to 42V | 0.6V to 42V | 10A | DC/DC Step Down converter. Parallel operation possible. (same as used in Power & communication board (PC_261_101_26_C00) |
TPSM5D1806 | Buck | 4.5V to 15V | 0.5V to 5.5V | Dual 6A / Single 12A | DC/DC PMIC |
LMZ31520 | Buck | 3V to 14.5V | 0.6V to 3.6V | 20A | DC/DC Buck converter. 30A version LMZ31530. |
LT3086 | LDO | 1.4V to 40V | 0.4V to 32V | 2.1A | Low Output Noise: 40µVRMS (10Hz to 100kHz). Parallel operation possible. |
LT3091 | LDO | –1.5V to –36V | 0V to –32V | -1.5A | Negative Linear Regulator. Low Output Noise: 18µVRMS (10Hz to 100kHz). Parallel operation possible. |
https://webench.ti.com/power-designer/switching-regulator
Component | Product number | Board | Voltage | Power consumption | Comments |
Quad SPI Configuration Memory | MT25QU01GBBB8E12 | Digital | 1.8V | Max 50mA | |
EEPROM | 24LC32A-I/MS | Digital | 2.5V | 1mA | |
JTAG | Digital | 1.8V | |||
Analog Monitor (SlowADC) ADC | ADS1217 | Both (key at the analog board, optional at the digital board) | AVDD=3.3V DVDD=1.8V | < 1mA | Maybe. The datasheet guarantees operation for digital down to 2.7V, in HR250 was put at 1.8. Check if it is fine! |
Analog Monitor MUX (x5) | MAX4734 | may be needed depending on the number of channels we decide to monitor (all voltages and currents to the ASIC, humidity,...) | AVDD=3.3V | < 1uA | They are controlled by the ADC |
Humidity sensor | HIH_5031_001 | Analog | 3.3V | No | |
Thermistor | NTC_NHQM103B375T10 | Carrier | No | ||
Oscillators | •371 MHz XLL726371.428571I •156 MHz 536FB156M250DG •48 MHz CX3225SB48000D0FPJC1 | Digital | 2.5V | Both 1.8V and 2.5V solutions can be found depending on the voltage we want to use | |
Clock Fanout | SI53340-B-GM | Digital | 3.3V | ||
Clock Jitter cleaner | SI5345_64QFN | Digital | VDD=3.3V DVDD=1.8V | ||
Programmable Oscillator | LMK61E2 | Digital | 3.3V | Used? | |
High Speed ADC | AD9249 | Analog | 1.8V | Max 58mW/channel: 58*12 = 700mW | No |
ADC_MON_VCM Buffer | AD8607_MSO8 | Analog | 1.8V | ||
Bias DAC (HV Ring) | MAX5443 (DAC) + MAX14611 (Level Shifter) + REF192GS (Voltage reference) | Analog | 3.0 V (VCCA) | Needed? Will the sensors have an HV ring? | |
ASIC clk fanout | SI53340-B-GM | Analog | ??? | Probably not needed | |
HS DAC (Vcalib_p) | AD5541A (DAC) +OPA2626(Buffer) +ADR360B(Vref) | VDD = 3/3.3V VLogic = 1.8V Vref = 2V | |||
Level shifters for ASCI SACI interfacing | MAX3378EEUD+ | Digital | 1.3V -> 1.8V | ||
Level Shifter for Power controllers | MAX3378EETD (x2) MAX3373E_SOT23_8 (x1) | to be defined | ??? | ||
Serial number | DS2411R | Carrier, analog and digital boards | All at 2.5V provided from the digital board | ||
Line Equalizer | (check the one used for cryo) | Analog | Nice to have |
Functionality | Observations | link |
---|---|---|
Carrier to analog board |
| https://docs.google.com/spreadsheets/d/1b_nFUIKPOlVZJwAgv-RxHJQhuoHP3wuV?rtpof=true&usp=drive_fs |
Analog to digital board |
|
|
External power supply |
| |
Optical transceiver |
|
Current cooling block | Updated cooling block |
---|---|
The photo below shows the current cooling block designs (straight and angled), which is for a 30x6 SEAM/SEAF connector between analog and digital board. | The screenshot below shows that a 40x8 SEAM/SEAF connector can fit by extending the cutout in the cooling block without interfering with the pipe. Orange lines are the outline of the cooling block and the pipes in it. The new connector is 10*1.27 = 12.7 mm longer. |
Multi-board project GT-Readout-Platform-single | Digital Board GT-Readout-Platform-single-digital | Analog Board GT-Readout-Platform-single-analog | Carrier Template Board GT-Readout-Platform-single-carrier-template | |
---|---|---|---|---|
3D view | ||||
Description | Digital board for the GT Readout Platform with FPGA and optical transceivers | Analog board for the GT Readout Platform with power supplies, monitoring and calibration circuit for the ASIC carrier board that plugs into it | Template board for ASICs that plugs into the analog board in the GT Readout Platform | |
Altium 365 project | https://stanford-linear-accelerator-center.365.altium.com/designs/5186839B-3CA3-4D54-8947-817E5400B729 | https://stanford-linear-accelerator-center.365.altium.com/designs/15351422-ADCD-4239-875B-3D8D5E0BD347 | https://stanford-linear-accelerator-center.365.altium.com/designs/40A8ABB0-467C-443B-B76D-D732550EB38A | https://stanford-linear-accelerator-center.365.altium.com/designs/AE5F4BFE-DBC6-4B3C-9281-8A1FBAC9D0BA |
Board tracking | GT Readout ASICs | PC_261-101-38_C00 | PC_261-101-39_C00 | PC_261-101-40_C00 |
The components listed in the expansion box below are currently missing from the SLAC Altium library located on OneDrive (Altium_Yee_lib).
The digital board contains the FPGA, supporting ICs and the Amphenol optical transceiver module. The DC/DC on this board are the ones related to the components located here.
UltraScale+ Device Packaging and Pinouts Product Specification User Guide (UG575)
The analog board contains all the DC/DC converters that are needed to support the different power rails of the ASICs (see above). The data and control signals between the ASICs and the FPGA are routed through this board between the two high-density connectors.
The carrier board contains the specific ASICs and any passive components that are needed.
TODO
TODO
TODO