← ePixUHR35kHz - Megapixel Cameras
Tip |
---|
The readout electronics for the 3x2 sensor module is split into two parts; the ASIC carrier and the readout board. This page describe these parts in more detail. |
I/O needs
The sections below indicate how many inputs and outputs (I/Os) are needed for the main blocks of the system.
GT transceiver signals
The FPGA has two types of GT quads:
- GTY capable at bitrates from 0.5 Gbit/s to 32.75 Gbit/s
- 8 quads with 4 TX/RX in each for a total of 32 transceivers
- GTH capable at bitrates from 0.5 Gbit/s to 16.3 Gbit/s
- 11 quads with 4 TX/RX in each for a total of 44 transceivers
LEAP transceivers
The LEAP transceiver can operate up to 25 Gbit/s and therefore require GTY transceivers. There are 12 TX and 12 RX channels in total.
ASICs
There are 8 high-speed data outputs from each ASIC that can operate up to ~6 Gbit/s. For each ASIC there is also one GT clock input which the FPGA has to provide a clock to (6 Gbit/s = 3 GHz clock). In total each ASIC needs 8 RX and 1 TX channel of the FPGA. There are 6 ASICs in total.
- 6*8=48 RX channels
- 6*1=6 TX channels
This is more than what is available in GTH or GTY separately, which means that some ASIC GT channels will be in GTH and some in GTY. It would also be beneficial to have the TX and RX channels separated to avoid constraining the resources and there are enough FPGA GTs available for this. As shown in the table below, there are 8x ASIC RX channels that are placed in two GTY quads.
Total data rates
- 6x ASICs, 8x data outputs per ASIC, up to 6 Gbit/s per data output: 6*8*6=288 Gbit/s
- 12x LEAP transceiver channels up to 25 Gbit/s:
- 1 channel will be used for timing?
- Data rate left: 11*25=275 Gbit/s
- 275/(6*8) ≈ 5.7 Gbit/s per ASIC in this case (+encoding)
- Running PGPv4 at 25 Gbit/s has not been proven yet!
From FPGA to PC:
- 168*192*12(bit)*66/64(PGP encoding) = 399168 bits/frame/ASIC
- @35 kHz frame rate: 6*399168*35e3 = 83.82528 Gbit/s ≈ 83.8 Gbit/s
- 83.82528/11 = 7.62048 Gbit/s ≈ 7.6 Gbit/s per LEAP channel (11 channels in total)
- @100 kHz frame rate: 6*399168*100e3 = 239.5008 Gbit/s ≈ 239.5 Gbit/s
- 239.5008/11 = 21.7728 Gbit/s ≈ 21.8 Gbit/s per LEAP channel (11 channels in total)
Reverse calculation assuming the current PGPv4 running at 15 Gbit/s:
- 11 channels at 15 Gbit/s: 11*15*64/66(PGP encoding) = 160 Gbit/s
- 6 ASICs: 160/6*12/14(encoding) = 22.85714286 ≈ 22.9 Gbit/s per ASIC
- 22.9/8 = 2.8625 Gbit/s per data output
- 22.85714286e9/(168*192*12) = 59051.39834449 ≈ 59 kHz frame rate
GT reference clocks
Quads can source their reference clock from a quad that is at most two quads away. This is to ensure the best jitter performance.
In the table below there are the following reference clock inputs (marked with X and purple/blue):
- One for the LEAP channels for the GTY quads 0 to 2
- One for the ASIC clock channels for the GTY quads 4 and 5
- One for the ASIC data channels for the GTY quads 6 and 7
- One for the ASIC data channels for the GTH quads 0 to 4
- One for the ASIC data channels for the GTH quads 5 to 9
This means at least one reference clock is needed for the LEAP channels and four reference clocks are needed for the ASIC channels. See 3x2 readout board overview above for more details on the system clock structure.
Note |
---|
Important note on clocking of GTY transceivers from UG578 page 48: "QPLL0 must use GTREFCLK0 and QPLL1 must use GTREFCLK1 when the channel is operating above 16.375 Gb/s" |
Summary
Quad 0 | Quad 1 | Quad 2 | Quad 3 | Quad 4 | Quad 5 | Quad 6 | Quad 7 | Quad 8 | Quad 9 | Quad 10 | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | GTY | 4x LEAP | 4x LEAP | 4x LEAP | 4x LEAP | 4x LEAP | 4x LEAP | N/C | 4x ASIC | N/C | 4x ASIC | 3x ASIC | N/C | 3x ASIC | N/C | Clock | |← | ← X → | →| | | X → | →| | | X → | →| | GTH | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | Clock | |← | - | ← X → | - | →| | |← | - | ← X → | - | →| | There is one GTY quad and one GTH quad left over.
Useful resources
Table of contents
Components
FPGA
- Package mechanical drawing
- FPGA measured 3D model
- Difference between the drawing and 3D model used:
- Total height (A = A1 + A2):
- 3D model is 3.86 mm
- Drawing is 3.71 mm (max)
- Solder ball height (A1):
- 3D model is 0.6 mm
- Drawing is 0.6 mm (max)
- Package height (A2):
- 3D model is 3.26 mm
- Drawing is 3.21 mm (max)
- → It seems that the 3D model of the package we have is not 100% accurate, but the difference (3.86-3.71=0.15 mm) is negligible
- Any thermal interface between the package and the cooling block should be able to "absorb" this difference
Optical transceiver
DC/DC converters
Clocks
Timing
ADC/DAC
Temperature
- One on carrier board going to ADC
- One on readout board going to the ADC
- One on carrier board going to external connector?
Humidity
LDO monitoring
ASIC monitoring
LDOs
Various peripherals
ID
Flash
EEPROM
JTAG
Block diagram
Power
- Busbars for input power or TFM connector?
Different options for power components
...
Image Removed
...
FPGA
ASICs
ePixUHR Throughput Calculations
- 168*192*12(bit)*14/12(encoding) = 451584 bits/frame
- @35 kHz frame rate: 451584 bits/frame * 35 kHz = 15.8 Gbit/s per ASIC
- 15.8/8=1.975 Gbit/s per data output
- @100 kHz frame rate: 451584 bits/frame * 100 kHz = 45.2 Gbit/s per ASIC
- 45.2/8=5.65 Gbit/s per data output
Power
- ASIC Version 2 - 2023: ePixUHR-100kHz: power consumption ← Using these numbers in the table below
- ASIC Version 1 - 2022: ePixUHR: power consumption
...
Analog
...
Digital and I/O
...
Analog test system
...
Net
...
G_AS
...
G_DS
...
G_AS_2V5
...
Voltage
...
1.3 V
...
1.3V
...
2.5 V
...
Required current per ASIC
...
1.85 A ≈ 1.9 A
...
0.468 A ≈ 0.5 A
...
0.01 A
...
System requirement with 6 ASICs
(adding +30% current for PVT variation)
...
6*1.9*1.3=14.82 A
+1.3 V @ 15 A
1.3*15=19.5 W
...
6*0.5*1.3=3.9 A
+1.3 V @ 4.0 A
1.3*4=5.2 W
...
6*0.01*1.3=0.078 A
+2.5 V @ 0.5 A
2.5*0.5=1.25 W
Power
A GUI from Linear Technologies (now Analog Devices) called LTPowerPlanner has been used to calculate all the required currents and voltages in the system. It also calculates the estimated losses and efficiency of the system.
The system is designed to operate at an input voltage of 48 V, but can go as low as 30 V.
LTPowerPlanner | Datasheets |
---|
Image Added | |
Expand |
---|
title | Previous 24 V power system... |
---|
|
This system was initially designed to operate at an input voltage of 24 V to be compatible with the GT Readout Platform. It was abandoned in favor of the 48 V system above to make this 3x2 readout compatible with the upcoming SparkPix-S pixel detector, which has ~4x as many pixels in the same area and therefore needs almost double the analog power. Power block diagram | LTPowerPlanner | Datasheets |
---|
Image Added | Image Added | |
See Power supply modules and regulators page for details on interesting power supplies and regulators. Gliffy Diagram |
---|
displayName | power-components-info |
---|
name | power-components-info |
---|
pagePin | 4 |
---|
|
| Option A Same components as used in the GT readout platform Common ASIC supplies | Option B New components Separate ASIC supplies | Option C LT3071 instead of LTM4709 Common ASIC supplies |
---|
| Gliffy Diagram |
---|
size | 200 |
---|
displayName | power-supply-option-a |
---|
name | power-supply-options |
---|
pagePin | 59 |
---|
|
| Gliffy Diagram |
---|
size | 200 |
---|
displayName | power-supply-option-b |
---|
name | power-supply-option-b |
---|
pagePin | 75 |
---|
|
| Gliffy Diagram |
---|
size | 200 |
---|
displayName | power-supply-option-c |
---|
name | power-supply-option-c |
---|
pagePin | 3 |
---|
|
|
---|
| - 5x LT8638S: 5*69 = 345 mm2
- 1x LMZ31520: 246 mm2
- 2x TPSM5D1806: 2*44 = 88 mm2
- 19x LT3086EFE: 19*35 = 665 mm2
- Total: 1344 mm2
- 27 components
| - 3x LTM4676A: 3*256 = 768 mm2
- 1x LTM8060: 190 mm2
- 2x LT3045EDD: 2*9 = 18 mm2
- 4x LTM4709: 4*72 = 288 mm2
- 1x LTM8080: 56 mm2
- Total: 1320 mm2
- 11 components
| - 3x LTM4676A: 3*256 = 768 mm2
- 1x LTM8060: 190 mm2
- 6x LT3071: 6*20 = 120 mm2
- 1x LT3045EDD: 9 mm2
- 1x LTM4709: 72 mm2
- Total: 1159 mm2
- 12 components
|
---|
| - 5x LT8638S: 5*13= $65
- 1x LMZ31520: $20
- 2x TPSM5D1806: 2*24=$48
- 19x LT3086EFE: 19*9=$171
- Total: $304
| - 3x LTM4676A: 3*65=$195
- 1x LTM8060: $34
- 2x LT3045EDD: 2*8=$16
- 4x LTM4709: 4*35=$140
- 1x LTM8080: $25
- Total: $410
| - 3x LTM4676A: 3*65=$195
- 1x LTM8060: $34
- 6x LT3071: 6*10=$60
- 1x LT3045EDD: $8
- 1x LTM4709: $35
- Total: $332
|
---|
| - LT8638S:
- <10 mV pk-pk output ripple
- LMZ31520:
- ±1.8% total output voltage variation
- 1% output voltage ripple @ 20 MHz bandwidth
- TPSM5D1806:
- ±1% voltage reference accuracy
- LT3086EFE:
- ±2% tolerance over line, load and temperature
- 40 μVRMS output noise (10 Hz to 100 kHz)
| - LTM4676A:
- ±0.5% DC output error over temperature
- 10 mV pk-pk output voltage ripple
- LTM8060:
- 0.05% line regulation
- 0.1% load regulation
- 10 mV output RMS ripple
- LT3045EDD:
- 0.8 µVRMS output noise (10 Hz to 100 kHz)
- LTM4709:
- ±1.5% output voltage regulation over line, load,
and temperature - 1.3 μVRMS output noise (10 Hz to 100 kHz)
- LTM8080:
- <1 μVRMS (10 Hz to 100 kHz)
| - LT3071:
±1% accuracy over line, load and temperature, 25 μVRMS output noise (10 Hz to 100 kHz)
|
---|
|
Board power maps
The figures below give an approximation of where the power is consumed on the readout and ASIC carrier boards. The numbers are from the LTPowerPlanner block diagram above.
Readout board power map | Carrier board power map |
---|
Image Added | Image Added |
DrawIO source file: power-maps.drawio |
FPGA
Summary | Graphs |
---|
Image Added | Image Added |
LEAP transceivers
Running at 3.3V, see datasheet on restricted OneDrive for all values. Max power condition of 7.9 W (2.4 A @ 3.3 V) is used in the calculations below.
ASICs
| Analog | Digital and I/O | Analog test system |
---|
Net | G_AS | G_DS | G_AS_2V5 |
---|
Voltage | 1.3 V | 1.3 V | 2.5 V |
---|
Required current per ASIC | 1.85 A ≈ 1.9 A | 0.468 A ≈ 0.5 A | 0.01 A |
---|
System requirement with 6 ASICs (+30% current for PVT variation) | 6*1.9*1.3 = 14.82 A +1.3 V @ 15 A 1.3*15 = 19.5 W | 6*0.5*1.3 = 3.9 A +1.3 V @ 4.0 A 1.3*4 = 5.2 W | 6*0.01*1.3 = 0.078 A +2.5 V @ 0.5 A 2.5*0.5 = 1.25 W |
---|
Power rail current draw
Using worst-case values below for conservative estimate of the currents needed for the different supplies
Power supply rail | Part | Quantity | Max current |
---|
P0V85D | FPGA VCCINT/VCCINT_IO | 1 | 7 A |
---|
Total: | 7 A |
P0V8D_ASIC | ASIC GT bias | 6*2 | < 0.1 A |
---|
Total: | < 0.1 A |
P0V9D | FPGA MGTAVCC | 4 | 4 A |
---|
Total: | 4 A |
P1V2D | FPGA MGTAVTT | 4 | 8 A |
---|
Total: | 8 A |
P1V3D | SN74AXC4T774RSVR | 4 | 4*26 uA = 104 uA |
---|
Total: | < 0.1 A |
P1V3D_ASIC[1..6] | ASIC digital and IO supply | 1 | 0.5*1.3 = 0.65 A |
---|
Total: | < 0.7 A |
P1V8D | SN74AXC4T774RSVR | 4 | 4*26 uA = 104 uA |
---|
SI5345A-D-GM | 1 | 260 mA |
FPGA VCCAUX | 1 | ~500 mA |
FPGA VCCAUX_IO | 1 | ~100 mA |
FPGA MGTVCCAUX | 1 | ~300 mA |
FPGA VCCO 0 | 1 | < 100 mA |
FPGA XADC_VCC | 1 | ~10 mA |
FPGA VCCO 64/65/66/70/71/91 | 1 | < 100 mA |
MT25QU01GBBB8E12-0SIT | 1 | 55 mA |
AD5541ABRMZ | 2 | < 1 mA |
Total: | < 1.5 A |
P2V5D | XLL726371.428571I | 1 | 44 mA |
---|
536FB156M250DG | 1 | 98 mA |
AT24C64D-MAHM-T | 1 | 3 mA |
FPGA VCCO 90 | 1 | < 100 mA |
DS2411R | 2 | < 10 uA |
Total: | < 0.3 A |
P3V3D | SN74AXC4T774RSVR | 4 | 4*26 uA = 104 uA |
---|
SI5345A-D-GM | 1 | 130 mA |
LMK61E2BAA-SIAT | 1 | 196 mA |
FPGA VCCO 93/94 | 1 | < 50 mA |
SN74AHC1G04DBVR | 3 | 3*4 = 12 mA |
SN74LVC3G34DCUR | 3 | ~3*1 = 3 mA |
LEAP transceiver | 1 | 2.4 A |
Total: | < 3 A |
P1V3A_ASIC[1..6] | ASIC analog supply | 1 | 1.9*1.3=2.47 A |
---|
Total: | < 2.5 A |
P2V5A_ASIC | ASIC analog test system | 6 | 6*10*1.3 = 78 mA |
---|
Total: | < 0.1 A |
P3V3A | ADS1217IPFBT | 1 | 1.325 mA |
---|
HIH-5031-001 | 1 | 0.5 mA |
ADR360BUJZ | 1 | < 1 mA |
ADR361BUJZ | 1 | < 1 mA |
OPA2626IDGKR | 2 | 2*2*3.1 mA = 12.4 mA |
AD5541ABRMZ | 2 | < 1 mA |
MAX4781ETE | ??? | < 1 mA |
Total: | < 0.1 A |
I/O needs
The sections below indicate how many inputs and outputs (I/Os) are needed for the main blocks of the system.
GT transceiver signals
The FPGA has two types of GT quads:
- GTY capable at bitrates from 0.5 Gbit/s to 32.75 Gbit/s
- 8 quads with 4 TX/RX in each for a total of 32 transceivers
- GTH capable at bitrates from 0.5 Gbit/s to 16.3 Gbit/s
- 11 quads with 4 TX/RX in each for a total of 44 transceivers
LEAP transceivers
The LEAP transceiver can operate up to 25 Gbit/s and therefore require GTY transceivers. There are 12 TX and 12 RX channels in total.
ASICs
There are 8 high-speed data outputs from each ASIC that can operate up to ~6 Gbit/s. For each ASIC there is also one GT clock input which the FPGA has to provide a clock to (6 Gbit/s = 3 GHz clock). In total each ASIC needs 8 RX and 1 TX channel of the FPGA. There are 6 ASICs in total.
- 6*8 = 48 RX channels
- 6*1 = 6 TX channels
This is more than what is available in GTH or GTY separately, which means that some ASIC GT channels will be in GTH and some in GTY. It would also be beneficial to have the TX and RX channels separated to avoid constraining the resources and there are enough FPGA GTs available for this. As shown in the table below, there are 8x ASIC RX channels that are placed in two GTY quads.
Power block diagram
I/O needs
GT transceiver signals
- GTY capable at bitrates from 0.5 Gbit/s to 32.75 Gbit/s
- 8 quads with 4 TX/RX in each for a total of 32 transceivers
- GTH capable at bitrates from 0.5 Gbit/s to 16.3 Gbit/s
- 11 quads with 4 TX/RX in each for a total of 44 transceivers
LEAP transceivers
ASICs
- 6*8=48 RX channels
- 6*1=6 TX channels
Total data rates
- 6x ASICs, 8x data outputs per ASIC, up to 6 Gbit/s per data output: 6*8*6=288 Gbit/s
- 12x LEAP transceiver channels up to 25 Gbit/s:
- 1 channel will be used for timing?
- Data rate left: 11*25=275 Gbit/s
- 275/(6*8) ≈ 5.7 Gbit/s per ASIC in this case (+encoding)
- Running PGPv4 at 25 Gbit/s has not been proven yet!
From FPGA to PC:
- 168*192*12(bit)*66/64(PGP encoding) = 399168 bits/frame/ASIC
- @35 kHz frame rate: 6*399168*35e3 = 83.82528 Gbit/s ≈ 83.8 Gbit/s
- 83.82528/11 = 7.62048 Gbit/s ≈ 7.6 Gbit/s per LEAP channel (11 channels in total)
- @100 kHz frame rate: 6*399168*100e3 = 239.5008 Gbit/s ≈ 239.5 Gbit/s
- 239.5008/11 = 21.7728 Gbit/s ≈ 21.8 Gbit/s per LEAP channel (11 channels in total)
- 11 channels at 15 Gbit/s: 11*15*64/66(PGP encoding) = 160 Gbit/s
- 6 ASICs: 160/6*12/14(encoding) = 22.85714286 ≈ 22.9 Gbit/s per ASIC
- 22.9/8 = 2.8625 Gbit/s per data output
- 22.85714286e9/(168*192*12) = 59051.39834449 ≈ 59 kHz frame rate
GT reference clocks
- One for the LEAP channels for the GTY quads 0 to 2
- One for the ASIC clock channels for the GTY quads 4 and 5
- One for the ASIC data channels for the GTY quads 6 and 7
- One for the ASIC data channels for the GTH quads 0 to 4
- One for the ASIC data channels for the GTH quads 5 to 9
Note |
---|
Important note on clocking of GTY transceivers from UG578 page 48: "QPLL0 must use GTREFCLK0 and QPLL1 must use GTREFCLK1 when the channel is operating above 16.375 Gb/s" |
Summary
...
Useful resources
Table of contents
Components
FPGA
- Package mechanical drawing
- FPGA measured 3D model
- Difference between the drawing and 3D model used:
- Total height (A = A1 + A2):
- 3D model is 3.86 mm
- Drawing is 3.71 mm (max)
- Solder ball height (A1):
- 3D model is 0.6 mm
- Drawing is 0.6 mm (max)
- Package height (A2):
- 3D model is 3.26 mm
- Drawing is 3.21 mm (max)
- → It seems that the 3D model of the package we have is not 100% accurate, but the difference (3.86-3.71=0.15 mm) is negligible
- Any thermal interface between the package and the cooling block should be able to "absorb" this difference
Optical transceiver
DC/DC converters
Clocks
Timing
ADC/DAC
Temperature
- One on carrier board going to ADC
- One on readout board going to the ADC
- One on carrier board going to external connector?
Humidity
LDO monitoring
ASIC monitoring
LDOs
Various peripherals
ID
Flash
EEPROM
JTAG
Block diagram
Power
- Busbars for input power or TFM connector?
Different options for power components
...
Gliffy Diagram |
---|
size | 300 |
---|
displayName | power-supply-option-b |
---|
name | power-supply-option-b |
---|
pagePin | 4 |
---|
|
FPGA
ASICs
ePixUHR Throughput Calculations
- 168*192*12(bit)*14/12(encoding) = 451584 bits/frame
- @35 kHz frame rate: 451584 bits/frame * 35 kHz = 15.8 Gbit/s per ASIC
- 15.8/8=1.975 Gbit/s per data output
- @100 kHz frame rate: 451584 bits/frame * 100 kHz = 45.2 Gbit/s per ASIC
- 45.2/8=5.65 Gbit/s per data output
Power
- ASIC Version 2 - 2023: ePixUHR-100kHz: power consumption ← Using these numbers in the table below
- ASIC Version 1 - 2022: ePixUHR: power consumption
...
Analog
...
Digital and I/O
...
Analog test system
...
Net
...
G_AS
...
G_DS
...
G_AS_2V5
...
Voltage
...
1.3 V
...
1.3V
...
2.5 V
...
Required current per ASIC
...
1.85 A ≈ 1.9 A
...
0.468 A ≈ 0.5 A
...
0.01 A
...
System requirement with 6 ASICs
(adding +30% current for PVT variation)
...
6*1.9*1.3=14.82 A
+1.3 V @ 15 A
1.3*15=19.5 W
...
6*0.5*1.3=3.9 A
+1.3 V @ 4.0 A
1.3*4=5.2 W
...
6*0.01*1.3=0.078 A
+2.5 V @ 0.5 A
2.5*0.5=1.25 W
Power block diagram
I/O needs
GT transceiver signals
- GTY capable at bitrates from 0.5 Gbit/s to 32.75 Gbit/s
- 8 quads with 4 TX/RX in each for a total of 32 transceivers
- GTH capable at bitrates from 0.5 Gbit/s to 16.3 Gbit/s
- 11 quads with 4 TX/RX in each for a total of 44 transceivers
LEAP transceivers
ASICs
- 6*8=48 RX channels
- 6*1=6 TX channels
Total data rates
- 6x ASICs, 8x data outputs per ASIC, up to 6 Gbit/s per data output: 6*8*6=288 Gbit/s
- 12x LEAP transceiver channels up to 25 Gbit/s:
- 1 channel will be used for timing?
- Data rate left: 11*25=275 Gbit/s
- 275/(6*8) ≈ 5.7 Gbit/s per ASIC in this case (+encoding)
- Running PGPv4 at 25 Gbit/s has not been proven yet!
From FPGA to PC:
- 168*192*12(bit)*66/64(PGP encoding) = 399168 bits/frame/ASIC
- @35 kHz frame rate: 6*399168*35e3 = 83.82528 Gbit/s ≈ 83.8 Gbit/s
- 83.82528/11 = 7.62048 Gbit/s ≈ 7.6 Gbit/s per LEAP channel (11 channels in total)
- @100 kHz frame rate: 6*399168*100e3 = 239.5008 Gbit/s ≈ 239.5 Gbit/s
- 239.5008/11 = 21.7728 Gbit/s ≈ 21.8 Gbit/s per LEAP channel (11 channels in total)
- 11 channels at 15 Gbit/s: 11*15*64/66(PGP encoding) = 160 Gbit/s
- 6 ASICs: 160/6*12/14(encoding) = 22.85714286 ≈ 22.9 Gbit/s per ASIC
- 22.9/8 = 2.8625 Gbit/s per data output
- 22.85714286e9/(168*192*12) = 59051.39834449 ≈ 59 kHz frame rate
GT reference clocks
- One for the LEAP channels for the GTY quads 0 to 2
- One for the ASIC clock channels for the GTY quads 4 and 5
- One for the ASIC data channels for the GTY quads 6 and 7
- One for the ASIC data channels for the GTH quads 0 to 4
- One for the ASIC data channels for the GTH quads 5 to 9
Note |
---|
Important note on clocking of GTY transceivers from UG578 page 48: "QPLL0 must use GTREFCLK0 and QPLL1 must use GTREFCLK1 when the channel is operating above 16.375 Gb/s" |
Summary
...
Useful resources
Table of contents
Components
FPGA
- Package mechanical drawing
- FPGA measured 3D model
- Difference between the drawing and 3D model used:
- Total height (A = A1 + A2):
- 3D model is 3.86 mm
- Drawing is 3.71 mm (max)
- Solder ball height (A1):
- 3D model is 0.6 mm
- Drawing is 0.6 mm (max)
- Package height (A2):
- 3D model is 3.26 mm
- Drawing is 3.21 mm (max)
- → It seems that the 3D model of the package we have is not 100% accurate, but the difference (3.86-3.71=0.15 mm) is negligible
- Any thermal interface between the package and the cooling block should be able to "absorb" this difference
Optical transceiver
DC/DC converters
Clocks
Timing
ADC/DAC
Temperature
- One on carrier board going to ADC
- One on readout board going to the ADC
- One on carrier board going to external connector?
Humidity
LDO monitoring
ASIC monitoring
LDOs
Various peripherals
ID
Flash
EEPROM
JTAG
Block diagram
Power
- Busbars for input power or TFM connector?
Different options for power components
...
Image Removed
...
FPGA
ASICs
ePixUHR Throughput Calculations
- 168*192*12(bit)*14/12(encoding) = 451584 bits/frame
- @35 kHz frame rate: 451584 bits/frame * 35 kHz = 15.8 Gbit/s per ASIC
- 15.8/8=1.975 Gbit/s per data output
- @100 kHz frame rate: 451584 bits/frame * 100 kHz = 45.2 Gbit/s per ASIC
- 45.2/8=5.65 Gbit/s per data output
Power
- ASIC Version 2 - 2023: ePixUHR-100kHz: power consumption ← Using these numbers in the table below
- ASIC Version 1 - 2022: ePixUHR: power consumption
...
Analog
...
Digital and I/O
...
Analog test system
...
Net
...
G_AS
...
G_DS
...
G_AS_2V5
...
Voltage
...
1.3 V
...
1.3V
...
2.5 V
...
Required current per ASIC
...
1.85 A ≈ 1.9 A
...
0.468 A ≈ 0.5 A
...
0.01 A
...
System requirement with 6 ASICs
(adding +30% current for PVT variation)
...
6*1.9*1.3=14.82 A
+1.3 V @ 15 A
1.3*15=19.5 W
...
6*0.5*1.3=3.9 A
+1.3 V @ 4.0 A
1.3*4=5.2 W
...
6*0.01*1.3=0.078 A
+2.5 V @ 0.5 A
2.5*0.5=1.25 W
Power block diagram
I/O needs
GT transceiver signals
- GTY capable at bitrates from 0.5 Gbit/s to 32.75 Gbit/s
- 8 quads with 4 TX/RX in each for a total of 32 transceivers
- GTH capable at bitrates from 0.5 Gbit/s to 16.3 Gbit/s
- 11 quads with 4 TX/RX in each for a total of 44 transceivers
LEAP transceivers
ASICs
...
Total data rates
- 6x ASICs, 8x data outputs per ASIC, up to 6 Gbit/s per data output: 6*8*6=288 Gbit/s
- 12x LEAP transceiver channels up to 25 Gbit/s:
- 1 channel will be used for timing
...
- Data rate left: 11*25=275 Gbit/s
- 275/(6*8) ≈ 5.7 Gbit/s per ASIC in this case (+encoding)
- Running PGPv4 at 25 Gbit/s has not been proven yet!
...
- /display/ppareg/GS-12-1396
- Main limitation is the 64-bit data width of PGPv4 which would result in a 25000/64 ≈ 390 MHz fabric clock
- 11*15=165 Gbit/s
- 165/(6*8) = 3.4375 Gbit/s per ASIC in this case (+encoding)
From FPGA to PC:
...
- (PGP encoding) = 399168 bits/frame/ASIC
- @35 kHz frame rate: 6*399168*35e3 = 83.82528 Gbit/s ≈ 83.8 Gbit/s
- 83.82528/11 = 7.62048 Gbit/s ≈ 7.6 Gbit/s per LEAP channel (11 channels in total)
- @100
...
- kHz frame rate: 6*399168*
...
...
...
...
...
...
...
...
...
...
...
- 8 Gbit/s per LEAP channel (11 channels in total)
Reverse calculation assuming the current PGPv4 running at 15 Gbit/s:
- 11 channels at 15 Gbit/s: 11*15*64/66(PGP encoding)
...
- 239.5008/11 = 21.7728 Gbit/s ≈ 21.8 Gbit/s per LEAP channel (11 channels in total)
- = 160 Gbit/s
- 6 ASICs: 160/6*12/14(encoding) = 22.85714286 ≈ 22.9 Gbit/s per ASIC
- 22.9/8 = 2.8625 Gbit/s per data output
- 22.85714286e9/(168*192*12) = 59051.39834449 ≈ 59 kHz frame rate
Reverse calculation assuming the current PGPv4 running at 10 Gbit/s:
...
...
...
...
- 106.67/6*12/14(encoding) =
...
...
...
...
...
...
...
...
- 905 Gbit/s per data output
...
...
- 23857143e9/(168*192*12) =
...
- 39368.82913256 ≈ 39 kHz frame rate
GT reference clocks
Quads can source their reference clock from a quad that is at most two quads away. This is to ensure the best jitter performance.
In the table below there are the following reference clock inputs (marked with X and purple/blue):
...
- One for the LEAP channels for the GTY quads 0 to 2
- One for the ASIC clock channels for the GTY quads 4 and 5
- One for the ASIC data channels for the GTY quads 6 and 7
- One for the ASIC data channels for the GTH quads 0 to 4
- One for the ASIC data channels for the GTH quads 5 to 9
This means at least one reference clock is needed for the LEAP channels and four reference clocks are needed for the ASIC channels. See Clocks above for more details on the system clock structure.
Note |
---|
Important note on clocking of GTY transceivers from UG578 page 48: "QPLL0 must use GTREFCLK0 and QPLL1 must use GTREFCLK1 when the channel is operating above 16.375 Gb/s" |
Summary
| Quad 0 | Quad 1 | Quad 2 | Quad 3 | Quad 4 | Quad 5 | Quad 6 | Quad 7 | Quad 8 | Quad 9 | Quad 10 |
---|
| TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX | TX | RX |
---|
GTY | 4x LEAP | 4x LEAP | 4x LEAP | 4x LEAP | 4x LEAP | 4x LEAP |
|
| N/C | 4x ASIC | N/C | 4x ASIC | 3x ASIC | N/C | 3x ASIC | N/C |
|
|
|
|
|
|
---|
Clock | |← | ← X → | →| |
| | X → | →| | | X → | →| |
|
|
|
---|
GTH | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC | N/C | 4x ASIC |
|
|
---|
Clock | |← | - | ← X → | - | →| | |← | - | ← X → | - | →| |
|
---|
There is one GTY quad and one GTH quad left over.