Components

FPGA

The FPGA that will be used is XCKU15P-2FFVA1760E from AMD/Xilinx. Same as used in the GT Readout Platform.

Click here to expand...

Package mechanical drawing
FPGA measured 3D model
Difference between the drawing and 3D model used:
- Total height (A = A1 + A2):
  - 3D model is 3.86 mm
  - Drawing is 3.71 mm (max)
- Solder ball height (A1):
  - 3D model is 0.6 mm
  - Drawing is 0.6 mm (max)
- Package height (A2):
  - 3D model is 3.26 mm
  - Drawing is 3.21 mm (max)
- → It seems that the 3D model of the package we have is not 100% accurate, but the difference (3.86-3.71=0.15 mm) is negligible
- Any thermal interface between the package and the cooling block should be able to "absorb" this difference

Optical transceiver

Page about the LEAP transceiver from other project: https://confluence.slac.stanford.edu/display/ppareg/GS-12-1396

Clocks

Timing

External TTL timing in/out

ADC/DAC

Only slow-speed ADC.

Temperature

NHQM103B375T10 10k NTC type thermistors are used at:

One on carrier board going to ADC
One on readout board going to the ADC
One on carrier board going to external connector

Humidity

One HIH-5031-001 sensor mounted close to the carrier connector.

LDO monitoring

ASIC monitoring

Various peripherals

ID

A DS2411R+T&R chip is located on the readout board to provide a unique ID that can be read out by the FPGA. Another ID chip is also place on the carrier board.

Flash

Used to store the FPGA configuration.

EEPROM

Used to store operational settings and parameters.

JTAG

A 14-pin JTAG connector (Molex 87832-1420) is located at the bottom of the readout board to not interfere with the cold plate on the top.

Block diagram

3x2-readout-board-overview

Power

Busbars for input power or TFM connector?

FPGA

https://www.xilinx.com/products/technology/power/xpe.html

ASICs

ePixUHR Throughput Calculations

From ASIC to FPGA:

168*192*12(bit)*14/12(encoding) = 451584 bits/frame
@35 kHz frame rate: 451584 bits/frame * 35 kHz = 15.8 Gbit/s per ASIC
- 15.8/8=1.975 Gbit/s per data output
@100 kHz frame rate: 451584 bits/frame * 100 kHz = 45.2 Gbit/s per ASIC
- 45.2/8=5.65 Gbit/s per data output

Power

ASIC Version 2 - 2023: ePixUHR-100kHz: power consumption ← Using these numbers in the table below
ASIC Version 1 - 2022: ePixUHR: power consumption

	Analog	Digital and I/O	Analog test system
Net	G_AS	G_DS	G_AS_2V5
Voltage	1.3 V	1.3V	2.5 V
Required current per ASIC	1.85 A ≈ 1.9 A	0.468 A ≈ 0.5 A	0.01 A
System requirement with 6 ASICs (+30% current for PVT variation)	61.91.3=14.82 A +1.3 V @ 15 A 1.315=19.5 W*	60.51.3=3.9 A +1.3 V @ 4.0 A 1.34=5.2 W*	60.011.3=0.078 A +2.5 V @ 0.5 A 2.50.5=1.25 W*

Different options for power components

See Power supply modules and regulators page for details on interesting power supplies and regulators.

power-components-info

	Option A Same components as used in the GT readout platform Common ASIC supplies	Option B New components Separate ASIC supplies	Option B LT3071 instead of LTM4709 Common ASIC supplies
Diagram
Area	5x LT8638S: 569=345mm² 1x LMZ31520: 246mm² 2x TPSM5D1806: 244=88mm² 18x LT3086EFE: 1835=630mm² Total: 1309mm²* 26 components	3x LTM4676A: 3256=768mm² 1x LTM8060: 190mm² 1x LT3045EDD: 9mm² 5x LTM4709: 572=360mm² Total: 1327mm² 10 components	3x LTM4676A: 3256=768mm² 1x LTM8060: 190mm² 6x LT3071: 620=120mm² 1x LT3045EDD: 9mm² 1x LTM4709: 72mm² Total: 1159mm² 12 components
Cost January 2024	5x LT8638S: 513= $65 1x LMZ31520: $20 2x TPSM5D1806: 224=$48 18x LT3086EFE: 189=$162 Total: $295*	3x LTM4676A: 365=$195 1x LTM8060: $34 1x LT3045EDD: $8 5x LTM4709: 535=$175 Total: $412	3x LTM4676A: 365=$195 1x LTM8060: $34 6x LT3071: 610=$60 1x LT3045EDD: $8 1x LTM4709: $35 Total: $332
Noise performance	LT8638S: <10 mV pk-pk output ripple LMZ31520: ±1.8% total output voltage variation 1% output voltage ripple @20MHz bandwidth TPSM5D1806: ±1% voltage reference accuracy LT3086EFE: ±2% tolerance over line, load and temperature 40 μV_RMS output noise (10Hz to 100kHz)	LTM4676A: ±0.5% DC output error over temperature 10 mV pk-pk output voltage ripple LTM8060: 0.05% line regulation 0.1% load regulation 10 mV output RMS ripple LT3045EDD: 0.8 µV_RMS output noise (10Hz to 100kHz) LTM4709: ±1.5% output voltage regulation over line, load, and temperature 1.3 μV_RMS output noise (10Hz to 100kHz)	LT3071: ±1% accuracy over line, load and temperature, 25 μV_RMS output noise (10Hz to 100kHz)

Power block diagram

I/O needs

The sections below indicate how many inputs and outputs (I/Os) are needed for the main blocks of the system.

GT transceiver signals

The FPGA has two types of GT quads:

GTY capable at bitrates from 0.5 Gbit/s to 32.75 Gbit/s
- 8 quads with 4 TX/RX in each for a total of 32 transceivers
GTH capable at bitrates from 0.5 Gbit/s to 16.3 Gbit/s
- 11 quads with 4 TX/RX in each for a total of 44 transceivers

LEAP transceivers

The LEAP transceiver can operate up to 25 Gbit/s and therefore require GTY transceivers. There are 12 TX and 12 RX channels in total.

ASICs

There are 8 high-speed data outputs from each ASIC that can operate up to ~6 Gbit/s. For each ASIC there is also one GT clock input which the FPGA has to provide a clock to (6 Gbit/s = 3 GHz clock). In total each ASIC needs 8 RX and 1 TX channel of the FPGA. There are 6 ASICs in total.

6*8=48 RX channels
6*1=6 TX channels

This is more than what is available in GTH or GTY separately, which means that some ASIC GT channels will be in GTH and some in GTY. It would also be beneficial to have the TX and RX channels separated to avoid constraining the resources and there are enough FPGA GTs available for this. As shown in the table below, there are 8x ASIC RX channels that are placed in two GTY quads.

Total data rates

6x ASICs, 8x data outputs per ASIC, up to 6 Gbit/s per data output: 6*8*6=288 Gbit/s
12x LEAP transceiver channels up to 25 Gbit/s:
- 1 channel will be used for timing
- Data rate left: 11*25=275 Gbit/s
  - 275/(6*8) ≈ 5.7 Gbit/s per ASIC in this case (+encoding)
- Running PGPv4 at 25 Gbit/s has not been proven yet!
  - 15 Gbit/s according to this page https://confluence.slac.stanford.edu/display/ppareg/GS-12-1396
  - 11*15=165 Gbit/s
  - 165/(6*8) = 3.4375 Gbit/s per ASIC in this case (+encoding)
From FPGA to PC:
- 168*192*12(bit)*66/64(PGP encoding) = 399168 bits/frame/ASIC
- @35 kHz frame rate: 6*399168*35e3 = 83.82528 Gbit/s ≈ 83.8 Gbit/s
  - 83.82528/11 = 7.62048 Gbit/s ≈ 7.6 Gbit/s per LEAP channel (11 channels in total)
- @100 kHz frame rate: 6*399168*100e3 = 239.5008 Gbit/s ≈ 239.5 Gbit/s
  - 239.5008/11 = 21.7728 Gbit/s ≈ 21.8 Gbit/s per LEAP channel (11 channels in total)

Reverse calculation assuming the current PGPv4 running at 15 Gbit/s:

11 channels at 15 Gbit/s: 11*15*64/66(PGP encoding) = 160 Gbit/s
6 ASICs: 160/6*12/14(encoding) = 22.85714286 ≈ 22.9 Gbit/s per ASIC
- 22.9/8 = 2.8625 Gbit/s per data output
- 22.85714286e9/(168*192*12) = 59051.39834449 ≈ 59 kHz frame rate

GT reference clocks

Quads can source their reference clock from a quad that is at most two quads away. This is to ensure the best jitter performance.

In the table below there are the following reference clock inputs (marked with X and purple/blue):

One for the LEAP channels for the GTY quads 0 to 2
One for the ASIC clock channels for the GTY quads 4 and 5
One for the ASIC data channels for the GTY quads 6 and 7
One for the ASIC data channels for the GTH quads 0 to 4
One for the ASIC data channels for the GTH quads 5 to 9

This means at least one reference clock is needed for the LEAP channels and four reference clocks are needed for the ASIC channels. See 3x2 readout board overview above for more details on the system clock structure.

Important note on clocking of GTY transceivers from UG578 page 48: "QPLL0 must use GTREFCLK0 and QPLL1 must use GTREFCLK1 when the channel is operating above 16.375 Gb/s"

Summary

	Quad 0		Quad 1		Quad 2		Quad 3		Quad 4		Quad 5		Quad 6		Quad 7		Quad 8		Quad 9		Quad 10
	TX	RX	TX	RX	TX	RX	TX	RX	TX	RX	TX	RX	TX	RX	TX	RX	TX	RX	TX	RX	TX	RX
GTY	4x LEAP	4x LEAP	4x LEAP	4x LEAP	4x LEAP	4x LEAP			N/C	4x ASIC	N/C	4x ASIC	3x ASIC	N/C	3x ASIC	N/C
Clock	\|←		← X →		→\|				\| X →		→\|		\| X →		→\|
GTH	N/C	4x ASIC	N/C	4x ASIC	N/C	4x ASIC	N/C	4x ASIC	N/C	4x ASIC	N/C	4x ASIC	N/C	4x ASIC	N/C	4x ASIC	N/C	4x ASIC	N/C	4x ASIC
Clock	\|←		-		← X →		-		→\|		\|←		-		← X →		-		→\|

There is one GTY quad and one GTH quad left over.

Space shortcuts

Confluence Content

Child pages

Useful resources

Table of contents

Components

FPGA

Optical transceiver

Clocks

Timing

ADC/DAC

Temperature

Humidity

LDO monitoring

ASIC monitoring

Various peripherals

ID

Flash

EEPROM

JTAG

Block diagram

Power

FPGA

ASICs

ePixUHR Throughput Calculations

Power

Different options for power components

Power block diagram

I/O needs

GT transceiver signals

LEAP transceivers

ASICs

Total data rates

GT reference clocks

Summary