Discussion on Jan. 18, 2022

with claus, weaver, cpo

Matt's proposal:

timing on LC, but still handled by BOS
networking might be on LC as well (unless 40Gb or 100Gb, which is natively MPO). NOT handled by BOS (also XTCAV point-to-point)
pgp is on MPO and handled by the BOS
detectors are all LC going to the BOS
BOS-to-208-patch-panel is MPO (via breakout cable) and some LC as described above
don't need breakout cables in SRCF according to Matt's proposal (wasteful of fibers between 208 and SRCF?)
try to use the SFP-to-QSFP-adapters (LC to MPO) as much as possible in SRCF

order:

bos has 320 pairs, but only half of these are used for SRCF, the other half goes to detectors (LC). Implies 40 breakout cables with 4 fiber-pairs per MPO for room 208?
56 cmp, 11 eb, 19 mon nodes. currently only cmp have kcu's. maximum of 112 transceivers, but some should be sfp-to-qsfp. hsd's use lr4 (currently 12 channels in TMO and 4 channels in RIX). in feb. 2023 tmo needs 14 for hexanodes plus extra channels (spare and miscellaneous signals). so guess 25 LR4. expect 36 in TMO eventually. all others use plr4. 112-25 ~= 90 PLR4 max. Matt guess half will be sfp-to-qsfp. So 45 PLR4, and 45 sfp-to-qsfp
56 cmp nodes corresponds to 56*8=448 fiber pairs (more than the half of the bos devoted to srcf (other half is for detectors))
order our own patch fibers? (1 foot through 3m or 5m?)
in short-term need to cable test detectors incrementally
in long-term consider ripping out and having professionals do it (requires planning). we do have a short window (2 or 3 weeks?) setup SRCF in advance using estimates of detector numbers. ideally so we can maintain it.
need to figure out fiber swaps (tx/rx) and how we do it in a neat way. more complex with MPO. Maybe BOS could do this? We think not: monitoring might not work. proposal: best place is from BOS→SRCFpatchpanel. indicate the swap by removing the plastic piece that holds tx/rx in place.

3 sets of limited fiber resources:

Sept. 2022 Cabling Proposal

Executive summary: I think at 100kHz we can keep roughly the same number of nodes that we have in the FEE alcove.

Issues:

which fibers should be MPO vs LC?
- only use MPO for fim's.
should we fanout timing at SRCF or put it through the bos?
- use the fanouts at SRCF for simplicity
- should we fanout in 208?
- maybe put camlink data fibers through the bos?
routing mono-encoder udp network over to srcf
sharing xtcav between hutches with bos-switchable timing

100kHz data volumes:

wave8: 8waveforms*256samples/sample*2bytes/sample*0.1MHz = 400MB/s (need to add FEX data too)
piranha: 2048px*2bytes/px*0.1MHz=410MB/s
hsd full wf: 60000samples*2bytes/sample*0.1MHz = 12GB/s
- max waveform length appears to be ~60000 samples looking at xtc2 file?
- FEX needs to reduce it to 4GB/s (3x)
bld: 336bytes/event*0.1MHz = 33MB/s (from xtcreader -f /cds/data/psdm/prj/public01/xtc/tmoc00118-r0222-s008-c000.xtc2)
timing: 309bytes/event*0.1MHz = 31MB/s

recabling proposal:

current inventory (sept. 13, 2022):