Page History
...
- bos: 320 pairs, 160 for detectors, 160 for srcf
- nodes: 56*8=448 pairs
- fibers between 208 and srcf: 300 fiber pairs (75% 48 MPO, 25% 48 LC?). Total of 240?
SRCF Fiber Order
...
- 27 nodes needed
- shutdown 30
- 3 broken (cmp006, 13, 15)
SRCF cabling
Nodes
| drp-srcf-cmpNNN | KCU firmware | Comment | Application (Oct 17, 2022) | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 001 | DrpTDet | TMO low rate, XTCAV nic (SXR, cpo believes) | slow tmo tdet (+xtcav (SXR, cpo believes)) | ||||||||||||||||||||||||||||||
| 002 | DrpTDet | RIX low rate, encoder nic | slow rix tdet (+mono) | ||||||||||||||||||||||||||||||
| 003 | DrpTDet | TMO high rate | tmo high rate tdet | ||||||||||||||||||||||||||||||
| 004004 | XilinxKcu1500Pgp4_6Gbps_ddr | QSFP 1 cage is not workingrix/tmo wave8 Collection host | |||||||||||||||||||||||||||||||
| 005 | DrpPgpIlv | rix hsd's | |||||||||||||||||||||||||||||||
| 006 | ClinkKcu1500Pgp2b | LCLSECSD-1844. Possibly repaired spare camlink? | spare camlink node? | ||||||||||||||||||||||||||||||
| 007 | ClinkKcu1500Pgp2b | tmo fzp piranha | |||||||||||||||||||||||||||||||
| 008 | DrpTDet | didn't show up on xpm | rix user meb | ||||||||||||||||||||||||||||||
| 009 | DrpPgpIlv | rix hsd's | |||||||||||||||||||||||||||||||
| 010 | DrpTDet | RIX high rate | rix fast tdet | 010 | DrpTDet | ||||||||||||||||||||||||||||
| 011 | ClinkKcu1500Pgp2b | tmo fzp opal | |||||||||||||||||||||||||||||||
| 012 | ClinkKcu1500Pgp2b | ||||||||||||||||||||||||||||||||
| 013 | No KCUClinkKcu1500Pgp2b | tmo atm piranha | |||||||||||||||||||||||||||||||
| 014 | Lcls2EpixHrXilinxKcu1500Pgp4_6Gbps | epix hr | |||||||||||||||||||||||||||||||
| 015 | No mlx5; Bad Ethernet | 016 | DrpTDet | 017 | DrpPgpIlv | 018 | DrpPgpIlv | 019 | DrpPgpIlv | 020 | DrpPgpIlv | 021 | DrpPgpIlv | 022 | DrpPgpIlv | 023 | DrpPgpIlv | 024 | DrpPgpIlv | 025 | DrpTDet | 026 | DrpTDet | 027 | DrpTDet | 028 | DrpTDet | 029 | DrpTDet | 030 | DrpTDetLCLSECSD-1305 Unreliable machine: bad for TEB / MEB | nothing | |
| 016 | DrpTDet | No mlx5. Error messages LCLSECSD-1305 Unreliable machine: bad for TEB / MEB | nothing | ||||||||||||||||||||||||||||||
| 017 | DrpPgpIlv | tmo hsd_3 | |||||||||||||||||||||||||||||||
| 018 | DrpPgpIlv | tmo hsd_5,hsd_7 | |||||||||||||||||||||||||||||||
| 019 | DrpPgpIlv | tmo hsd_4,hsd_13 | |||||||||||||||||||||||||||||||
| 020 | DrpPgpIlv | tmo hsd_2 | |||||||||||||||||||||||||||||||
| 021 | DrpPgpIlv | tmo hsd_6 | |||||||||||||||||||||||||||||||
| 022 | DrpPgpIlv | tmo hsd_8,hsd_9 | |||||||||||||||||||||||||||||||
| 023 | DrpPgpIlv | peppex hsd's | |||||||||||||||||||||||||||||||
| 024 | DrpPgpIlv | tmo hsd_10,hsd_11 | |||||||||||||||||||||||||||||||
| 025 | DrpTDet | rix slow tdet | |||||||||||||||||||||||||||||||
| 026 | ClinkKcu1500Pgp2b | tmo atm opal | |||||||||||||||||||||||||||||||
| 027 | ClinkKcu1500Pgp2b | rix atm opal | |||||||||||||||||||||||||||||||
| 028 | DrpTDet | XPM:6 | tmo ami | ||||||||||||||||||||||||||||||
| 029 | DrpTDet | XPM:5 | rix ami | ||||||||||||||||||||||||||||||
| 030 | DrpTDet | NC | tmo user meb | ||||||||||||||||||||||||||||||
| 031 | XilinxKcu1500Pgp4_6Gbps_ddr | LCLSECSD-2030: BIOS not upgradable | tmo user meb | ||||||||||||||||||||||||||||||
| 032 | DrpTDet | LCLS1 HXR hutches | |||||||||||||||||||||||||||||||
| 033 | DrpTDet | XPM:0. update bios settings: only one pcie bus. | |||||||||||||||||||||||||||||||
| 034 | DrpTDet | XPM:0 | mona slurm tests | ||||||||||||||||||||||||||||||
| 035 | DrpTDet | XPM:0; LCLSECSD-2032: BIOS not upgradable from v2.0 but is useable as-is | mona slurm tests | ||||||||||||||||||||||||||||||
| 036 | DrpTDet | epixM data | epixM | ||||||||||||||||||||||||||||||
| 037 | DrpTDet | ||||||||||||||||||||||||||||||||
| 038 | DrpTDet | used in the spare project XPM:7 | |||||||||||||||||||||||||||||||
| 039 | DrpTDet | LCLSECSD-2028: Broken IPMI | |||||||||||||||||||||||||||||||
| 040 | XilinxKcu1500Pgp4_6Gbps | used in the spare project XPM:0 | |||||||||||||||||||||||||||||||
| 041 | Lcls2XilinxKcu1500Pgp2b | used in the spare project XPM:0 | |||||||||||||||||||||||||||||||
| 042 | DrpPgpIlv | used in the spare project XPM:0 | |||||||||||||||||||||||||||||||
| 043 | DrpTDet | XPM:0 | EpixHrEmu | ||||||||||||||||||||||||||||||
| 044 | DrpTDet | XPM:0 | Laser hall | ||||||||||||||||||||||||||||||
| 045 | Todo: XilinxKcu1500Pgp4_6Gbps | XPM:0 | TXI Wave8 | ||||||||||||||||||||||||||||||
| mon001 | tcmp03mo meb and ami_manager neh-base.cnf, hsd.cnf, rix-hsd.cnf | ||||||||||||||||||||||||||||||||
| 046 | DrpPgpIlv | TMO HSD_15 and TMO HSD_16 | |||||||||||||||||||||||||||||||
| 048 | DrpPgpIlv | TMO HSD_17 and TMO HSD_18 | |||||||||||||||||||||||||||||||
| 049 | DrpPgpIlv | not working, possibly defective KCU | |||||||||||||||||||||||||||||||
| 050 | DrpPgpIlv | TMO HSD_01 and TMO HSD_19 | |||||||||||||||||||||||||||||||
| 051 | |||||||||||||||||||||||||||||||||
| 052 | |||||||||||||||||||||||||||||||||
| 053 | |||||||||||||||||||||||||||||||||
| 054 | |||||||||||||||||||||||||||||||||
| 055 |
| drp-srcf-gpuNNN | KCU firmware | Comment | Application |
|---|---|---|---|
| 001 | in progress | Adding kcu1500 | |
| 002 | n/a | ||
| 003 | n/a | ||
| 004 | n/a |
Cabling
Top Patch Panel:
NOTE: the rightmost MPO connections are MPO12 (2 rows, 4 columns). The left ones are MPO8. We should use MPO8 for KCU. The MPO12 are reserved for "future expansion" (can be converted to MPO8, I think).
(6-strand breakout cable) | DON'USE FOR NOW | DON'T USE FOR NOW | (breakout cable) 1: cmp008 2: cmp010 3: cmp016 4: cmp025 this connection is presumably been moved from the last one of this row (RM) | (breakout cable) 1: cmp022(away from USB port) 2: cmp022(close to USB port) 3: cmp023 (away from USB port) 4: cmp023 (close to USB port) | (breakout cable) 1: 2: 3: cmp019 (away from USB port) 4: cmp019 (close to USB port) | (breakout cable) 1: cmp009(away from USB port) 2: cmp009(close to USB port) 3: cmp017 (away from USB port) 4: cmp017 (close to USB port) | (breakout cable) | ||||
| (break cable) 1: cmp033 (away from usb port) 2: cmp035 (away from usb port) 3: cmp036 (away from usb port) 4: cmp032 (away from usb port) | (breakout cable) 2: cmp046 (away from usb) | (breakout cable) 1: cmp024(away from USB port) 2: cmp024(close to USB port) 3: -cmp018(away from USB port) 4: -cmp018(close to USB port) | (breakout cable) 1: cmp020(away from USB port) 2: cmp020(close to USB port) 3: cmp021 (away from USB port) 4: cmp021 (close to USB port) | (breakout cable) 1: cmp029 2: cmp030 3: cmp037 4: cmp038 | (breakout cable) 1: cmp028 2: - 3: - 4: - | ||||||
(breakout cable) |
Bottom Patch Panel (MPO):
| cmp006 (timing) (close to the USB port) | cmp011 (timing) (close to the USB port) | cmp012 (timing) (close to the USB port) | cmp004(close to usb port) | cmp013(timing) (close to the USB port) | cmp026(timing) (close to the USB port) | cmp027(timing) (close to the USB port) | cmp004(away from usb port) |
| cmp006 (data) (away from USB port) | cmp011 (data) (away from USB port) | cmp012 (data) (away from USB port) | cmp013 (data) (away from USB port) | cmp045 (away from USB port) | cmp026 (data) (away from USB port) | cmp027 (data) (away from USB port) | cmp014(away from usb port) |
Bottom Patch Panel (LC):
| IT | IT | cmp007 (close to the USB port) | cmp001 (away from the USB port) | |||
| IT | cmp007 (away from the USB port) | cmp002 (away from the USB port) | ||||
| IT | cmp003 (away from the USB port) | |||||
| IT | cmp005 (close to the USB port) | |||||
| IT | cmp005 (awy from the USB port) | |||||
| IT |
LC
Fibers 15-16: XTCAV (SXR, cpo believes) from Room 208 Rack 2 PH2 fibers 61-62 → cmp001. 61/62 means sixth cassette from the left, top fiber.
Fibers 21-22: MonoEncoder from ATCA Switch in FEE → cmp002
TMO Hutch Fiber Connections
| Source | Destination | Notes |
| XPM 2 input on RTM | Rack 4 in TMO hutch, FODU2, Cassette 4, fibers 3-4 (a.k.a TMO2.4.3-4) | Note that cassettes 3 and 4 are swapped between rack 3/4 |
| XPM 2 network connection | Rack 3, Cassette 3, fibers 23-24, goes to Rack 4 FODU 5, Cassette 4, fibers 23-24, goes to Rack 4 FODU 2 cassette 4 fibers 5-6 | Note that cassettes 3 and 4 are swapped between rack 3/4 |
RIX Hutch Fiber Connections
| Source | Destination | Notes |
|---|---|---|
| XPM 3 network connection | Rack 1 FODU Cassette 1 fibers 15-16 | Don't immediately know what the official fodu number is |
| XPM 3 AMC1 port 5 counting from 0 | Room 208 RIX1 fodu, cassette 1 fibers 11-12 | Wasn't connected on the room 208 side on Oct. 26, 2022. Maybe not necessary any more? |
MFX Hutch Fiber Connections
| Source | Destination | Notes |
|---|---|---|
| BOS 1.8.1, 1.8.2 and XPM7 output | Building B950 RB02 in room 208: SM S04 To B999 MEZZ FODU 43 R11, Ports 1,2,4 | 3 fiber pairs for epixHR data (two pairs) and timing (one pair) |
XTCAV Fiber Route
SXR
From Matt: I tried to look up the exact path of the xtcav fiber, but couldn't trace it all the way through captar. I know it comes from B921 Rack 73 to B921 BDF1; Chris and I traced this fiber. From there it goes to B005 Rack 611 and then to B950-R42 (now renumbered as R02) at elevation 36, I believe. There are lots of fibers at that location, so we may need two people to locate it - hold a strobe to one end and look for it on the other end.
And rougher notes that cpo doesn't currently understand:
rack B921-BDF1
SM slot 3 to B005 FODU 3 RK 611
fibers 3 4 in fodu
In building 921 rack 73 the DMPH (HXR) server is at height 24 and DMPS (SXR) server is at height 22.
HXR
TBD, but presumably similar to SXR.
Pictures of B921 Rack 73
Front and back
Room 208 Fiber Connections
See FEE Alcove for description of which patch panel in 208 goes to which detector. Note that the fiber numbering on that page is different from here. On the FEE alcove page we number fibers left-to-right, then top-to-bottom. Here we use a better scheme: using the actual fiber numbers marked on the cassettes. We should convert the FEE-alcove page to use the better fiber-numbering scheme.
| Source | Destination | Notes |
|---|---|---|
| ATCA FEE router port 0 | ATCA room 208 router port 1 | 10Gb network to SRCF node, the connection is done via the octopus fiber #4 |
| room 208 ATCA crate switch optical fiber port 0 | SRCF LC cassette 4 (counting from 1, left to right) fibers 11/12 | 10Gb network to SRCF node (drp-srcf-cmp004?) |
| TMO XPM 2 network connection (TMO FODU 1, Cassette 4 fibers 5-6) | ATCA switch optical port 7 counting from 0 | needed to swap tx/rx fibers to get link-lock |
| RIX XPM 3 network connection (RIX FODU, cassette 1, fibers 15-16) | ATCA switch optical port 6 counting from 0 | |
| Mono Encoder Network | From FEE Fodu 1 cassette 3 fibers 1,2 to SRCF LC fodu fibers 21,22 | |
| Timing output of XPM2 in 208 | TMO FODU 1, Cassette 3, Fibers 17-18: 1.3.17-18 | Cassette 1 is missing |
| Timing output of XPM3 in 208 | RIX FODU 1, Cassette 1, Fibers 7-8: 1.1.7-8 | |
| XPM0-AMC0-PORT0 | RIX 1.1.3-4 | This is XPM0 → XPM3 in RIX, FODU 1 for RIX is FODU 5 in 208 |
| XPM0-AMC0-PORT1 | TMO 2.4.3-4 | This is XPM0 → XPM2 in TMO, FODU 2 for TMO is FODU 7 in 208 |
| XPM0-AMC1-PORT2 | TMO 2.3.33-34 (tmo cassette 3 fiber 33-34) | Backup connection for XPM0->XPM2 |
| XPM5-AMC0-PORT0 | RIX 1.1.3-4 | This is XPM3→ XPM5 |
| XPM6-AMC0-PORT0 | TMO 1.3.17-18 | This is XPM2→ XPM6 |
| XPM0-AMC1-PORT0 | FEE FODU 1, Cassette 4 fiber 11-12 (top right) | LCLS2 timing for FEE Teststand |
| TMO ATM CAMLINK | TMO FODU 1 cassette 3 fiber 1-2 (a.k.a. TMO1.3.1-2) | cassette 1 is missing, swapped tx/rx at bos |
| RIX FIM MR4K2 | RIX FODU 1 cassette 1 fibers 25-26 (a.k.a RIX1.1.25-26) | |
| RIX FIM MR3K2 | RIX1.1.27-28 | |
| RIX FIM CHEMRIX | RIX1.1.29-30 | |
| TMO FIM1 | TMO1.3.5-6 | Not sure which FIM this really corresponds to in TMO |
| TMO FIM2 | TMO1.3.7-8 | Not sure which FIM this really corresponds to in TMO |
| RIX HSD2 | RIX1.1.19-20 | I may be swapping hsd2/3 (this was FEE fodu 1.3.18, old numbering scheme) |
| RIX HSD3 | RIX1.1.23-24 | I may be swapping hsd2/3 (this was FEE fodu 1.3.15, old numbering scheme) |
| TMO CAMLINK OPAL1 | TMO1.4.1-2 | swapped tx/rx at bos |
| TMO CAMLINK OPAL2 | TMO1.3.3-4 | swapped tx/rx at bos |
| TMO CAMLINK FZP | TMO1.2.11-12 | swapped tx/rx at bos |
| RIX CAMLINK ATM | RIX1.3.13-14 | |
| TMO HSD3 or companion | TMO1.4.13-14 | I may be swapping hsd3 with its companion (don't know the name: not in tmo.cnf?). This one was in fee fodu1.1.11 (old numbering scheme). Needed tx/rx swap at bos. |
| TMO HSD3 or companion | TMO1.4.15-16 | I may be swapping hsd3 with its companion (don't know the name: not in tmo.cnf?). This one was in fee fodu1.1.9 (old numbering scheme). Needed tx/rx swap at bos. |
| TMO HSD5 or HSD7 | TMO1.4.31-32 | I may be swapping hsd5/hsd7. This one was in fee fodu 1.1.1 (old numbering scheme). Needed tx/rx swap at bos. |
| TMO HSD5 or HSD7 | TMO1.4.29-30 | I may be swapping hsd5/hsd7. This one was in fee fodu 1.1.5 (old numbering scheme). Needed tx/rx swap at bos. |
| TMO HSD4 | TMO1.4.27-28 | Was in fee fodu 1.1.4 (old numbering scheme). tx/rx swap at bos. |
| TMO HSD4 companion | TMO1.4.33-34 | Was in fee fodu 1.1.6 (old numbering scheme). tx/rx swap at bos. |
| TMO HSD1 | TMO1.4.17-18 | Was in fee fodu 1.1.12 (old numbering scheme). tx/rx swap at bos. |
| TMO HSD2 | TMO1.4.19-20 | Was in fee fodu 1.1.8 (old numbering scheme). NO tx/rx swap at bos! |
| TMO HSD6 | TMO1.4.35-36 | Was in fee fodu 1.1.3 (old numbering scheme). tx/rx swap at bos. |
| TMO HSD6 companion | TMO1.4.25-26 | Was in fee fodu 1.1.2 (old numbering scheme). tx/rx swap at bos. |
| TMO HSD9 | TMO1.4.21-22 | Was in fee fodu 1.1.7 (old numbering scheme). tx/rx swap at bos. |
| TMO HSD8 | TMO1.4.23-24 | Was in fee fodu 1.1.10 (old numbering scheme). tx/rx swap at bos. |
| RIX HSD1 | RIX1.1.21-22 | Was in fee fodu 1.3.8 (old numbering scheme) |
| RIX HSD0 | RIX1.1.17-18 | Was in fee fodu 1.3.9 (old numbering scheme) |
| PEPPEX HSD1 | Room 208 Rack 2 PH4 "To B920 R90-S SLOT B" fibers 21-22 (second from bottom). Also labelled "slot 4" on the slot itself. | Was in fee fodu 2.4.3 (old numbering scheme) |
| PEPPEX HSD0 | Room 208 Rack 2 PH4 "To B920 R90-S SLOT B" fibers 19-20 (third from bottom). Also labelled "slot 4" on the slot itself. | Was in fee fodu 2.4.4 (old numbering scheme) |
| PEPPEX OPAL | Room 208 Rack 2 PH4 "To B920 R90-S SLOT B" fibers 23-24 (bottom pair). Also labelled "slot 4" on the slot itself. | Was in fee fodu 2.4.5 (old numbering scheme) |
| XTCAV (SXR, cpo believes) | Room 208 Rack 2 PH2 fibers 61-62 from B005 (each cassette has 12 fibers, so top of sixth cassette from left) | Was in fee fodu 1.2.2 (old numbering scheme) |
| TXI FIM | Room 208
Data To FODU 8 Cassete 2 Port 11-12 | |
| TMO LCLS1 timing | XPM7 AMC1 port1 to TMO FODU 7 Cassette 3 (counting from 1 with 4 cassettes total, one currently unoccupied) fibers 11-12 | |
| XPM0-AMC1-PORT4 | FEE FODU 6, Cassette 3 fiber 23-24 (lower right) | LCLS2 timing for FEE Teststand XPM10 RTM |
| TMO HSD 88 or 89 | BOS (2.1)1-2 or 5-6 | In TMO Hutch connected to FODU1 Cassette 1 position 8 (most left in the middle row might be broken) and 15 (lowest row int he middle) RACK 4 in 208 FODU 7 RACK 4 |
| TMO HSD 89 or 88 | BOS(2.1) 5-6 or 1-2 | in TMO Hutch connected to FODU1 Cassette 1 position 9 and 10 RACK 4 in 208 FODU 7 RACK 4 N.B. position 9 (in the middle row third from the left) was already taken in 208 but not in TMO. Connected to FODU4 (208) cassette 2 (second from the left) position 7 (first from the left in the middle), but removed because no fiber in TMO. |
| TMO HSD 1 (1B:B) | BOS 1.8.3 | In TMO hutch connected to FODU1 Cassette 1 position 3 (counting from top left from 1) RACK4 |
| TMO HSD 18 (B1:B) | BOS 1.8.4 | In TMO hutch connected to FODU1 Cassette 1 position 4 (counting from top left from 1) RACK4 |
| RIX High-rate encoder in FEE (data and timing, in that order) | From FEE: B940-008-R03-FODU-U1 PAIRS 2 and 3 COUNTING FROM 1 Through FEE alcove: B940-009-R06-FOD3-U1 pairs 2 and 5 counting from 1 To B950-208-r42-FOD3-U2 PAIRS 2 and 5 COUNTING FROM 1 | Swapped tx/rx at xpm5 amc1 port 2 (counting from 0). Data also needed to be swapped going into BOS (1.7.8). |
XPM Topology
| Code Block |
|---|
lcls2Timing -> xpm0 -> 2 (tmo) -> 4 (tmo)
-> 6 (208) -> BOS
-> 3 (rix) -> 5 (208) -> BOS
-> 10(fee) -> 11(fee)
lcls1Timing -> xpm7 |
Problematic connections SRCF ↔ 208
- DRP-SRCF-CMP018-QSFP1 → 5.3.6 (Shows -7.27, is an HSD)
- DRP-SRCF-CMP008 (reported by ric): Connecting DRP-SRCF-CMP008.0_TIMING (5.5.1) to XPM-6_AMC-0_SFP-1 (5.8.4) doesn’t bring up ‘TDetSim/cmp008’ in xpmpva and shows LinkRxErrs
- move base processes to mon001
Cleanup:
- old neh cmp013 going to xpm3 amc0 port 5
- old neh cmp015 going to xpm3 amc1 port 0
- LC patch panel connections to SRCF
- leftover long fibers in 208
Overview
Content Tools