Versions Compared

Old Version 12

changes.mady.by.user Larry Ruckman

Saved on

compared with

New Version Current

changes.mady.by.user Larry Ruckman

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Publications

Requirements

  • Support targeting "point-to-point" and "
    • But network
    " mode
    • "point-to-point" by default (better logic optimization)mode is possible
  • 3 layer of Ethernet:
    • Layer1: PHY
    • Layer2: MAC/FEC
    • Layer3: PGP ETH V1HTSP

  • Layer 3 Ethernet framing

    • Wrapper on the Ethernet MAC
    • Targeting 100G Hard IP MAC
      • 512-bit AXI Stream interface
      • IEEE 802.3bj Reed-Solomon Forward Error Correction (RS-FEC)
      • Frame Check Sequence (FCS) checking, adding and deleting
  • 512-bit header and 48-bit footer
    • Header and payload always 512-bit wide
    • Simplify the logic for 100GbE MAC interface (no AxiStreamShifting required because matched to MAC's AXI stream 512-bit width)
  • Chuck up the stream to MAX_SIZE =1024B (8196B default)
    • 92.1% efficient = 1024B/(1024B + 4B FCS + 12B intergap + 8B preamble + 64B PGP Ethernet Header)
    • 9598.9% efficient = 2048B8196B/(2048B 8196B + 4B FCS + 12B intergap + 8B preamble + 64B PGP 64B HTSP Ethernet Header )
    • 97.9% efficient = 4096B/(4096B + 4B FCS + 12B intergap + 8B preamble + 64B PGP Ethernet Header)
    • 98.9% efficient = 8196B/(8196B + 4B FCS + 12B intergap + 8B preamble + 64B PGP Ethernet Header+ 4B HTSP Ethernet Footer)
    • Smaller MAX_SIZE lower the latency for publishing the virtual channel "pause" status to the remote side but increases overhead
  • Use the MAC's FCS for error checkingIn both point-to-point and network mode, using the global ETH pause (IEEE 802.3x) to flow control the Ethernet layer1
    Not using Priority Flow Control (IEEE 802.1Qbb)
  • Virtual channel flow control done at the L3 layer

...

  • (instead of ETH pause)
  • Support up to 16 Virtual channels on the same link
    • Same as PGPv4


Word#Word's BYTETypeNameDescriptionNote
0[5:0]HeaderDestMacDestination MAC
  • Network mode only
0[11:6]HeaderSrcMacSource MAC
  • Network mode only
0[
15
13:12]HeaderEtherTypeTBD Value
  • Network mode only
014
16
HeaderVersion0x1
  • Hard coded
0
17
15HeaderTIDTransaction ID
  • Increments once per packet
  • Used for out-of-order frame reorganizing in network mode
0[
19
17:
18
16]HeaderPauseVirtual Channel Pause
  • 1 bit per Virtual Channel
0
20
18HeaderVCVirtual Channel Index
  • BIT[3:0]: Virtual Channel Index
  • BIT[7:4]: Reserved
019
21
HeadertUserFirstfirst 8-bits of tUser
 

0
0
20
22
Header
tUserLast
OpCodeEn
last 8-bits of tUser 
OP-code Enable
0[
31
29:
23
20]HeaderReservedReserved
  • Zeros
0[31:30]HeaderHdrXsum16-bit Header Checksum
  • Header Only
0[47:32]HeaderOpCodeData128-bit OP-code Data
  • Supports 64-bit timestamp + more information
0[63:48]HeaderUserData128-bit User Data
  • Sampled every packet sent
1[63:0]PayloadAXIS DataUser Payload Data
 

...............
 

(MAX_SIZE/64)[63:0]PayloadAXIS DataUser Payload Data
(MAX_SIZE/64)+10FootertKeepLastTKEEP on last payload word
  • Converted from byte map to an integer
(MAX_SIZE/64)+11FooterEOF/EOFEEOF and EOFE marker
  • BIT0 = EOF, BIT1 = EOFE
(MAX_SIZE/64)+1[3:2]FooterPauseVirtual Channel Pause
  • paused bits are sampled and latched during the payload transport
(MAX_SIZE/64)+1[5:4]FooterPayloadSizeNumber of bytes in payload
  • Units of bytes


Performance

Image Added

Going "point-to-point" we were measuring 97.3 Gb/s bandwidth for 1MB frames using 8196B HTSP burst size.  97.3 Gb/s is < 98.9% theoretical limit, which we think is related to some unaccounted inefficiency in the CMAC4 hard IP core that we used in this testing.  The DMA bandwidth limit in this test was 101 Gb/s, which means the DMA was not back pressuring on the HTSP link.  Auto-polling register channel access on the HTSP link on a different virtual channel was enabled in this testing.