Link To IEEE Poster

Introduction

RSSI is a simple packet based transport protocol. RSSI is based on RFC-908, RFC-1151, draft-ietf-sigtran-reliable-udp-00 Reliable Data Protocols and modified internal simplifications and improvements. The communication interface is SSI (AXI-Stream based SLAC Streaming protocol). RSSI can be layered on the UDP/IP Protocols or attached to other custom AXI-Stream communication layer. It provides reliable in-order delivery (up to a maximum number of re-transmissions) for virtual connections.

The protocol is connection and packet based. Initially a connection has to be established between a Client and a Server. Connection procedure exchanges and negotiates parameters such as sequence number, timeouts, max outstanding segments, etc.  If the parameters are compatible the connection is open. An open connection allows full duplex communication between the server and the client where both sides have to acknowledge the received segments.

The transmitting side is buffering sent segments until it gets the acknowledgment from the peer. If the buffer becomes full (max number of outstanding segments reached) the transmitting side stops sending segments. After the re-transmission timeout is reached it re-sends all the buffered segments. It keeps retrying until it receives a valid acknowledgment from the peer. If the max number of re-transmissions is reached the connection is closed.

The receiving side verifies the received segments and acknowledges the segments if received in order. If the segment is received out of order the segment is dropped and the receiver waits for the re-transmission before it sends the next acknowledgment. No out of order Acknowledgments are supported in the current version of the protocol.

The RSSI connection has to be maintained. If there is no data flow the Client has to keep transmitting empty segments (NUL) to the server to actively maintain the connection. If the client fails, the server timeouts, and the connection is closed.

Segment formats

General Segment format

Each RSSI segment starts with an eight byte header shown in Figure 1 (SYN segment contains a longer header). The first byte contains a series of single bit flags. The next three fields are each one byte in size. They are: Header length, Sequence number, and Acknowledgment number. These three bytes are followed by a spare 16-bits and two byte checksum.

Note: Spare bits are added for two reasons hardware implementation simplification (64-bit word HW processing) and possible future format extension.

Figure 1: General header structure

Control bits

The control bits indicate what is present in the packet.

• SYN bit indicates that a synchronization segment is present.
• ACK bit indicates the acknowledgment number in the header is valid.
• EAC bit is not used in this version of the protocol. Reserved for an extended acknowledge segment indication.
• RST bit indicates the packet is a reset segment.
• NUL bit indicates the packet is a null segment.
• BUSY bit indicates the receiver is busy.

Some of the control bits can be combined to carry as much information in the single segment as possible. The SYN, BUSY and RST are mutually exclusive. The ACK bit is always set when the NUL bit is set. A segment with user data in it always has the ACK bit set and it is called a DATA segment. User data cannot be present in packets with the NULL, BUSY, or RST bits set.

Header length

The Header length field indicates where user data begins in the packet. If total length of the packet is greater than the value of the header length field, user data exists in the packet.

Sequence number

Each segment contains a sequence number. When a connection is first opened, each peer picks an initial sequence number (the initial sequence number can be randomly picked). This sequence number is used in the SYN segments to open the connection. Each transmitter increments the sequence number before sending a DATA, NUL, or RST segment.

Acknowledgment Number

The ACK number field indicates to a transmitter the last in-sequence segment the receiver has received.

Checksum

A checksum is calculated on the RSSI header to ensure integrity. The checksum algorithm used in RSSI is the same algorithm used in UDP and TCP headers which is the 16-bit one’s complement of the one’s complement sum of checked bytes. The header checksum calculation/verification can be turned off in SYN header properties.

Note: This version of the protocol does not support the payload checksum calculation.

Rationale: The payload checksum is already calculated by the UDP layer.

SYN Segment format

The SYN is used to establish a connection and synchronize sequence numbers between two hosts. All configurable RSSI parameters that the peer must know about are contained in this segment. This includes the maximum number of segments the local RSSI is willing to accept, various timeout values, and option flags that indicate the features of the connection being established. A SYN segment must not be combined with user data. The format is shown in Figure 2.

Figure 2: SYN header structure

Sequence Number

The sequence number field contains the initial sequence number selected for this connection.

Acknowledgment Number

This field is valid only if the ACK flag is set. In that case, this field will contain the sequence number of the SYN segment received from the other RSSI peer.

Version (non-negotiable)

The version field contains the version of RSSI. The initial version is one 0x1.

Note: If the version received by the peer is different the connection will be closed.

CHK flag (non-negotiable)

Checksum flag indicates if the header checksum is enabled ‘1’ or disabled ‘0’. This parameter is non-negotiable.

Note: If the CHK received by the peer is different the connection will be closed.

Maximum Number of Outstanding Segments (non-negotiable)

The maximum number of segments that can be sent by the transmitter without receiving an acknowledgment from the peer. This is used by the receiver as a means of flow control. The number is selected during connection initiation and may not be changed later in the life of the connection. This is a special non-negotiable parameter. Each side must use the value provided by its peer when sending data.

Note: The peer’s parameter will be accepted. If the received value is higher than the maximum segment buffer capacity the accepted value will be set to buffer max.

Maximum Segment Size (non-negotiable)

The maximum data segment size (in bytes) that can be received by the peer’s receiver. Each peer may specify a different value. Each peer must not send packets greater than the value of this field received from its peer during connection negotiation. This number includes the size of the does not include the RSSI header. This is a non-negotiable parameter.

Note: The peer’s parameter will be accepted. If the received value is higher than the maximum segment buffer capacity the accepted value will be set to buffer max.

Re-transmission Timeout (negotiable)

The timeout value for retransmission of unacknowledged segments. This value is specified in timeout units (timeout unit field). The valid range is 0x1 to 0xFFFF. This is a negotiable parameter, both peers must agree on the same value for this parameter.

Cumulative Acknowledgment Timeout (negotiable)

The timeout value for sending an acknowledgment segment if another segment is not sent. This value is specified in timeout units (timeout unit field). The valid range is 0x1 to 0xFFFF.This is a negotiable parameter, both peers must agree on the same value for this parameter. In addition, this parameter should be smaller than the Retransmission Timeout Value.

Null Timeout (negotiable)

Null Segment Timeout value is the server timeout value for monitoring an active connection. If no DATA or NULL packets are received in time the server closes the connection. Null Segment Timeout/3 is the Client timeout to send a null segment if a data segment has not been sent. The null segment acts as a keep-alive mechanism. This value is specified in timeout units (timeout unit field). The valid range is 0x1 to 0xFFFF. This is a negotiable parameter, both peers must agree on the same value for this parameter.

Maximum Number of Retransmissions (negotiable)

The maximum number of times consecutive retransmission(s) will be attempted before the connection is closed. The valid range for this value is 1 to 255. This is a negotiable parameter, both peers must agree on the same value for this parameter.

Maximum Number of Cumulative Ack (negotiable)

The maximum number of acknowledgments that will be accumulated before sending an acknowledgment if another segment is not sent. The valid range for this value is 0 to 255. A value of 0 indicates an acknowledgment segment will be send immediately when a data, null, or reset segment is received. This is a negotiable parameter, both peers must agree on the same value for this parameter.

Timeout unit (non-negotiable)

The unit is applied to all the RSSI timeouts. The value corresponds to (-log Unit). For microseconds (10E-6) the value would be 6. For millisecond (10E-3) the value would be 3.

Note: If the Timeout value received by the peer is different the connection will be closed.

Connection Identifier (negotiable)

When opening a new connection each peer transmits a connection identifier that is unique among all current RSSI connections. Each side then saves the connection ID received.

Connection

A connection is defined as a unique communication interface between two peers. The connection procedure handles RSSI parameter exchange and negotiation between a Client and a Server. The Client refers to the peer that initiates the connection and the Server refers to the peer that listened for a connection.

The connection FSM is shown in Figure 3. Connection procedure is as follows. At the server side the user has to request a passive connection. The server goes to the LISTEN state. This can also be done automatically by tying the connection request input to ‘1’. In this state the server will wait to receive the SYN segment from the client. At the Client side the user has to actively request a connection. After the request the Client sends the SYN segment to the Server. The Server receives the SYN segment and acknowledges it by sending a SYN segment back with a valid ACK number. If the SYN parameters are accepted by the Client the client then opens the connection. The Sever also opens the connection after receiving a valid ACK segment from the Client. See Figure 3 for more detailed description of the particular states and transitions.

Figure 3: Connection Finite State Machine

A part of the connection procedure is a parameter negotiation faze. Both Sever and Client have the option to accept or reject the parameters from the peer. Each peer checks the parameters values against the allowed limits. Some of the parameters are negotiable i.e. proposed by one peer and accepted by the other peer, and some are non-negotiable i.e. the values received have to be exact (correct). The parameters and their limits are listed in Table 1. The parameter negotiation differs if Server to Client. The Server receives the parameters from the Client and can decide to reject the parameters and propose different ones. The Client initially proposes the parameters from the Server and can then only accept or reject the received parameters.

Table 1: RSSI Parameters

Data flow

When the connection is established the data can flow in full duplex mode between the Client and the Server. Some of the examples of data flow follow. 

Half-duplex example

The data is flowing in only one direction from Peer 1 to Peer 2. In A) the Transmitter 1 sent four DATA segments with SEQ numbers from 203 to 206. The transmission is halted because no ACK has been received and the Transmitter buffer is full. Note that all the DATA segments contain the ACK number of 100. The Receiver 2 received the four DATA segments and, after one of the ACK conditions are met, sends back the ACK segment with the ACK number of the last segment received in order 206. Note that the ACK segment contains a SEQ number of 101 (indicating next data segment in the opposite direction).

In the Half-duplex case the adjustment of ACK conditions can change the amount of bandwidth or is used for flow control. In A) the Maximum Number of Cumulative Acknowledgments parameter is four or more and the Cumulative Acknowledgment Timeout is longer or equal to the reception time of four segments. This results in some dead time between the Transmitter buffer is full and getting the ACK. The ACK has to come before the Retransmission Timeout of Transmitter 1 (otherwise the segments would be transmitted).  In B) the ACK conditions are adjusted as such that the ACK is sent immediately after receiving the data segment and the data flow is thus continuous.

Full-duplex example

The data flows in both directions. Both Peer 1 and Peer 2 are sending the data segments with incrementing SEQ numbers. The received data segment already contains the ACK number (of the last segment received in order) within its header and in this example no standalone ACK segments are necessary.

Re-transmission

Re-transmission occurs if no ACK is received from the peer and the Re-transmission Timeout is reached. In the example below the DATA segment 203 was lost/corrupted. And the Peer 2 only received the next three segments 204 -205. Because the segment 203 has not been received no new ACK is send back. And the other out of order segments are dropped. After the Re-transmission Timeout the Transmitter 1 re-sends all the unacknowledged segments from its buffer. The Receiver 2 then accepts the re-transmitted segments and finally acknowledges the segments.

If there is no ACK the Transmitter resends the buffer up to Maximum Number of Retransmissions and closes the connection by sending the RST segment.

Note that RSSI does not support out of order acknowledgments.

Null Segments

If there is no data flow between the peers the connection has to be maintained with NUL segments. The NUL segment operation is different at the Server and Client side. The Server has a monitoring function while the Client has the “Heartbeat” function.

If there is no data the Client has to keep sending NUL segments to the Sever. The NUL transmission period is defined as 1/3 of Null Timeout parameter. The NUL segment has a SEQ number and has to be acknowledged by the Server. The Server has to send the ACK immediately after receiving the NUL segment (the Cumulative ACK parameters are here ignored). If no NUL or DATA segments are received within the duration of Null Timeout the Server detects inactivity and closes the connection.

Flow control using BUSY flag

Some flow control can be achieved by timing the acknowledgments but it is limited to the Retransmission time. Thus the RSSI enables flow control using a BUSY flag. This enables a slow/software receiver to control the DATA flow without risking broken connections due to Max Number of Retransmissions.

To apply flow control the receiver should apply BUSY flag to outgoing data segments (if there are any) or periodically send ACK segments with BUSY flag (recommended period Retransmission Timeout/2). The received busy flag will reset the re-transmission timer of PEER transmitter. The flow control example is shown below.

In Figure A) the Receiver 2 is busy and, to keep the Transmitter 1 from re-transmitting its Tx buffer, it periodically sends the ACK segments with BUSY flag. The BUSY segment has an ACK number of the last received segment before the receiver was busy (202). And a SEQ number of the next data to be sent in the opposite direction (101).

After the Receiver 2 is ready it reinitiates the data flow by sending the ACK segment with the last ACK number received in order (206). The Tx buffer of the Transmitter 1 is freed and it continues to send new segments.

Links

• MS Word form of this document: RssiDoc.docx

• Drafts: RFC-908, RFC-1151, draft-ietf-sigtran-reliable-udp-00.

• Performance for RSSI + UDP:
  
  • HW-to-HW Performance
  • SW-to-HW Performance
• Standard library documentation of RssiCore:
  • StdLib+RssiCore

Differences from RUDP (RFC/draft)

General deviation summary:

• No auto reset functionality.

• No transfer state.

• No out of sequence acknowledgments (EACK).

• Different header structure (see headers on figures below).

• Unit for the timeouts can be set in generic. The timeout unit is also communicated in SYN header.

• Value in header corresponds to (-log Unit). 
Example Unit 10E-6. Value =6.
• Minor deviation in re-transmission mechanism (Re-transmission mechanism below).
• Improved flow control using BUSY flag (Flow control using BUSY flag below). 

  Handling of BUSY/flow-control implemented from server->client only (i.e., the client may signal BUSY; the server never does).

• Version without re-transmissions. Implemented only in firmware transmitter module.
  Software client should be implemented to "Smart" ACK even though the segments are 
  missing or out of order (Version with  no re-transmissions below).

Contact

Uros Legat

ulegat@slac.stanford.edu