namespace RCE::chantype {

    enum {
        ETHERNET,
        CONFIG_FLASH,
        ...
    } Number;
}

class ChannelRegistry {

    ChannelRegistry();

    // Register a channel and assign it a sequence number i starting from zero.
    // The argument may not be a null pointer.
    void addChannel(Channel*);

    int numChannels();

    // If i is a valid sequence number then return the i'th channel
    // else return a null pointer.
    Channel* factory(int i);

);

typedef uint32_t StringOff; class Channel {
public:

    ChannelFactory* factory() const; // OffsetThe withinfactory thethat Strings table.

struct Channel {created me.

    uint32_t  keyunsigned lanesUsed() const;

    uint32_t  factoryKey;     // Ref. to Factories table.unsigned pibsUsed() const;

    unsigned pebsUsed() const;

    unsigned ecbsUsed() const;

    unit32_t  lanesUsed;unsigned flbsUsed() const;

    unsigned  // Bit-mask of lanes used.
    uint32_t  pibsUsed;   pipeUsed() const;

    const char* description() const;

    unsigned orphanFrameCount() const;

    // Bit-mask of Pending Import Blocks used.
    uint32_t  pebsUsed;       // Bit-mask of Pending Export Blocks used.
    uint32_t  ecbsUsed;    Create a Port with a specified port number.
    // Returns null if the number is already taken.
    virtual Port* createPort(int portNum) = 0;

    // Create a Port with a number assigned by the channel.
    // Bit-mask of Event Completion Blocks used Returns null if no port number is free.
    uint32_t  flbsUsed;   virtual Port* createPort() = 0;

    virtual void destroyPort(Port *) = 0;

    // Bit-mask of Free List Blocks used.
 Return null or the Port with the given number.
    Port StringOff description;
};

*getPort(unsigned portNum) const;

    // Start managing a new buffer or reclaim an old one.
    void claimBuffer(void *);

    void enable();

    void disable();

    void isEnabled() const;

    // Return null or a pointer to the Port that should
    // receive the data.
    virtual Port* deducePort(void *frameHeader) const;

protected:
    // Create a Channel which is initially in the disabled state.
    Channel(
        unsigned channelTableKey,
        RCE::config::Tables&,
	ChannelFactory *,
    );

  virtual ~Channel();

};

structclass FactoryPort {
public:
  Port(Channel *, uint32_t  keyunsigned portNumber);
  Channel*  unit32_tchannel() const;
  typeNumber;unsigned portNumber() const;
  virtual  // The official channel type number.
    uint32_t  firmwareVersionvoid* wait(); // FirmwareReturns versiona number.
pointer to the frame uint32_theader when headerSize;one      // Bytesbecomes available.
  void  uint32_t  maxPayloadSize;  // Bytes.
    uint32_t  containerNameclaimBuffer(void *frameHeader) const;
};

struct DataPath {
    uint32_t  key;
    uint32_t  channelKey;
    uint32_t  pipeUsed;    // Where the channel's signals "come out."
};

The Factories table:

The Data Paths table:

How the tables appear in RAM

We copy the tables without alteration and set the members of an instance of the following structure:

struct Tables {
    unsigned numChannels;
    Channel *channel;      // Pointer to first element of table.
    unsigned numFactories;
    Factory *factory;
    unsigned numDataPaths;
    DataPath *dataPath;
};

Use case: System startup

We'll need to use a quick and dirty way to read the configuration tables from flash; either that or just use faked-up configuration tables to create the config. flash Channel object. Once we can have that we can create the rest of the Channel objects.

1. Boot code
   1. Loads and starts the system core.
2. System core
   1. Initializes the CPU.
   2. Initializes RTEMS.
   3. Initializes any extra C++ support.
   4. Sets up the MMU's TLB and enables the MMU.
   5. Uses a basement level technique to read the tables from configuration flash.
   6. Creates and registers the channel factory for configuration flash.
   7. Creates and registers the configuration flash channel.
   8. Allocates frame buffers in uncached RAM for config. flash pages.
   9. Gives the frame buffers to the configuration flash channel and enables it.
   10. Creates the default instance of the dynamic linker.
   11. Loads and links the factory code marked as loadable in the Factories table.
   12. Calls the entry points in the factory code modules just loaded.
   13. Iterates on the Channels table and creates all Channel objects.
   14. Iterates over all Channel objects (except config. flash) to get their recommended buffer allocations.
   15. Allocate the buffers in uncached memory.
   16. Assigns buffers to channels.
   17. Enables all Channel objects not already enabled.
   18. Loads and links the application code using the default dynamic linker.
   19. Initializes the network.
       1. Initializes each ethernet channel.
       2. Initializes IP, UDP, TCP and BSD sockets.
       3. Gets a DHCP lease if required.
   20. Calls the application entry point.
3. Factory code entry point.
   1. Creates a channel factory object using the given configuration table info as input.
   2. Registers the object with the core.

Use case: Frame import

TBD.

Use case: Frame export

TBD.

Low-level API

Official RCE channel type numbers

These are given in a header file made available to both core and application code. The numbers are members of an enumeration.

namespace RCE::chantype {

    enum {
        ETHERNET,
        CONFIG_FLASH,
        CATCH_ALL,
        ...
    } Number;
}

Channel factory registry

An instance of this class will be exported by the core code using some design pattern such as Singleton or functional equivalent. This instance is created during system startup and lasts until system shutdown. Assignment or copying of instances is forbidden.

class ChannelFactoryRegistry {

    ChannelFactoryRegistry();

    // Register a factory and assign it a sequence number i starting from zero.
    // The argument may not be a null pointer.
    void addFactory(ChannelFactory*);

    int numFactories();

    // If i is a valid sequence number then return the i'th factory
    // else return a null pointer.
    ChannelFactory* factory(int i);

);

Channel registry

An instance of this class will be exported by the core code using some design pattern such as Singleton or functional equivalent. This instance is created during system startup and lasts until system shutdown. Assignment or copying of instances is forbidden.

class ChannelRegistry {

    ChannelRegistry();

    // Register a channel and assign it a sequence number i starting from zero.
    // The argument may not be a null pointer.
    void addChannel(Channel*);

    int numChannels();

    // If i is a valid sequence number then return the i'th channel
    // else return a null pointer.
    Channel* factory(int i);

);

Channel factory

The abstract base class for objects that manufacture Channel instances. Channel factories are created during system startup and last until system shutdown. Assignment or copying of instances is forbidden.

Derived classes add the type name, the recommended number of buffers and the description string to the information gotten from the configuration tables.

class ChannelFactory {
public:

    RCE::chantype::Number typeNumber() const;

    const char* typeName() const;

    unsigned firmwareVersion() const;

    unsigned headerSize() const;

    unsigned maxPayloadSize() const;

    unsigned numBuffersAdvised() const;

    unsigned codeContainerName() const;

    const char* description() const;

    virtual Channel* createChannel(
        unsigned channelTableKey,
        RCE::config::Tables&
    ) = 0;

protected:

    ChannelFactory(
        unsigned factoryTableKey,
        const RCE::config::Tables&,
	const char* typeName,
	unsigned numBuffersAdvised,
	const char *description
    );

    virtual ~ChannelFactory();
};

Channel

A channel object manages a set of I/O buffers and a particular instance of a protocol plug-in. At the beginning of each buffer the channel will build a FrameBuffer object. Each channel object is created at system startup, initially in a disabled state. Afterward the system startup code will allocate the buffers, feed them to the channel objects and then enable them. Channel objects live until system shutdown.

Each channel creates and destroys Port objects on demand. Each port is assigned one of the channel's legal port numbers not used by any other port. The client code may request a port number for the type of channel, e.g., a well-known TCP port number. The client may also allow the channel to assign an ID not currently in use by any port.

A channel knows how to derive a port number from the header of a frame it has received.

Channel and its derived classes do not allow copying or assignment of their instances.

class Channel {
public:

    ChannelFactory* factory() const; // The factory that created me.

    unsigned lanesUsed() const;

    unsigned pibsUsed() const;

    unsigned pebsUsed() const;

    unsigned ecbsUsed() const;

    unsigned flbsUsed() const;

    unsigned pipeUsed() const;

    const char* description() const;

    unsigned orphanFrameCount() const;

    // Create a Port with a specified port number.
    // Returns null if the number is already taken.
    virtual Port* createPort(int portNum) = 0;

    // Create a Port with a number assigned by the channel.
    // Returns null if no port number is free.
    virtual Port* createPort() = 0;

    virtual void destroyPort(Port *) = 0;

    // Return null or the Port with the given number.
    Port *getPort(unsigned portNum) const;

    // Start managing a new buffer or reclaim an old one.
    void claimBuffer(void *);

    void enable();

    void disable();

    void isEnabled() const;

    // Return null or a pointer to the Port that should
    // receive the data.
    virtual Port* deducePort(void *frameHeader) const;

protected:
    // Create a Channel which is initially in the disabled state.
    Channel(
        unsigned channelTableKey,
        RCE::config::Tables&,
	ChannelFactory *,
    );

  virtual ~Channel();

};

Implementation

General

Bit numbering: Bit zero is the least significant bit of a value (contrary to the convention in PowerPC documents).

Channel factory

The abstract base class for objects that manufacture Channel instances. Channel factories are created during system startup and last until system shutdown. Assignment or copying of instances is forbidden.

Derived classes add the type name, the recommended number of buffers and the description string to the information gotten from the configuration tables.

class ChannelFactory {
public:

    RCE::chantype::Number typeNumber() const;

    const char* typeName() const;

    unsigned firmwareVersion() const;

    unsigned headerSize() const;

    unsigned maxPayloadSize() const;

    unsigned numBuffersAdvised() const;

    unsigned codeContainerName() const;

    const char* description() const;

    virtual Channel* createChannel(
        unsigned channelTableKey,
        RCE::config::Tables&
    ) = 0;

protected:

    ChannelFactory(
        unsigned factoryTableKey,
        const RCE::config::Tables&,
	const char* typeName,
	unsigned numBuffersAdvised,
	const char *description
    );

    virtual ~ChannelFactory();
};

Virtex-4 (Gen I)

Frames are still divided into header and payload sections.

Location and format of the configuration tables

A data container in the configuration flash contains tables of information about the hardware and firmware; this information can't be gotten directly from the Virtex-4 hardware and firmware.

The tables are called PpiDefs and Pipes. Each table is an array of plain-old-data structs. To make alignment easier we use 32-bit fields wherever possible, even for 16-bit quantities such as port numbers. All of the declarations for the configuration structs are in namespace RCE::config.

The PpiDefs table will come first, at the beginning of the container, followed by the Pipes table.

PpiDefs table

The PpiDefs table will have eight elements, one for each possible PPI in the system. An element is considered unused if the type member contains 0xffffffff (EMPTY) in which case all the other members of that element are zero.

struct PpiDef {
    unit32_t  type;
    unit32_t  lanes;
    uint32_t  version;
    uint32_t  maxPorts;
    uint32_t  pibs;
    uint32_t  pebs;
    uint32_t  ecbs;
    uint32_t  flbs;
    uint32_t  payloadSize;
    uint32_t  headerSize;
};

Member descriptions (when type != EMPTY):

• type. The type number of the resulting Channel object as given in the header ChannelTypes.hh.
• lanes. Bit N is set to indicate that lane N is used by this PPI.
• version. The version number for the PPI firmware.
• maxPorts. The size of the port number space.
• pibs, pebs, ecbs, and flbs. Bit N is set to indicate that the PPI uses the N'th PIB, PEB, ECB or FLB, respectively.
• payloadSize. The maximum number size in bytes of a frame payload.
• headerSize. The size of a frame header.

Pipes table

In order to discover which pipe PPI uses will use you take the lowest lane number used by the PpiDef entry and use it to index the Pipes table, which has 32 elements. A pipe number of 0xffffffff indicates an unused element, otherwise the ID of the pipe is given. A pipe ID will probably contain several subfields telling what kind of target the pipe connects to, etc.

struct Pipe {
    uint32_t  pipeId;
};

Class FrameBuffer

A FrameBuffer instance is placed at the beginning of a buffer that will also hold the frame proper. The instances are created at system startup and live until system shutdown.

A FrameBuffer contains regions of fixed size used in the management of the frame itself and its buffer:

• One or more links used to make singly-linked lists.
• The firmware's in-memory descriptor. The firmware TDE for the frame points to the descriptor.
• Status. The firmware writes operation completion status here.

Immediately following the end of the FrameBuffer buffer in we have first the frame's header and then its payload.

The links, descriptor and status areas have the same sizes for all channel types.

The descriptor must begin on a 64-byte boundary; for ease of layout the entire buffer also has that alignment. The buffer should also begin on a cache line boundary in order to reduce cache thrashing but since a cache line is only 32 bytes long this requirement is already met.

FrameBuffer instances may not be copied or assigned.

class FrameBuffer {
public:
    FrameBuffer(void *bufferAddress);
    FrameBuffer* link(unsigned linkNum) const;
    const Descriptor& descriptor() const;
    const OpStatus& status() const;
    char* header() const;
    char* payload() const;
private:
    FrameBuffer* link[N];
    Descriptor descr;
    OpStatus status;
    // The frame header starts here.
};

Use case: System startup

For Gen I we need to store the configuration tables and factory code modules in configuration flash. We'll use the core's boot-level flash-reading code to read them during this startup process. That way the configuration flash's channel object factory and channel object will be constructed in the usual way, so there must be a corresponding factory code module in flash. Once startup is complete the boot-level flash code will no longer be used.

1. Boot code:
   1. Loads and starts the system core.
2. System core
   1. Initializes the CPU.
   2. Initializes RTEMS.
   3. Initializes any extra C++ support.
   4. Sets up the MMU's TLB and enables the MMU.
   5. Creates the default instance of the dynamic linker.
   6. Calls the ChannelList constructor.
   7. Loads and links the application code using the default dynamic linker.
   8. Initializes the network.
      1. Initializes each ethernet channel.
      2. Initializes IP, UDP, TCP and BSD sockets.
      3. Gets a DHCP lease if required.
   9. Calls the application entry point.

ChannelList constructor:

1. Reads each possible PPI defn. and for each non-empty one:
   1. Reads from flash and links the factory code module based on PPI type (if not already done).
   2. Calls the entry point in the factory module that creates channel factories (if not already done).
   3. Uses the resulting factory to create a channel object.
   4. Adds the resulting channel to the list.
   5. Saves the recommended number of buffers for the channel.
2. Decides how many buffers of what size to allocate for each channel.
3. For each Channel on the list:
   1. Gives the assigned buffers to the channel and enables it.

Use case: Frame import

TBD.

Use case: Frame export

TBD.

Virtex-5,6 (Gen II)

Location of the configuration tables

Probably somewhere in DCR space so we'll have to use mfdcr instructions to read them. The exact organization is TBD.

No PIC blocks

Protocol plugins will no longer be implemented using PIC blocks.

No header/payload division

The old division of frames into header and payload will no longer be made.

Port

Remembers the Channel that created it.

Allows client code to wait for new frames.

Reclaims frame buffers that the client code has finished using (or has just
created).

class Port {
public:
  Port(Channel *, unsigned portNumber);
  Channel* channel() const;
  unsigned portNumber() const;
  virtual void* wait(); // Returns a pointer to the frame header when one becomes available.
  void claimBuffer(void *frameHeader) const;
};

API implementation classes

These classes are not exposed to the client but are used internally by the API.

Class FrameBuffer

A FrameBuffer instance is placed at the beginning of a buffer that will also hold the frame proper. The instances are created at system startup and live until system shutdown.

A FrameBuffer contains regions of fixed size used in the management of the frame itself and its buffer:

• One or more links used to make singly-linked lists.
• The firmware's in-memory descriptor. The firmware TDE for the frame points to the descriptor.
• Status. The firmware writes operation completion status here.

Immediately following the end of the FrameBuffer buffer in we have first the frame's header and then its payload.

The links, descriptor and status areas have the same sizes for all channel types.

The descriptor must begin on a 64-byte boundary; for ease of layout the entire buffer also has that alignment. The buffer should also begin on a cache line boundary in order to reduce cache thrashing but since a cache line is only 32 bytes long this requirement is already met.

FrameBuffer instances may not be copied or assigned.