Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


commonATCA package

The common ATCA driver is an API that intermediates ATCACommon EPICS module and CPSW. It abstracts common ATCA registers and provides get and set functions to control them using CPSW. It also makes transparent all the peculiarities of CPSW.

Introduction

The common ATCA package is responsible for the following:

  • Connect to CPSW package and establish communication with the registers of the firmware common structure and are
    • DaqMux 0 and 1 configurations : Sends waveforms to software for visualization. Waveform source is configurable.
    • JESD Top (AMC) 0 and 1 : protocol communicating with ADC/DACs on the AMC cards
    • Waveform engine: An intermediate component between DaqMux (oscilloscope) and the software
    • bitstream build information
    • Temperature information
    • timing cross bar
  • Provide API class, once instantiated using static function, the user is permitted to control the registers and create and read streams

Package file structure

The files in the package are as follows

  • atcaCommon.h: Contains definition of a base class IATCACommonFw class. 
  • atcaCommon.cc: Contains the implementation of new class CATCACommonFwAdapt that implements class IATCACommonFw. This class uses CPSW to establish communication with the DaqMuxes, waveform engines, ADC interfaces (JESD), and firmware information, and provides API functions to write/read to/from the registers.
  • crossbarControlYaml.hh: Defines class CrossbarControlYaml. This class uses CPSW to establish communication with the crossbar in hardware. It provides API functions to read and write to crossbar configurations.
  • crossbarControlYaml.cc: implements CrossbarControlYaml class

Operation

The common ATCA package uses the common platform software package (establishes connections with hardware based on protocols and addresses described in YAML files). The API assumes that the CPSW YAML files were already read and parsed successfully. The API requires a path the denote where all the registers will reside. 

...


YAML fileStream/Register nameDescription
1

AxiVersion.yaml

UpTimeCnt

Number of seconds since last reset

2

BuildStamp

Time stamp of the FPGA build

3

FpgaVersion

Version
FPGA firmware version

4

GitHash

Git hash of firmware project
5

AxiSysMonUltraScale.yaml

Temperature

ADC temperature values
6

AmcCarrierBsi.yaml

EthUpTime

Uptime of ethernet in seconds
7

JesdRx.yaml 2 instantiations






StatusValidCnt[0]

 Shows stability of JESD lanes. Counts number of JESD re-syncronisationssynchronizations

8

StatusValidCnt[1]

9StatusValidCnt[2]
10StatusValidCnt[3]
11StatusValidCnt[4]
12StatusValidCnt[5]
13

DaqMuxV2.yaml 2 instantiations













TriggerCascMask

covered in DaqMux documentation









14TriggerHwAutoRearm
15DaqMode
16PacketHeaderEn
17FreezeHwMask
18DecimationRateDiv
19DataBufferSize
20TrigCount
21DbgInputValid

22

DbgLinkReady

23

InputMuxSel[0/1/2/3]

24

StreamPause[0/1/2/3]

25

StreamReady[0/1/2/3]

26

StreamOverflow[0/1/2/3]

27

StreamError[0/1/2/3]

28

InputDataValid[0/1/2/3]

29

StreamEnabled[0/1/2/3]

30

FrameCnt[0/1/2/3]

31

FormatSignWidth[0/1/2/3]

32

FormatDataWidth[0/1/2/3]

33

FormatSign[0/1/2/3]

34

DecimationAveraging[0/1/2/3]

35

Timestamp[0/1]

36

TriggerDaq

37

ArmHwTrigger

38

FreezeBuffers

39

ClearTrigStatus

40AxiStreamDmaRingWrite.yaml 2 instantiations









Initialize

covered in waveform engine documentation









41

StartAddr[0/1/2/3]

42

EndAddr[0/1/2/3]

43

WrAddr[0/1/2/3]

44

Enabled[0/1/2/3]

45

Mode[0/1/2/3]

46

MsgDest[0/1/2/3]

47

FramesAfterTrigger[0/1/2/3]

48

Status[0/1/2/3]

49AmcCarrierCore.yaml

OutputConfig[0/1/2/3]

Crossbar configuration. Four outputs choosing from four inputs. Output and input enumeration is as follows:

  • 0: NC TPG timing
  • 1: Mini TPG
  • 2: Back plane
  • 3: SC TPG timing
50000TopLevel.yamlStreams (Stream0 - Stream7)Stream names to be initialized



atca Common API class (atcaCommon.cc/hh)

This class is responsible for DaqMux 0 and 1 configurations, JESD Top (AMC) 0 and 1, Waveform engine, Build information, and temperature information. 

...

Gliffy Diagram
macroId19d3b11c-e911-4022-af30-4fa9570d4525
displayNameatcaCommon driver class UML diagram
nameDriver class diagram
pagePin4


API Instantiation (necessary for module developer)

The static function if the parent class ATCACommonFw allows the instantiation of API class. It seems to return an instantiation of the child class as follows:

...

Code Block
languagecpp
themeRDark
titleInstantiation
linenumberstrue
atcaCommon = IATCACommonFw::create(p_atcaCommon);


Stream instantiation (necessary for module developer)

The stream can be created using the createStream method in the API. Streams can be instantiated separately by calling the CPSW stream creation function directly as follows

Code Block
languagecpp
themeRDark
titleStream instantiation
linenumberstrue
    try {
        _stream[0] = IStream::create(p_root->findByName(stream0));
    } 
    catch (InvalidArgError &e) {
        // Don't print error if the stream name is empty, as the user didn't
        // want to create this channel anyway.
    }
    catch (CPSWError &e) {
        fprintf(stderr, "CPSW Error: %s, file: %s, line: %d\n", e.getInfo().c_str(), __FILE__, __LINE__);
    }


YAML string mapping


CrossbarControlYaml API class (crossbarControlYaml.ccc/hh)

This class is responsible for configuring the timing cross bar. The timing cross bar is nothing more than four multiplexers configuring four outputs. The four outputs and the four inputs are as follows:

...

The functions available are simply to instantiate a crossbar, and to configure and read current configuration. 

Exception handling

The package does not throw any exceptions. Nonetheless, CPSW throws CPSWError errors and the package propagates these exceptions. In these contexts the package prints to stderr. Upper layers should catch exceptions of type CPSWError.


ATCACommon EPICS module

The ATCACommon module integrates the common ATCA API with EPICS, and therefore allowing the control of the different components in EPICS.

Introduction

The ATCACommon module is responsible for the following:

  • Provides EPICS asynDrivers and IOC shell commands to Instantiates the commonATCA package structures and use them to communication with the registers of the firmware common structure
    • Provides an ASYN driver with PVs to communicate to all ATCA Common registers, namely: DaqMux 0 and 1 registers, JESD Top (AMC) 0 and 1 registers, Waveform engine registers, firmware build information registers, temperature  registers
    • Provides an ASYN driver with PVs to manage streams and talk straight to CPSW skipping the API
    • Provide IOC shell commands, Asyn driver and PVs to configure timing cross bar

Module file structure

The files in the package are as follows

  • ATCACommon.h: Contains the ASYN driver class definition
  • ATCACommon.cc: Contains the method definitions of ATCACommon asyn driver that instantiates PVs to read status and control registers of DaqMux 0 and 1, JESD Top (AMC) 0 and 1, Waveform engine, firmware build information and ADC temperatures
  • crossbarControl.hh: Defines two classes: CrossbarControlAsynDriver and CrossbarControlDriver
  • crossbarControl.cpp: contains the methods of both classes and the ICO shell command to instantiate the asyn driver
  • debugStream.h: contains the DebugStreamAsynDriver class definition and several APIs to allow upper layer software to register callbacks with stream data reception
  • debugStream.cpp: contains the meothods of DebugStreamAsynDriver class, and the stream related API function instantiations
  • debugStreamInterface.h : contains API function to register a stream callback
  • versionComparator.cpp: exports an IOC command to compare the current FPGA firmware version/FPGA hash with a list of firmware versions/FPGA hashes (Not aware of anyone using this)

The database files associated are as follows

  • atcaCommon.db: database file containing general information PVs
  • daqMux.db: database file containing daqMux general control and status PVs
  • daqMuxChn.db: database file containing daqMux channel specific control and status PVs
  • jesdCount.db: database file containing Jesd PVs 
  • waveformEngine.db: database file containing waveformEngine general control and status PVs
  • waveformEngineChannel.db: database file containing waveformEngine channel specific control and status PVs
  • ATCACommon.substitutions: Instantiation of upper mentioned db files once or several times with different macro values
  • crossbarAlarn.db: database file containing crossbar alarm
  • crossbarControl.db: database file containing crossbar input select PV
  • crossbarCtrl.substitutions: instantiation of crossbarAlarm and crossbarControl database files once or several times
  • streamTypeCh.db : database file containing channel specific configurations
  • streamType.substitutions: database file instantiating streamTypeCh several times (4)
  • streamWf.db: database file containing stream waveforms
  • StreamDouble.substitutions: instantiate streamWf.db 4 times with double type
  • StreamFloat.substitutions: instantiate streamWf.db 4 times with float type
  • StreamLong.substitutions: instantiate streamWf.db 4 times with long type
  • StreamShort.substitutions: instantiate streamWf.db 4 times with short type
  • StreamUlong.substitutions: instantiate streamWf.db 4 times with ulong type
  • StreamUshort.substitutions: instantiate streamWf.db 4 times with ushort type

CrossbarControlDriver and CrossbarControlAsynDriver classes (crossbarControl.h/cpp)

Gliffy Diagram
macroId73ef1a1e-229c-4db7-82bd-ec80e42d93a1
nameCrossbarControlDriver and CrossbarControlAsynDriver
pagePin1

...

The CrossbarControlDriver instantiates the commonATCA API classCrossbarControlYaml. It also provides 2 methods: report and control . It is not clear why this class was not implemented in the commonATCA API or even if it is necessary at all. The CrossbarControlAsynDriver class on the other hand inherits from AsynDriver, and calls the CrossbarControlDriver with the proper parameters.

ATCACommonAsynDriver class (ATCACommon.h/cpp)

Gliffy Diagram
macroId5c6a9316-21c4-46d7-b730-94ccdac2a793
nameATCACommonAsynDriver class and linked list structure
pagePin1

The ATCAcommonAsynDriver instantiates the lower level commonATCA API class called ATCACommonFw. The Asyn driver is then given access to all the relevant registers in the hardware and exports them as PVs in EPICS.

DebugStreamAsynDriver class (debugStream.h/cpp)

Gliffy Diagram
macroIde2067dfe-2fa6-4e43-89b0-c4e0af7e6513
displayNamedebugStream
namedebugStream
pagePin5

...

Gliffy Diagram
macroIdea91832f-b5d6-4a84-b5f5-e0a16bbb8339
displayNamedebugStream execution flowchart
namedebugStream execution flowchart
pagePin2

Registering stream callback

The functionality of debugStream could be extended to call one's own callback along with the default. The API function provided for registering the callback is in the debugStreamInterface.h header file and is as follows

Code Block
languagecpp
themeRDark
titleRegister stream callback function
/**
 * @brief Register a callback when data arrives on a specific channel of stream
 * 
 * @param portName : name of the asyn port generating the stream
 * @param stream_channel : Number of the channel to be parsed (1-4)
 * @param cb_func : callback function to register
 * @param cb_usr : private structure that will be passed to the callback
 * @return int : -1 if channel not found, else 0
 */
int registerStreamCallback(const char *portName, const int stream_channel, STREAM_CALLBACK_FUNCTION cb_func, void *cb_usr);

Module exported IOC shell commands, description, and parameters

TBD

PV name list

TBD

Reading your own streams

...

The callback function paramters have to be as follows

Code Block
languagecpp
themeRDark
titleCallback parameters
/**
 * @brief Callback function format
 * 
 * @param pBuf : buffer pointer
 * @param size  : size of buffer
 * @param time : Time of event
 * @param timeslot :  timeslot information from the timing pattern modifier 
 * @param usr : user private object pointer
 */
void callback(void* pBuf, unsigned size,  epicsTimeStamp time, int timeslot, void* usr)

versionComparator.cpp

This file contains an IOC shell command that compares the installed firmware version and/or hash to a list of firmwares and hashes. The hash and version are already exported as PVs, but this is an extra diagnostic tool for comparision. Also, it does not seem to function correctly with multiple sets of YAML files. Needs debugging if deemed necessary, otherwise should be removed.

Module exported IOC shell commands, description, and parameters

The IOC shell commands are as follows

Commanddescriptionparameters

cpswDebugStreamDump

Dump stream contents on screen

  1. Stream port name

  2. Channel number

  3. Number of 64-bit words to dump

  4. Number of sequential packets

cpswDebugStreamAsynDriverConfigure

Initialize Asyn driver for one or more streams and their PVs
  1. Asyn port name
  2. Buffer size in bytes (including header if enabled)
  3. Header enabled
  4. Stream[0] name (optional)
  5. Stream[1] name (optional)
  6. Stream[2] name (optional)
  7. Stream[3] name (optional)

cpswATCACommonAsynDriverConfigure

Initialize Asyn driver for connecting ATCA common registers to PVs
  1. Asyn port name
  2. CPSW top path name (usually mmio)
  3. root name (optional)

cpswRelease

Returns the current CPSW versionNo parameters

crossbarControlAsynDriverConfigure

Initialize Asyn driver to configure crossbar using PVs
  1. Asyn port name
  2. path for AmcCarrierCore

  3. root name (optional)

crossbarControlDriverConfigure

Configures crossbar in case PCIe is used, otherwise, does nothing

PCIe fixed configurations are

  • OutputConfig[0]: LCLS1 SFP loopback
    OutputConfig[1]: LCLS2 SFP loopback
    OutputConfig[2]: FPGA (LCLS1) <--- LCLS1 SFP
    OutputConfig[3]: FPGA (LCLS2) <--- LCLS2 SFP

  1. path for AmcCarrierCore

  2. root name (optional)

crossbarControlDriverReport

reports how the crossbar inputs are currrently configuredNo parameters

crossbarControl

Configure cross bar straight from IOC Shell
  1. output: RTM_OUT0 | FPGA | BP | RTM_OUT1

  2. source: RTM_IN0 [LCLS1] | FPGA | BP | RTM_IN0 [LCLS2]

  3. named_root (optional)

atcaCheckFirmwareVersion

Compared current firmware version to input and exit IOC if requested (buggy)
  1. Y/N to stop IOC
  2. List of accepted firmware versions (strings)

PV name list

The DaqMux PVs are shown here.

Waveform engine PVs are shown here.

The remaining PVs are summarized as follows


Register indexPV nameDescription
7-12$(DEVICE):JESD[1:0]_[7:0]Counts number of JESD re-synchronizations for the corresponding JESD lanes
1$(DEVICE):AMC_UPTIMECNTNumber of seconds since last reset
2$(DEVICE):AMC_BUILDSTAMPTime stamp of the FPGA build
4$(DEVICE):AMC_GITHASHGit hash of firmware project
3$(DEVICE):AMC_FPGAVERSIONFPGA firmware version
5$(DEVICE):AMC_FPGATEMPADC temperature values
6$(DEVICE):AMC_ETHUPTIMECNTUptime of ethernet in seconds
Software abstraction$(DEVICE):JESDCNT_RESETReset references for JESD count in software
Software abstraction$(DEVICE):JESDCNT_MODEShow count from the last time reset was done or the absolute count since the firmware booted
Software abstraction$(DEVICE):CPSW_RELEASE_TAGCPSW release tag string
49$(DEVICE):TCRB0:OUTPUTCONFIG

configure crossbar input for RTM_IN0. Possible inputs are

  • 0: RTM_IN0 (LCLS1)
  • 1: FPGA
  • 2: BP
  • 3: RTM_IN1 (LCLS2)
49$(DEVICE):TCRB1:OUTPUTCONFIG

configure crossbar input for FPGA. Possible inputs are

  • 0: RTM_IN0 (LCLS1)
  • 1: FPGA
  • 2: BP
  • 3: RTM_IN1 (LCLS2)
49$(DEVICE):TCRB2:OUTPUTCONFIG

configure crossbar input for BP. Possible inputs are

  • 0: RTM_IN0 (LCLS1)
  • 1: FPGA
  • 2: BP
  • 3: RTM_IN1 (LCLS2)
49$(DEVICE):TCRB3:OUTPUTCONFIG

configure crossbar input for RTM_IN1. Possible inputs are

  • 0: RTM_IN0 (LCLS1)
  • 1: FPGA
  • 2: BP
  • 3: RTM_IN1 (LCLS2)

Reading your own streams

In order to read your own (custom stream), following these instructions

  1. Your driver class should inherit from DebugStreamAsynDriver class (and therefore will become an asyn driver)
  2. call the DebugStreamAsynDriver in the initializer list or the constructor of the asyn driver
  3. create your parameters in constructor
  4. Overload the virtual asyn write and read functions if necessary
  5. Overload the virtual streamPoll method of the parent class. This is where you will read and process the streams. 
  6. Create IOC shell command that instantiates your driver class

This is a streamPoll method example


Code Block
languagecpp
themeRDark
titlestreamPoll method
myStreamClass::myStreamClass(const char *portName, const char *named_root, const unsigned size, const bool header, const char *stream0, const char *stream1, const char *stream2, const char *stream3)
    : DebugStreamAsynDriver(portName,
                            named_root,
                            size,
                            header,
                            stream0,
                            stream1,
                            stream2,
                            stream3)
{
    createParam(p_myParamString,       asynParamFloat64, &p_myParam);
}


void myStreamClass::streamPoll(const int ch) {
    // First check if the user created the channel
    if(! _stream[ch]) {
        return;
    }

    try {
        rdLen[ch] = _stream[ch]->read(buff[ch], size, CTimeout(2000000));
    }
    catch (IOError &e) {
        // A timeout happened
        timeoutCnt ++;
        timeoutCnt_perStream[ch] ++;
    }
    catch (CPSWError &e) {
        // Don't print, as we are inside a polling. Wait for the next try, as
        // rdLen[ch] will be zero.
    }
    
    if(rdLen[ch] == 0 ) {
        return;
    }

    epicsTimeStamp time;
        
    rdCnt++; rdCnt_perStream[ch]++;

    /* extract data in stream */          
	stream_with_header_t *p = (stream_with_header_t *) buff[ch];
    time.nsec               = p->packet.time.secPastEpoch;
    time.secPastEpoch       = p->packet.time.nsec;   /* Set PVs from extracted data */    
	setTimeStamp(&time);

	/* Set your PVs with extracted data and update them */
    setDoubleParam(p_myParamString, p->myParam);
    callParamCallbacks();

}