commonATCA package

The common ATCA driver is an API that intermediates ATCACommon EPICS module and CPSW. It abstracts common ATCA registers and provides get and set functions to control them using CPSW. It also makes transparent all the peculiarities of CPSW.

Introduction

The common ATCA package is responsible for the following:

• Connect to CPSW package and establish communication with the registers of the firmware common structure and are
  • DaqMux 0 and 1 configurations : Sends waveforms to software for visualization. Waveform source is configurable.
  • JESD Top (AMC) 0 and 1 : protocol communicating with ADC/DACs on the AMC cards
  • Waveform engine: An intermediate component between DaqMux (oscilloscope) and the software
  • bitstream build information
  • Temperature information
  • timing cross bar
• Provide API class, once instantiated using static function, the user is permitted to control the registers and create and read streams

Package file structure

The files in the package are as follows

• atcaCommon.h: Contains definition of a base class IATCACommonFw class. 
• atcaCommon.cc: Contains the implementation of new class CATCACommonFwAdapt that implements class IATCACommonFw. This class uses CPSW to establish communication with the DaqMuxes, waveform engines, ADC interfaces (JESD), and firmware information, and provides API functions to write/read to/from the registers.
• crossbarControlYaml.hh: Defines class CrossbarControlYaml. This class uses CPSW to establish communication with the crossbar in hardware. It provides API functions to read and write to crossbar configurations.
• crossbarControlYaml.cc: implements CrossbarControlYaml class

Operation

The common ATCA package uses the common platform software package (establishes connections with hardware based on protocols and addresses described in YAML files). The API assumes that the CPSW YAML files were already read and parsed successfully. The API requires a path the denote where all the registers will reside. 

The YAMLs necessary for the correct operation of the common ATCA API are shown as follows

• AppTop.yaml
• AxiSysMonUltraScale.yaml
• AppCore.yaml
• AmcCarrierBsi.yaml
• DaqMuxV2.yaml
• AppTopJesd.yaml
• AmcCarrierCore.yaml
• AxiVersion.yaml
• JesdRx.yaml
• AmcCarrierBsa.yaml
• BsaWaveformEngine.yaml
• AxiStreamDmaRingWrite.yaml
• 000TopLevel.yaml

Some of these YAMLs call one another, so the YAMLs that actually contain the registers are just some of them. The paths that are required for the registers are as follows

• AmcCarrierCore/AxiVersion

• AmcCarrierCore/AxiSysMonUltraScale

• AmcCarrierCore/AmcCarrierBsi

• AppTop/AppTopJesd(0/[0])

• AppTop/AppTopJesd(1/[1])

• AppTop/DaqMuxV2[0]

• AppTop/DaqMuxV2[1]

• AmcCarrierCore/AmcCarrierBsa/BsaWaveformEngine[0]/WaveformEngineBuffers

• AmcCarrierCore/AmcCarrierBsa/BsaWaveformEngine[1]/WaveformEngineBuffers

All the registers and streams that will be accessed are in the previously mentioned paths. A summary of the used registers are as follows:

atca Common API class (atcaCommon.cc/hh)

This class is responsible for DaqMux 0 and 1 configurations, JESD Top (AMC) 0 and 1, Waveform engine, Build information, and temperature information. 

API Instantiation (necessary for module developer)

The static function if the parent class ATCACommonFw allows the instantiation of API class. It seems to return an instantiation of the child class as follows:

ATCACommonFw IATCACommonFw::create(Path p)
{
    return IEntryAdapt::check_interface<ATCACommonFwAdapt, DevImpl>(p);
}

The code to instantiate the API is as follows

atcaCommon = IATCACommonFw::create(p_atcaCommon);

Stream instantiation (necessary for module developer)

The stream can be created using the createStream method in the API. Streams can be instantiated separately by calling the CPSW stream creation function directly as follows

    try {
        _stream[0] = IStream::create(p_root->findByName(stream0));
    } 
    catch (InvalidArgError &e) {
        // Don't print error if the stream name is empty, as the user didn't
        // want to create this channel anyway.
    }
    catch (CPSWError &e) {
        fprintf(stderr, "CPSW Error: %s, file: %s, line: %d\n", e.getInfo().c_str(), __FILE__, __LINE__);
    }

YAML string mapping

CrossbarControlYaml API class (crossbarControlYaml.ccc/hh)

This class is responsible for configuring the timing cross bar. The timing cross bar is nothing more than four multiplexers configuring four outputs. The four outputs and the four inputs are as follows:

• 0: NC TPG timing
• 1: Mini TPG
• 2: Back plane
• 3: SC TPG timing

The UML diagram is shown as follows.

The functions available are simply to instantiate a crossbar, and to configure and read current configuration. 

Exception handling

The package does not throw any exceptions. Nonetheless, CPSW throws CPSWError errors and the package propagates these exceptions. In these contexts the package prints to stderr. Upper layers should catch exceptions of type CPSWError.

ATCACommon EPICS module

The ATCACommon module integrates the common ATCA API with EPICS, and therefore allowing the control of the different components in EPICS.

Introduction

The ATCACommon module is responsible for the following:

• Provides EPICS asynDrivers and IOC shell commands to Instantiates the commonATCA package structures and use them to communication with the registers of the firmware common structure
  • Provides an ASYN driver with PVs to communicate to all ATCA Common registers, namely: DaqMux 0 and 1 registers, JESD Top (AMC) 0 and 1 registers, Waveform engine registers, firmware build information registers, temperature  registers
  • Provides an ASYN driver with PVs to manage streams and talk straight to CPSW skipping the API
  • Provide IOC shell commands, Asyn driver and PVs to configure timing cross bar

Module file structure

The files in the package are as follows

• ATCACommon.h: Contains the ASYN driver class definition
• ATCACommon.cc: Contains the method definitions of ATCACommon asyn driver that instantiates PVs to read status and control registers of DaqMux 0 and 1, JESD Top (AMC) 0 and 1, Waveform engine, firmware build information and ADC temperatures
• crossbarControl.hh: Defines two classes: CrossbarControlAsynDriver and CrossbarControlDriver
• crossbarControl.cpp: contains the methods of both classes and the ICO shell command to instantiate the asyn driver
• debugStream.h: contains the DebugStreamAsynDriver class definition and several APIs to allow upper layer software to register callbacks with stream data reception
• debugStream.cpp: contains the meothods of DebugStreamAsynDriver class, and the stream related API function instantiations
• debugStreamInterface.h : contains API function to register a stream callback
• versionComparator.cpp: exports an IOC command to compare the current FPGA firmware version/FPGA hash with a list of firmware versions/FPGA hashes (Not aware of anyone using this)

The database files associated are as follows

• atcaCommon.db: database file containing general information PVs
• daqMux.db: database file containing daqMux general control and status PVs
• daqMuxChn.db: database file containing daqMux channel specific control and status PVs
• jesdCount.db: database file containing Jesd PVs 
• waveformEngine.db: database file containing waveformEngine general control and status PVs
• waveformEngineChannel.db: database file containing waveformEngine channel specific control and status PVs
• ATCACommon.substitutions: Instantiation of upper mentioned db files once or several times with different macro values
• crossbarAlarn.db: database file containing crossbar alarm
• crossbarControl.db: database file containing crossbar input select PV
• crossbarCtrl.substitutions: instantiation of crossbarAlarm and crossbarControl database files once or several times
• streamTypeCh.db : database file containing channel specific configurations
• streamType.substitutions: database file instantiating streamTypeCh several times (4)
• streamWf.db: database file containing stream waveforms
• StreamDouble.substitutions: instantiate streamWf.db 4 times with double type
• StreamFloat.substitutions: instantiate streamWf.db 4 times with float type
• StreamLong.substitutions: instantiate streamWf.db 4 times with long type
• StreamShort.substitutions: instantiate streamWf.db 4 times with short type
• StreamUlong.substitutions: instantiate streamWf.db 4 times with ulong type
• StreamUshort.substitutions: instantiate streamWf.db 4 times with ushort type

CrossbarControlDriver and CrossbarControlAsynDriver classes (crossbarControl.h/cpp)

The CrossbarControlDriver instantiates the commonATCA API classCrossbarControlYaml. It also provides 2 methods: report and control . It is not clear why this class was not implemented in the commonATCA API or even if it is necessary at all. The CrossbarControlAsynDriver class on the other hand inherits from AsynDriver, and calls the CrossbarControlDriver with the proper parameters.

ATCACommonAsynDriver class (ATCACommon.h/cpp)

The ATCAcommonAsynDriver instantiates the lower level commonATCA API class called ATCACommonFw. The Asyn driver is then given access to all the relevant registers in the hardware and exports them as PVs in EPICS.

DebugStreamAsynDriver class (debugStream.h/cpp)

The DebugStreamAsynDriver is probably the most complex class of the ATCACommon module. A flow diagram showing the execution flow of this class is shown as follows

Registering stream callback

The functionality of debugStream could be extended to call one's own callback along with the default. The API function provided for registering the callback is in the debugStreamInterface.h header file and is as follows

/**
 * @brief Register a callback when data arrives on a specific channel of stream
 * 
 * @param portName : name of the asyn port generating the stream
 * @param stream_channel : Number of the channel to be parsed (1-4)
 * @param cb_func : callback function to register
 * @param cb_usr : private structure that will be passed to the callback
 * @return int : -1 if channel not found, else 0
 */
int registerStreamCallback(const char *portName, const int stream_channel, STREAM_CALLBACK_FUNCTION cb_func, void *cb_usr);

The callback function paramters have to be as follows

/**
 * @brief Callback function format
 * 
 * @param pBuf : buffer pointer
 * @param size  : size of buffer
 * @param time : Time of event
 * @param timeslot :  timeslot information from the timing pattern modifier 
 * @param usr : user private object pointer
 */
void callback(void* pBuf, unsigned size,  epicsTimeStamp time, int timeslot, void* usr)

versionComparator.cpp

This file contains an IOC shell command that compares the installed firmware version and/or hash to a list of firmwares and hashes. The hash and version are already exported as PVs, but this is an extra diagnostic tool for comparision. Also, it does not seem to function correctly with multiple sets of YAML files. Needs debugging if deemed necessary, otherwise should be removed.

Module exported IOC shell commands, description, and parameters

The IOC shell commands are as follows

PV name list

The DaqMux PVs are shown here.

Waveform engine PVs are shown here.

The remaining PVs are summarized as follows

Reading your own streams

In order to read your own (custom stream), following these instructions

1. Your driver class should inherit from DebugStreamAsynDriver class (and therefore will become an asyn driver)
2. call the DebugStreamAsynDriver in the initializer list or the constructor of the asyn driver
3. create your parameters in constructor
4. Overload the virtual asyn write and read functions if necessary
5. Overload the virtual streamPoll method of the parent class. This is where you will read and process the streams. 
6. Create IOC shell command that instantiates your driver class

This is a streamPoll method example

myStreamClass::myStreamClass(const char *portName, const char *named_root, const unsigned size, const bool header, const char *stream0, const char *stream1, const char *stream2, const char *stream3)
    : DebugStreamAsynDriver(portName,
                            named_root,
                            size,
                            header,
                            stream0,
                            stream1,
                            stream2,
                            stream3)
{
    createParam(p_myParamString,       asynParamFloat64, &p_myParam);
}


void myStreamClass::streamPoll(const int ch) {
    // First check if the user created the channel
    if(! _stream[ch]) {
        return;
    }

    try {
        rdLen[ch] = _stream[ch]->read(buff[ch], size, CTimeout(2000000));
    }
    catch (IOError &e) {
        // A timeout happened
        timeoutCnt ++;
        timeoutCnt_perStream[ch] ++;
    }
    catch (CPSWError &e) {
        // Don't print, as we are inside a polling. Wait for the next try, as
        // rdLen[ch] will be zero.
    }
    
    if(rdLen[ch] == 0 ) {
        return;
    }

    epicsTimeStamp time;
        
    rdCnt++; rdCnt_perStream[ch]++;

    /* extract data in stream */          
	stream_with_header_t *p = (stream_with_header_t *) buff[ch];
    time.nsec               = p->packet.time.secPastEpoch;
    time.secPastEpoch       = p->packet.time.nsec;   /* Set PVs from extracted data */    
	setTimeStamp(&time);

	/* Set your PVs with extracted data and update them */
    setDoubleParam(p_myParamString, p->myParam);
    callParamCallbacks();

}