You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 23 Next »

This is a specification document to help the firmware engineer to integrate the DaqMux with the application and the amc-carrier-core.

Overview

The DaqMux was originally developed as an oscilloscope to transmit streams from the FPGA to software for debugging purposes. Then it started being used as a data streamer by the application to software, and it's configurations were exported to EPICS/software. 

Feature summary

The DaqMux can perform the following functionalities

  • Multiplex up to 16 streams (lanes) and send forward (to software) - default is 4
  • Stream packets can be acquired continuously every time a trigger arrives in continuous mode, or only once upon trigger with or without header (meta data) in trigger mode, or many times with or without header (meta data) in trigger mode when auto re-arm is enabled
  • Possibility of cascading trigger of DaqMux blocks in case streams from several DaqMuxes acquired at the same instant is required
  • Pause streaming by not overwriting buffers (freeze functionality), therefore nothing is forwarded to software
  • Down sampling and averaging

 

Definitions

  • Stream : a group of bytes transmitted in sequence.
  • Lane : a physical bus at which the stream crosses 
  • Packet : User abstraction. Bunches of bytes that the user wants forward to software from each lane every time there is a trigger
  • AXI transaction : Hardware abstraction. Packets are broken into a 1 or more AXI transactions
  •  

Applications

Figure 1 shows the DaqMux instantiated in the Common platform firmware in the application side. 

Figure 1: Common platform firmware top-level

Generics & IO description

Table 1: DaqMux generics list and description

Generic name default value Description
TPD_G 1 Simulation variable; register clock to output delay in nanoseconds
DECIMATOR_EN_G True Decimator enable
WAVEFORM_TDATA_BYTES_G 4 Output lane width in bytes
FRAME_BWIDTH_G 10 Power of two, which defines AXI stream transaction size in words (4 bytes)(i.e. when FRAME_BWIDTH_G = 12 → 210 x 4 = 4096 bytes)
BAY_INDEX_G - Index of the DaqMux
N_DATA_IN_G 16 Number of input data lanes (N_DATA_IN_G must be between 1 and 29)
N_DATA_OUT_G 4 Number of output data lanes (must be between 1 and 16)

 

Table 2: IO port list and description

Category Name Direction Clock domain Width Description

 

 

 

Clock & reset logic

 axiClk Input - 1 AXI Lite clock
axiRst Input axiClk 1 AXI Lite reset
devClk_i Input - 1 Development clock
devRst_i Input Synced to devClk_i in DaqMux (for some reason) 1 Development logic reset
wfClk_i Input - 1 Ouput lanes' clock
wfRst_i Input wfClk_i 1 Output lanes' reset

 

 

 

 

 

 

 

 

DaqMux control signals and timing information

 trigHw_i Input Synced to devClk_i in DaqMux 1

Trigger signal to start the DaqMux streaming

  • 0 indicates no trigger event
  • 1 indicates a trigger event
trigCasc_i Input Synced to devClk_i in DaqMux 1

Cascaded trigger input. Can be used along with trigHw_i when enabled in the register file

  • 0 indicates no trigger event
  • 1 indicates a trigger event
trigCasc_o Output devClk_i 1

Output trigger signal connected to the SW Trigger Enable control register

  • 0 indicates no trigger event
  • 1 indicates a trigger event
armCasc_i Input Synced to devClk_i in DaqMux 1

Cascaded trigger Arm. Arms the trigger and prepares DaqMux for trigger arrival

  • 0 indicates no arm
  • 1 indicates armed
armCasc_o Output devClk_i 1

Output cascade signal connected to the Arm HW Trigger control register

  • 0 indicates no arm
  • 1 indicates armed
freezeHw_i Input Synced to devClk_i in DaqMux 1

Adds invalid flag to the streams that are forwarded, and they will be discarded in one of the posterior blocks in the pipeline

  • 0 indicates valid (not freeze)
  • 1 indicates invalid (freeze)
timeStamp_i Input Synced to devClk_i in DaqMux 64 Time stamp coming from the AMC carrier core
bsa_i Input Synced to devClk_i in DaqMux 128 BSA information coming from the AMC carrier core
dmod_i Input Synced to devClk_i in DaqMux 192 Dmod timing information coming from the AMC carrier core

 

 

 

 

 

 

 

 

 

 

 

AXI Lite register memory mapped interface for reading and writing to register file

 axilReadMaster Input axiClk 1

AXI Lite record containing

Read Address channel

  • araddr  : slv(31 downto 0);
  • arprot  : slv(2 downto 0);
  • arvalid : sl;

Read data channel

  • rready  : sl;
axilReadSlave Output axiClk 1

AXI Lite record containing

 Read Address channel

  • arready : sl;

Read data channel

  • rdata   : slv(31 downto 0);
  • rresp   : slv(1 downto 0);
  • rvalid  : sl;
axilWriteMaster Input axiClk 1

AXI Lite record containing

Write address channel

  • awaddr  : slv(31 downto 0);
  • awprot  : slv(2 downto 0);
  • awvalid : sl;

Write data channel

  • wdata   : slv(31 downto 0);
  • wstrb   : slv(3 downto 0);
  • wvalid  : sl;

Write ack channel

  • bready  : sl;
axilWriteSlave Output axiClk 1

AXI Lite record containing

Write address channel

  • awready : sl;

Write data channel

  • wready  : sl;

Write ack channel

  • bresp   : slv(1 downto 0);
  • bvalid  : sl;

 

Input lane array - data valid based

 sampleDataArr_i Input devClk_i N_DATA_IN_G Input data stream array of 32-bit words
sampleValidVec_i Input devClk_i N_DATA_IN_G Input valids with each valid corresponding to the respective stream in that cycle
linkReadyVec_i Input devClk_i N_DATA_IN_G Indicate that the stream source is ready. If is 0, the DaqMux sets the Stream Error control register to 1

 

 

 

 

 


Output lane array

 rxAxisMasterArr_o Output wfClk_i N_DATA_OUT_G

AXI compliant output data stream array of 32-bit words. Nonetheless, the type of rxAxisMasterArr_o is defined as follows in the AxiStreamPkg.vhd file

  • tValid : sl;
  • tData  : slv(511 downto 0);
  • tStrb  : slv(63 downto 0);
  • tKeep  : slv(63 downto 0);
  • tLast  : sl;
  • tDest  : slv(7 downto 0);
  • tId    : slv(7 downto 0);
  • tUser  : slv(63 downto 0);
rxAxisSlaveArr_i Input wfClk_i N_DATA_OUT_G AXI streams data ready signals
rxAxisCtrlArr_i Input Synced to devClk_i in DaqMux N_DATA_OUT_G

AXI stream control signals as follows

  • pause    : sl;
  • overflow : sl;
  • idle     : sl;

If pause is 1, the DaqMux will not start and error is set. Represents status(DONE_C) signal in the AxiStreamDmaRingWrite module of the waveforEngine. Not clear what is the meaning of this flag. Found this comment in that module

--  status(DONE_C) indicates a push, but maybe more than one

 

 

Register address mappings and description

Table 3: DaqMux register address mapping and description

Register name

Address

 Access Bits Alias Description

Control

 0x0 RW

0

SW Trigger Enable

Triggers DAQ on all enabled channels. Must be set to 1, then set to 0 again.
1 Cascade Trigger enable

Enabling/disabling cascaded trigger

  • '0' - Disable Cascaded Trigger
  • '1' - Enable Cascaded Trigger
2 Auto Rearm HW Trigger

Enabling/disabling hardware automatic trigger. If disabled it has to be rearmed by Arm Hw Trigger

  • '0' - Disabled (has to be armed with bit3 otherwise disabled)
  • '1' - Enabled
3 Arm HW Trigger Arms the hardware trigger on rising edge. After trigger occurs the trigger has to be rearmed using this register.
4 Trigger Clear Status Trigger status will be cleared (On the rising edge).
5 Daq Mode

Select the data acquisition mode ( Stream stops if Error occurs )

  • '0' - Trigger mode - Normal DAQ mode
    • Has to be triggered to start every time
  • '1' - Continuous mode - The data is framed and continuously streamed out after enabled. (Still requires a trigger to start)
    • Disable the stream to stop
6 Packet Header Enable

Add 128-bit header (otherwise only data will be inserted)(Applies only to Triggered mode only)

  • '0' - Disabled
  • '1' - Enabled
7 SW Freeze Buffer  Freezes all enabled circular buffers
8 HW Freeze Buffer Enable

Enabling/disabling hardware freeze buffer request

  • '0' - Disabled
  • '1' - Enabled
Status 0x1 RO

0

 Software Trigger Status

Software Trigger Status (Registered on first trigger until cleared by Trigger Clear Status - Control[4] ).
1 Cascade Trigger Status Cascade Trigger Status (Registered on first trigger until cleared by Trigger Clear Status - Control[4] )
2 HW Trigger Status Hardware Trigger Status (Registered on first trigger until cleared by Trigger Clear Status - Control[4] )
3 HW Trigger Armed Status Hardware Trigger Armed Status (Registered on rising edge Arm HW Trigger - Control[3] - and cleared when hardware trigger occurs )
4 Combined Trigger Status Combined Trigger Status (Registered when trigger condition is met until cleared by Trigger Clear Status - Control[4] )
5 Freeze Buffers Status  Freeze buffer occurred (Registered on first freeze until cleared by Trigger Clear Status - Control[4] )
Decimation 0x2 RW 15:0 Decimation Rate Divisor

Sample rate divider (Decimator):

  • Averaging Enabled: (powers of two) 1,2,4,8,16,etc (max 2^12)
  • Averaging Disabled (32-bit): 1,2,3,4,etc (max 2^16-1).
  • Averaging Disabled (16-bit): 1,2,4,6,8,etc (max 2^16-1).

 
DataSize 0x3 RW 31:0 Data Buffer Size

Number of 32-bit words to forward at each trigger (Ignored in continuous mode) (if enabled header will be included in the first 14 words of data). Minimum size is 14 (the size of the header).
TimeStamp 0x4 RO 31:0 Timestamp[31:0] Timestamp [31:0] - secPastEpoch
0x5 RO 31:0 Timestamp[63:32] Timestamp [63:32] - nsec
BSA 0x6 RO 31:0 bsa(0)

edefAvgDn
0x7 31:0 bsa(1) edefMinor
0x8 31:0 bsa(2) edefMajor
0x9 31:0 bsa(3) edefInit
TrigCount 0xA RO 31:0 Trigger Count Counts valid data acquisition triggers
DbgInputValid 0xB RO 31:0 Debug Input Valid All DaqMux AXI input streams valid signals 
DbgLinkReady 0xC RO 31:0 Debug Link Ready All DaqMux AXI input streams ready signals
InputMuxSel 0x10 RW 4:0 Input Mux Select[0]

0x1x: Stream x: Channel select Multiplexer 

0 - Disabled, 1 - Test, 2 - Ch0, 3 - Ch1, 4 - Ch2 etc.(up to Ch29)

Test mode will output counter data
0x11-0x1E 4:0 Input Mux Select[1]-[14]
0x1F 4:0 Input Mux Select[15]
DaqStatus 0x20-0x23 (one register for each of the four lanes) RO 0 Stream Pause

(rxAxisCtrlArr_i port signal) Raw diagnostic stream control pause (Waveform engine bufferDone signal. When an AXI frame is completely  written to DRAM, this bit is set)
1 Stream Ready Debug flag: Raw diagnostic stream control Ready (Waveform engine FIFO output stream ready signal)
2 Stream Overflow (rxAxisCtrlArr_i port signal) Raw diagnostic stream control Overflow (set to 0 in waveform engine)
3 Stream Error Debug flag: Error during last Acquisition (Raw diagnostic stream control Ready or incoming data valid dropped)
4 Data source ready Debug signal: The data source is ready. If not, DaqMux sets error signal to 1
5 Stream Enable Debug signal: Output stream enabled
31:6 Frame Count Number of 4096 word frames sent
024-0x2F   Not used  
DataFormat


 0x30-0x3F (one register for each of the 16 (enabled) lanes)


 RW


 4:0 Sign Bit Position Indicating sign extension point (all bits after sign bit will be overwritten with sign)
5 Data Width

Data width 32-bit or 16-bit

  • '0' : 32-bits
  • '1' : 16-bits
6 Sign enable

Signed/unsigned

  • '0' : Unsigned
  • '1' : Signed
7 Decimation Averaging Enable

Decimation Averaging

  • '0' : Disable
  • '1' : Enable

Functional description & block diagrams

The DaqMux has two data acquisition (Daq) modes: Trigger mode and continuous mode. The names are deceiving because both modes need a trigger to operate. A more appropriate description may be automatic (continuous) and manual (trigger) modes. The DaqMux block diagram is shown in Figure 2.

Figure 2: DaqMux Block diagram

 

The block diagram is composed of the following blocks:

  • Input synchronization logic: Logic to synchronize input signals to devclk_i. Multi-bit signals are synchronized with double flopping as well as it is assumed that they will be stable 1/360s in worst case scenario
  • DaqTrigger: Receives all trigger sources and generates a single-cycle trigger and freeze signals to be used in the rest of the logic
  • DaqLane[X]: Instantiated N_DATA_OUT_G times. Each DaqLane receives all input streams performs multiplexing upon trigger, generates headers, decimation and averaging if enabled
  • Decimator & Avg: Performs the decimation (down-sampling) and averaging
  • FIFO[X]: Dual clock fifo to synchronize output lane array to wfClk_i
  • DaqRegltf: AXI Lite memory mapped register file that contains control and status registers of the DaqMux

A state machine diagram of the DaqMux operation is shown in Figure 3. For more detailed explanations, see the following sections.

 

Figure 3: DaqMux operation state machine

Continuous mode

As mentioned earlier, the continuous mode multiplexes raw data, and outputs either the raw data or the down sampling with or without averaging. It is activated with a trigger, and terminated upon error or lane disabling. After termination a new trigger is required. detailed state transition is described as follows:

Idle state exit conditions

The continuous mode starts in the presence of all of the following conditions

  • rxAxisCtrl input pause signal is set to 0 (Comes from the waveform engine of the amc-carrier-core) Not documented anywhere, but a speculation is when the buffers in the end of the pipeline are full, the DaqMux stops forwarding data. A much less confusing solution and implementation would be to use the back pressure feature of AXI streaming (ready goes to 0).
  • LinkReadyVec_i is 1. This indicates that the Jesd is ready
  • Control bit 5: this bit indicates that continuous mode is chosen

And one of the following conditions:

  • Software trigger indicated by control bit 0
  • Cascaded trigger indicated by both the setting of the cascaded arm and trigger signals
  • Hardware trigger indicated by the input trigger signal trigHw_i and the arm or auto arm control registers

Stream data state

In the stream data state one of the following is transmitted from the multiplexed lanes

  • Raw data
  • Down sampled data with or without averaging

Stream data termination conditions

In continuous mode, the stream state is exited in the following conditions

  • An error is established
  • The channel is disabled

Trigger mode

The trigger mode multiplexes raw data, and outputs a burst number of words upon trigger arrival. The bust may or may not contain a header (not available in continuous mode), and the burst data can be either the raw data or the down sampling with or without averaging. It is activated with a trigger, and terminated upon error . After termination a new trigger is required. detailed state transition is described as follows:

Idle state exit conditions

The continuous mode starts in the presence of all of the following conditions

  • rxAxisCtrl input pause signal is set to 0 (Comes from the waveform engine of the amc-carrier-core) Not documented anywhere, but a speculation is when the buffers in the end of the pipeline are full, the DaqMux stops forwarding data. A much less confusing solution and implementation would be to use the back pressure feature of AXI streaming (ready goes to 0).
  • LinkReadyVec_i is 1. This indicates that the Jesd is ready
  • Control bit 5: this bit indicates that trigger mode is chosen

And one of the following conditions:

  • Software trigger indicated by control bit 0
  • Cascaded trigger indicated by both the setting of the cascaded arm and trigger signals
  • Hardware trigger indicated by the input trigger signal trigHw_i and the arm or auto arm control registers

Stream data state

In the stream data state a burst of predefined number of the following is forwarded

  • A fixed number of words containing raw data words
  • A fixed number of words containing a header followed by raw data words
  • A fixed number of words containing header followed by down sampled data with or without averaging

Stream data termination conditions

The stream state is exited in the following conditions

  • An error is established
  • The channel is disabled
  • Request packet size is reached

The header data is 14 32-bit words as follows:

  • HeaderWord[0]: dmod(31:0) - Timing pattern information
  • HeaderWord[1]: dmod(63:32) - Timing pattern information 
  • HeaderWord[2]: dmod(95:64) - Timing pattern information 
  • HeaderWord[3]: dmod(127:96) - Timing pattern information 
  • HeaderWord[4]: dmod(159:128) - Timing pattern information 
  • HeaderWord[5]: dmod(191:160) - Timing pattern information 
  • HeaderWord[6]: timeStamp_i(31:0)   –  secPastEpoch
  • HeaderWord[7]: timeStamp_i(63:32)  –  nses 
  • HeaderWord[8]: bsa(127:96) - edefAvgDn
  • HeaderWord[9]: bsa(95:64) -edefMinor
  • HeaderWord[10]: bsa(63:32) -edefMajor
  • HeaderWord[11]: bsa(31:0) -edefInit
  • HeaderWord[12]: packetSize_i
  • HeaderWord[13]: Firmware specific signals
    • [BIT31:BIT27] = "00000"
    • [BIT26:BIT26] = Trigger Header: Hardware Trigger – s_trigHw
    • [BIT25:BIT25] = Trigger Header: Cascaded Trigger – s_trigCascRe
    • [BIT24:BIT24] = Trigger Header: Software Trigger – s_trigSw
    • [BIT23:BIT23] = Data format – dec16or32_i
    • [BIT22:BIT22] = Enable decimation averaging – averaging_i
    • [BIT21:BIT21] = Test mode – test_i
    • [BIT20:BIT20] = AMC BAY Index – BAY_INDEX_G
    • [BIT19:BIT16] = Channel Index – axiNum_i
    • [BIT15:BIT00] = decimation rate divide – rateDiv_i

Clocking/performance requirements

The maximum clock frequency has not been evaluated, nonetheless, this logic was used with clock frequencies as follows

  • devclk_i : 185 MHz
  • wfClk_i : (set to axiClk in the amc-carrier-core) 156.25 MHz
  • axiClk : 156.25 MHz

Connectivity

The following DaqMux ports need to be routed to the AMC carrier core, and the amc-carrier-core (namely waveform engine) does the rest of the magic moving this data to the DRAM and from the DRAM to the CPU memory.

  • wfClk_i
  • wfRst_i
  • rxAxisMasterArr_o
  • rxAxisSlaveArr_i
  • rxAxisCtrlArr_i

Signals timngStamp_i , bsa_i, and dmod_i come from the timingTrig record that is also provided by the amc-carrier core.

The rest of the signals are application specific.

 

 

  • No labels