Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Info

Draft, in work.

Table of Contents

Include Page
Xtc Topic Map
Xtc Topic Map

The DAQ Side: Generating Xtc

For tutorial purposes, the following streamlined example shows XXX. For an example involving more detectors, formats and algorithms, see xtcdata/xtcdata/app/xtcwriter.cc.

A Notional DAQ Harness

Let's motivate the example. Assume we're developing a an Xtc writer class to output HSD data. This class will get plugged in to a (notional) DAQ framework.

Code Block
languagecpp
linenumberstrue
# Assume some totally made-up data acquisition callback framework named DAQFramework
# Developers of the psdaq package are welcome to come in and change this toy example
# to match the real DAQ API.

HSDXtcWriter hsdwriter()

DAQFramework.registerHandler("configure", hsdwriter.setup)
DAQFramework.registerHandler("runstart", std::bind(hsdwriter.openXtcFile, filename))
// Assume writeFeature is called per event
DAQFramework.registerHandler("readout", hsdwriter.writeFeature)
DAQFramework.registerHandler("runend", hsdwriter.closeXtcFile)

"setup" Method: Set Up The Names Structure

Deck of Cards
idsetup
Card
defaulttrue
labelCoding Steps

Register the algorithms and "shape" of the data associated with detector elements so the processing pipeline knows what to do with it.

  1. First some plumbing...
    1. Create an XtcData:Dgram as the root container. %codesample1:1-11%
  2. Next code sample summary.

 

Card
idsetuptldr
labelTL;DR
Panel
  • Sub-class from XtcData::VarDef to define data shapesSubclass XtcData::VarDef to build a data structure specific to the detector in question.

      At some initialization step, for example in response to a DAQ configure signal:

        • VarDef gives you a NameVec with push_back() to xxxx

     

     

    "writeFeature" Method: Add Readout Data

    Deck of Cards
    idsetup
    Card
    defaulttrue
    labelCoding Steps

    (Introduce CreateData and its set_value (assign values explicitly) and allocate (set aside structured memory for the data then full in the structure with values) methods here. What about set_array_shape? With FexDef, we're positing that the data coming out has a well-defined shape, i.e. definable as a record, so per-event, you will update and emit this data record into the output stream)

    Another list of steps/big ideas this time about appending data.

    1. .
    2. A chunk of code summary.

     

    Card
    idsetuptldr
    labelTL;DR
    Panel
    • Use CreateData for XYZ applications:
      • set_value() for scalar data
      • allocate() for data vectors or matrices
    • Use DescribedData for XYZ applications

    The Pipeline Side: Parsing Whole Xtc Files, Using Small Data Files

    xtcreader.cc and XtcIterator.hh

    A Notional Offline Pipeline Harness


    Notes for Real-World Code

    Realistic Xtc Writers

    (Bullet lists of gotchas and recommendations)

    • Thread safety issue #1
    • Thread safety issue #2
    • (You'll likely need to sub-class from XYZ in order to ABC)

    Realistic Xtc Readers

    (Bullet lists of gotchas and recommendations)

    • Thread safety issue #1
    • Thread safety issue #2
    • (You'll likely need to sub-class from XYZ in order to ABC)

    Full "Hello Xtc" Code Listings

    HSDXtcWriter Example

    Code Block
    languagecpp
    linenumberstrue
    // @sample:codesample1
    
    #include "xtcdata/xtc/ShapesData.hh"
    #include "xtcdata/xtc/DescData.hh"
    #include "xtcdata/xtc/Dgram.hh"
    #include "xtcdata/xtc/TypeId.hh"
    #include "xtcdata/xtc/XtcIterator.hh"
    #include "xtcdata/xtc/VarDef.hh"
    
    using namespace XtcData;
    
    #define BUFSIZE 0x4000000
    
    HSDXtcWriter::HSDXtcWriter()
    {
    	// Define the shape/contours of each record in our feature extraction data set
    	class FexDef:public VarDef
    	{
    		public:
    		  enum index
    		  {
    			  floatFex,
    		      arrayFex,
    		      intFex
    	      };
    
    		FexDef()
       		{
           		NameVec.push_back({"floatFex",Name::DOUBLE});
    			NameVec.push_back({"arrayFex",Name::FLOAT,2});
    			NameVec.push_back({"intFex",Name::INT64});
    		}
    	};
    
    	// Define some instance variables.
    	FILE *this->xtcoutfile;
    	// An Xtc::Dgram to store metadata about the output
    	Dgram& this->configDgram;
    	std::vector<NameIndex>& this->namesVec;
    	this->FexDef = new FexDef();
    
    	// Now initialize various things
    	_initializeConfigDgram();
     }
    
    void HSDXtcWriter::_initializeConfigDgram()
    {
    	TypeId tid(TypeId::Parent, 0);
    	void* configbuf = malloc(BUFSIZE);
    
        this->configDgram = *(Dgram*)configbuf;
    	this->configDgram.xtc.contains = tid;
        this->configDgram.xtc.damage = 0;
    	// Check if Xtc::Xtc auto-alloc will take care of updating this extent as we go?
        this->configDgram.xtc.extent = sizeof(Xtc);
    }
    
    void HSDXtcWriter::_addNames(Xtc& parent, std::vector<NameIndex>& namesVec)
    {
    	// Use Alg to define metadata for a feature extraction ("fex") algorithm
    	Alg hsdFexAlg("fex",4,5,6);
    	// "Insert" a Names structure into the config Dgram's Xtc container.
    	Names& this->fexNames = *new(parent) Names("xpphsd", hsdFexAlg, "hsd","detnum1234")
    	// And now we populate the structure with the definition of the FexDex shapes
    	this->fexNames.add(parent, this->FexDef);
        namesVec.push_back(NameIndex(this->fexNames));
    }
    
    void HSDXtcWriter::setup()
    {
    	// Add a Names structure to the configuration Dgram
    	_addNames(this->configDgram.xtc, this->namesVec);
    }
    
    void HSDXtcWriter::openXtcFile(xtcoutfilename)
    {
    	this->xtcoutfile = fopen(xtcoutfilename, "w");
    }
    
    void HSDXtcWriter::writeFeature()
    {
    
    	// Need some makebelieve code here to show how this callback method unpacks data from DAQ
    	// to cram into these Xtc structures...
    
    	// Talk to Chris. Is this event by event? Would you want to destruct this CreateData after each event?
     	CreateData fex(parent, NamesVec, nameId);
    
    	// set_value() lets you set scalar values in the data record.
        // The 'floatFex' ID is defined in our custom FexDef class.
    	fex.set_value(FexDef::floatFex, (double)41.0);
    
    	// Use allocate() to set up data that's better represented as vectors or matrices.
        // The 'arrayFex' ID is defined in our custom FexDef class.
    	unsigned shape[MaxRank] = {2,3}; //MaxRank is an Xtc library global upper limit on data complexity.
        Array<float> arrayT = fex.allocate<float>(FexDef::arrayFex,shape);
        for(unsigned i=0; i<shape[0]; i++){
            for (unsigned j=0; j<shape[1]; j++) {
                arrayT(i,j) = 142.0+i*shape[1]+j;
            }
        };
    	
    	// Another scalar value setter.
        // The 'intFex' ID is defined in our custom FexDef class.
    	fex.set_value(FexDef::intFex, (int64_t) 42);
    
    }
    
    void HSDXtcWriter::closeXtcFile()
    {
    	fclose(this->xtcoutfile);
    }