Draft, in work.
The DAQ Side: Generating Xtc
For tutorial purposes, the following streamlined example shows XXX. For an example involving more detectors, formats and algorithms, see xtcdata/xtcdata/app/xtcwriter.cc.
A Notional DAQ Harness
Let's motivate the example. Assume we're developing a an Xtc writer class to output HSD data. This class will get plugged in to a (notional) DAQ framework.
// %sample:intro%
// Assume some totally made-up data acquisition callback framework named DAQFramework
// Developers of the psdaq package are welcome to come in and change this toy example
// to match the real DAQ API.
HSDXtcWriter hsdwriter()
DAQFramework.registerHandler("configure", hsdwriter.setup)
DAQFramework.registerHandler("runstart", std::bind(hsdwriter.openXtcFile, filename))
// Assume we get a "readout" message per event
DAQFramework.registerHandler("readout", hsdwriter.writeFeature)
DAQFramework.registerHandler("runend", hsdwriter.closeXtcFile)
//%endsample%
"setup" Method: Set Up The Names Structure
Register the algorithms and "shape" of the data associated with detector elements so the processing pipeline knows what to do with it.
- First some plumbing...
- Create an XtcData:Dgram as the root container %codesample1:1-11:setupexamples%.
- Next code sample summary, and a reference %codesample2:1:setupexamples%
- Next code sample summary, and a reference %codesample2:1-4:setupexamples%
- XtcData::Dgram is your root container for metadata (Names and Shapes) as well as the data itself
- Sub-class from XtcData::VarDef to define data shapes
- VarDef gives you a NameVec with push_back() method to xxxx
"writeFeature" Method: Add Readout Data
(Introduce CreateData and its set_value (assign values explicitly) and allocate (set aside structured memory for the data then full in the structure with values) methods here. What about set_array_shape? With FexDef, we're positing that the data coming out has a well-defined shape, i.e. definable as a record, so per-event, you will update and emit this data record into the output stream)
Another list of steps/big ideas this time about appending data.
- .
- A chunk of code summary.
- Use CreateData for XYZ applications:
- set_value() for scalar data
- allocate() for data vectors or matrices
- Use DescribedData for XYZ applications
The Pipeline Side: Parsing Whole Xtc Files, Using Small Data Files
xtcreader.cc and XtcIterator.hh
A Notional Offline Pipeline Harness
Notes for Real-World Code
Realistic Xtc Writers
(Bullet lists of gotchas and recommendations)
- Thread safety issue #1
- Thread safety issue #2
- (You'll likely need to sub-class from XYZ in order to ABC)
Realistic Xtc Readers
(Bullet lists of gotchas and recommendations)
- Thread safety issue #1
- Thread safety issue #2
- (You'll likely need to sub-class from XYZ in order to ABC)
Full "Hello Xtc" Code Listings
HSDXtcWriter Example
// %sample:codesample1%
#include "xtcdata/xtc/ShapesData.hh"
#include "xtcdata/xtc/DescData.hh"
#include "xtcdata/xtc/Dgram.hh"
#include "xtcdata/xtc/TypeId.hh"
#include "xtcdata/xtc/XtcIterator.hh"
#include "xtcdata/xtc/VarDef.hh"
using namespace XtcData;
#define BUFSIZE 0x4000000
HSDXtcWriter::HSDXtcWriter()
{
// Define the shape/contours of each record in our feature extraction data set
class FexDef:public VarDef
{
public:
enum index
{
floatFex,
arrayFex,
intFex
};
FexDef()
{
NameVec.push_back({"floatFex",Name::DOUBLE});
NameVec.push_back({"arrayFex",Name::FLOAT,2});
NameVec.push_back({"intFex",Name::INT64});
}
};
// Define some instance variables.
FILE *this->xtcoutfile;
// An Xtc::Dgram to store metadata about the output
Dgram& this->configDgram;
std::vector<NameIndex>& this->namesVec;
this->FexDef = new FexDef();
// Now initialize various things
_initializeConfigDgram();
}
void HSDXtcWriter::_initializeConfigDgram()
{
TypeId tid(TypeId::Parent, 0);
void* configbuf = malloc(BUFSIZE);
this->configDgram = *(Dgram*)configbuf;
this->configDgram.xtc.contains = tid;
this->configDgram.xtc.damage = 0;
// Check if Xtc::Xtc auto-alloc will take care of updating this extent as we go?
this->configDgram.xtc.extent = sizeof(Xtc);
}
void HSDXtcWriter::_addNames(Xtc& parent, std::vector<NameIndex>& namesVec)
{
// Instantiate and Alg to define metadata for a feature extraction ("fex") algorithm
Alg hsdFexAlg("fex",4,5,6);
// "Insert" a Names structure into the config Dgram's Xtc container.
Names& this->fexNames = *new(parent) Names("xpphsd", hsdFexAlg, "hsd","detnum1234")
// And now we populate the structure with the definition of the FexDex shapes
this->fexNames.add(parent, this->FexDef);
namesVec.push_back(NameIndex(this->fexNames));
}
void HSDXtcWriter::setup()
{
// Add a Names structure to the configuration Dgram
_addNames(this->configDgram.xtc, this->namesVec);
}
void HSDXtcWriter::openXtcFile(xtcoutfilename)
{
this->xtcoutfile = fopen(xtcoutfilename, "w");
}
void HSDXtcWriter::writeFeature()
{
// Need some makebelieve code here to show how this callback method unpacks data from DAQ
// to cram into these Xtc structures...
// Talk to Chris. Is this event by event? Would you want to destruct this CreateData after each event?
CreateData fex(parent, NamesVec, nameId);
// set_value() lets you set scalar values in the data record.
// The 'floatFex' ID is defined in our custom FexDef class.
fex.set_value(FexDef::floatFex, (double)41.0);
// Use allocate() to set up data that's better represented as vectors or matrices.
// The 'arrayFex' ID is defined in our custom FexDef class.
unsigned shape[MaxRank] = {2,3}; //MaxRank is an Xtc library global upper limit on data complexity.
Array<float> arrayT = fex.allocate<float>(FexDef::arrayFex,shape);
for(unsigned i=0; i<shape[0]; i++){
for (unsigned j=0; j<shape[1]; j++) {
arrayT(i,j) = 142.0+i*shape[1]+j;
}
};
// Another scalar value setter.
// The 'intFex' ID is defined in our custom FexDef class.
fex.set_value(FexDef::intFex, (int64_t) 42);
}
void HSDXtcWriter::closeXtcFile()
{
fclose(this->xtcoutfile);
}
// %endsample%
HSDXtcReader Example
// %sample:codesample2% cout<<"Hi world?" // %endsample%
Overview
Content Tools