Page History
Include Page | ||||
---|---|---|---|---|
|
Info |
---|
Draft, in work. |
Table of Contents |
---|
Include Page | ||||
---|---|---|---|---|
|
Major Xtc Library Classes
The most important data containers and associated classes in the Xtc library are listed below. (Inhereitance and collaboration trees are shown at right.)
Container Classes
- XtcData:Dgram, which is the root container, essentially an envelope, for other containers. By inheritance, Dgrams store information about LCLS machine state transitions and their associated pulseIDs and timestamps.
- XtcData:Xtc, the parent data container class for detector data and descriptive metadata. Each Dgram has one top-level Xtc container, which in turn contains other structures, including other Xtc/Xtc-derived containers.
Important Xtc Derived Classes and Utility Classes
- XtcData:Names, derived Xtc container for storing named metadata about data types and data dimensionality. Identified with an integer ID.
- XtcData:ShapesData, a derived convenience Xtc container with methods to access detector data.
- XtcData:Shapes, a derived Xtc container containing metadata about the structure of the raw detector data.
- XtcData:Data, a derived Xtc container for raw detector data itself.
- XtcData:NamesId, utility class that generates a composite ID, somewhat like a relational database composite key, combining the integer nodeID and Xtc:Names integer ID.
- XtcData:NamesVec, utility class for storing a set of XtcData:Names.
- XtcData:TypeId is used to identify which flavor of Xtc container (Parent, Names, ShapesData, Shapes, Data, etc.) is being worked with when recording or parsing.
Deck of Cards | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
|
The DAQ Side: Generating Xtc
For tutorial purposes, the following streamlined example shows XXXhow to use the Xtc library to develop a data writer and data reader for a fictitious but real-world HSD outputting feature extraction data. For an example involving more detectors and , formats and their corresponding algorithms, see the example xtcdata/xtcdata/app/xtcwriter.cc.
A Notional DAQ Harness
# Assume some totally made-up data acquisition callback framework named DAQFrameworkLet's motivate the example. Assume we're developing a an Xtc writer class to output HSD data. This class will get plugged in to a (notional) DAQ framework.
Code Block | ||||
---|---|---|---|---|
| ||||
CSPADXtcWriter padwriter// %sample:intro:cpp% // Assume some totally made-up data acquisition callback framework named DAQFramework // Developers of the psdaq package are welcome to come in and change this toy example // to match the real DAQ API. HSDXtcWriter hsdwriter() DAQFramework.registerHandler("configure", padwriter.setupstd::bind(hsdwriter.openXtcFile, filename)) DAQFramework.registerHandler("runstart", padwriterhsdwriter.setup) // Assume we get a "readout" message per event DAQFramework.registerHandler("readouteventreadout", padwriterhsdwriter.writeEventwriteFeature) DAQFramework.registerHandler("runend", padwriterhsdwriter.writeFilecloseXtcFile) |
//%endsample% |
"setup" Method: Define Data Signature and Create Names Structure
Deck of Cards | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||
|
Define Detector Metadata
Panel |
---|
At some initialization step, for example in response to a DAQ configure signal:
|
Define the "shape" of detector elements and associate algorithm version information with the data to facilitate pipeline processing.
|
"writeFeature" Method: Respond to Readout Events and Add Readout Data
Deck of Cards | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
|
The Pipeline Side: Parsing Whole Xtc Files, Using Small Data Files
xtcreader.cc and XtcIterator.hh
A Notional Offline Pipeline Harness
Notes for Real-World Code
Realistic Xtc Writers
(Bullet lists of gotchas and recommendations)
- Thread safety issue #1
- Thread safety issue #2
- (You'll likely need to sub-class from XYZ in order to ABC)
Realistic Xtc Readers
(Bullet lists of gotchas and recommendations)
- Thread safety issue #1
- Thread safety issue #2
- (You'll likely need to sub-class from XYZ in order to ABC)
Full "Hello Xtc" Code Listings
HSDXtcWriter Example
Code Block | ||||
---|---|---|---|---|
| ||||
// %sample:codesample1:cpp%
#include "xtcdata/xtc/ShapesData.hh"
#include "xtcdata/xtc/DescData.hh"
#include "xtcdata/xtc/Dgram.hh"
#include "xtcdata/xtc/TypeId.hh"
#include "xtcdata/xtc/XtcIterator.hh"
#include "xtcdata/xtc/VarDef.hh"
// various std library includes omitted...
using namespace XtcData;
#define BUFSIZE 0x4000000
HSDXtcWriter::HSDXtcWriter()
{
// Define the "signature" of the data in
// our feature extraction data set
class FexDef:public VarDef
{
public:
enum index
{
floatFex,
arrayFex,
intFex
};
FexDef()
{
NameVec.push_back({"floatFex",Name::DOUBLE});
// "arrayFex" has rank 2; 0 is the default, for scalar data
NameVec.push_back({"arrayFex",Name::FLOAT,2});
NameVec.push_back({"intFex",Name::INT64});
}
};
// Define some instance variables.
FILE *this->xtcoutfile;
// -------- Set up the metadata ------------
// Declare an Xtc::Dgram to store metadata about the output
Dgram& this->metadataDgram;
// Declare a NamesVec to store Names info
NamesVec this->namesVec;
// Declare a NamesId object to act as composite lookup key
NamesId this->namesId;
// Store the node ID of this node
unsigned this->nodeid = 1;
// -------- Set up the readout data ---------
// Declare an Xtc:Dgram to store readout data
Dgram& this->readoutDgram;
// Create one of our previously-defined FexDefs
this->FexDef = new FexDef();
// Now initialize various things
_initializeMetadataDgram();
}
void HSDXtcWriter::_initializeMetadataDgram()
{
TypeId tid(TypeId::Parent, 0);
void* configbuf = malloc(BUFSIZE);
this->metadataDgram = *(Dgram*)configbuf;
this->metadataDgram.xtc.contains = tid;
this->metadataDgram.xtc.damage = 0;
this->metadataDgram.xtc.extent = sizeof(Xtc);
}
void HSDXtcWriter::_initializeReadoutDataDgram()
{
TypeId tid(TypeId::Parent, 0);
void* buf = malloc(BUFSIZE);
this->readoutDgram = *(Dgram*)buf;
this->readoutDgram.xtc.contains = tid;
this->readoutDgram.xtc.damage = 0;
this->readoutDgram.xtc.extent = sizeof(Xtc);
}
void HSDXtcWriter::_addNames()
{
// Instantiate an Alg to define metadata for
// a feature extraction ("fex") algorithm,
// version 4.5.6
Alg hsdFexAlg("fex",4,5,6);
// Create a NamesId composite lookup key and
// then a Names structure (ID'd with that key)
// that identifies the location, algorithm,
// detector type and detectorID.
this->namesId.NamesId(this->nodeId,1); // nodeId=1, namesId = 1
Names& fexNames = *new(xtc) Names("xpphsd", hsdFexAlg, "hsd", "detnum1234", this->namesId)
// And now we add the Names structure to the
// parent Xtc metadata container along with
// the FexDef data signature.
fexNames.add(this->metadataDgram.xtc, this->FexDef);
// add this Names structure to the NamesVec
this->namesVec[namesId1] = NameIndex(this->fexNames);
}
void HSDXtcWriter::openXtcFile(xtcoutfilename)
{
this->xtcoutfile = fopen(xtcoutfilename, "w");
}
void HSDXtcWriter::setup()
{
// Behind this public method, call internal to
// set up the metadata
this->_addNames();
// Omitting code to confirm successful file operation....
fwrite(&this->metadataDgram, sizeof(this->metadataDgram) + this->metadataDgram.xtc.sizeofPayload(), 1, this->xtcoutfile)
}
void HSDXtcWriter::writeFeature()
{
// Initialize the Dgram for readout data
this->initializeReadoutDataDgram();
// Need some makebelieve code here to show how this
// callback method unpacks data from DAQ
// to cram into these Xtc structures...
// Talk to Chris. Is this event by event? Would you want
// to destruct this CreateData after each event?
CreateData fex(this->readoutDgram.xtc, this->namesVec, this->namesId);
// Need stub code here to represent getting values from DAQ
// and using set_value and allocate with something real-ish
// set_value() lets you set scalar values in the data record.
// The 'floatFex' ID is defined in our custom FexDef class.
fex.set_value(FexDef::floatFex, (double)41.0);
// Use allocate() to set up data that's better represented as
// vectors or matrices.
// The 'arrayFex' ID is defined in our custom FexDef class.
unsigned shape[MaxRank] = {2,3}; //MaxRank is an Xtc library
//global upper limit on data complexity.
Array<float> arrayT = fex.allocate<float>(this->FexDef::arrayFex, shape);
for(unsigned i=0; i<shape[0]; i++){
for (unsigned j=0; j<shape[1]; j++) {
arrayT(i,j) = 142.0+i*shape[1]+j;
}
};
// Another scalar value setter.
// The 'intFex' ID is defined in our custom FexDef class.
fex.set_value(FexDef::intFex, (int64_t) 42);
}
void HSDXtcWriter::closeXtcFile()
{
fclose(this->xtcoutfile);
}
// ...
// %endsample% |
HSDXtcReader Example
Code Block | ||||
---|---|---|---|---|
| ||||
// %sample:codesample2:cpp%
cout<<"Hi world?"
// %endsample% |
Parsing Whole Xtc Files
xtcreader.cc and XtcIterator.hh
Use of Xtc Small Data Files
this might be duplicate of what's on top page