Data file quality check
Datagram and XTC structure check
Stand-alone C++ application which uses the pdsdata
library.
Assumes the loop over all Drgams in a single XTC file with iterations over all XTC containers. Retrieved information is tested for consistency, as explained below.
Accumulate statistics for transitions
Print transition Id counters Total number of transition Id:13 Unknown : 0 Reset : 0 Map : 0 Unmap : 0 Configure : 1 Unconfigure : 0 BeginRun : 1 EndRun : 1 BeginCalibCycle : 1 EndCalibCycle : 1 Enable : 1 Disable : 1 L1Accept : 3520
Accumulate statistics for damages
Damage bit 1 : DroppedContribution Damage bit 12 : OutOfOrder Damage bit 13 : OutOfSynch Damage bit 14 : UserDefined Damage bit 15 : IncompleteContribution Damage bit 16 : ContainsIncomplete
Check correctness of the datagram time sequence
In perfect world it should be
t(Dg1) < t(Dg2) < t(Dg3) < ... < t(DgN)
search for datadrams with wrong time sequence, like
t(Dgi) > t(Dg[i+1])
Size consistency of XTC containers
- Check that the XTC container size does not exceed certain maximal value.
- The total size of internal XTC containers should be consistent (and not exceed) with size of overlapping XTC container.
Accumulate statistics for data types
Count occurrence of each data type in XTC containers.
Indexing?
The same procedure can be used for XTC data indexing..., but need to read all files for streams and chunks within run using the same algorithm like in psana
.
Data quality check
psana
-based analysis for selected types of data with known properties.- This is data-specific check, which can not be universal for all data types.
Command line interface
b. xtcqc -f <file-name>.xtc
Database for XTC Quality Check
Whenever the xtc file is processed, the results are saved in the database
/reg/d/psdm/<INSTRUMENT>/<experiment>/xtcqc/<xtc-like-file-name>.qc
Web-based control and access
Igor is going to produce web-based interface, like for translator, with functionality as follows.
- Show list of files with processed/not-processed flag,
- Control on job submission for quality check,
- Browser for QC results of particular file/run.
Overview
Content Tools