You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

Data file structure quality check

Datagram and XTC structure check

Stand-alone C++ application which uses the pdsdata library.
Assumes the loop over all Drgams in a single XTC file/stream with iterations over all XTC containers. Retrieved information is tested for consistency, as explained below.

Datagram header records in log-file:

Datagram:1  Transition:Configure/ 4  seqtype:0  2012-02-06 15:11:23.529  fiducial:01ffff  ticks:000000  pos 0x0 
Datagram:2  Transition:BeginRun/ 6  seqtype:0  2012-02-06 19:47:19.529  fiducial:01ffff  ticks:000000  pos 0x9b004 
Datagram:3  Transition:BeginCalibCycle/ 8  seqtype:0  2012-02-06 19:47:19.653  fiducial:01ffff  ticks:000000  pos 0x9b0cc 
Datagram:4  Transition:Enable/10  seqtype:0  2012-02-06 19:47:19.654  fiducial:01ffff  ticks:000000  pos 0x9b294 
Datagram:10  Transition:L1Accept/12  seqtype:0  2012-02-06 19:47:19.952  fiducial:00f6fe  ticks:05092c  pos 0x2f5d864 
Datagram:20  Transition:L1Accept/12  seqtype:0  2012-02-06 19:47:20.452  fiducial:00f7b2  ticks:050dd2  pos 0x8cdbe58 
Datagram:30  Transition:L1Accept/12  seqtype:0  2012-02-06 19:47:20.951  fiducial:00f866  ticks:050ab4  pos 0xea5d6a8 
Datagram:40  Transition:L1Accept/12  seqtype:0  2012-02-06 19:47:21.451  fiducial:00f91a  ticks:050cac  pos 0x147dbc9c 
Datagram:72  Depth:0  Type:Xtc  Damage bit 16: ContainsIncomplete
Datagram:72  Depth:1  Type:Xtc  Damage bit 16: ContainsIncomplete
Datagram:72  Depth:2  Type:CspadElement  Damage bit 15: IncompleteContribution
Datagram:100  Transition:L1Accept/12  seqtype:0  2012-02-06 19:47:24.450  fiducial:00fd52  ticks:050d7e  pos 0x378db6cc 
Datagram:123  Depth:0  Type:Xtc  Damage bit 16: ContainsIncomplete
Datagram:123  Depth:1  Type:Xtc  Damage bit 16: ContainsIncomplete
Datagram:123  Depth:2  Type:CspadElement  Damage bit 15: IncompleteContribution
Datagram:200  Transition:L1Accept/12  seqtype:0  2012-02-06 19:47:29.448  fiducial:01045a  ticks:050f30  pos 0x71fdace8 
Datagram:272  Depth:0  Type:Xtc  Damage bit  1: DroppedContribution
Datagram:272  Depth:1  Type:Xtc  Damage bit  1: DroppedContribution
Datagram:272  Depth:2  Type:Any  Damage bit  1: DroppedContribution
Datagram:284  Depth:0  Type:Xtc  Damage bit 14: UserDefined

XTC header records in log-file (if needed):

Datagram:100  Transition:L1Accept/12  seqtype:0  2012-02-06 19:47:24.450  fiducial:00fd52  ticks:050d7e  pos 0x378db6cc 
Depth:0  XTC header: dmg:0  payload:9803432  extent:9803452  Src:Level:3  Event:ProcInfo(172.21.26.100, pid=18143)  TypeId:ver:0 id:1:Xtc
Depth:1  XTC header: dmg:0  payload:500  extent:520  Src:Level:2  Segment:ProcInfo(172.21.26.39, pid=6099)  TypeId:ver:0 id:1:Xtc
Depth:2  XTC header: dmg:0  payload:32  extent:52  Src:Level:1  Source:DetInfo(CxiDg2.0:Ipimb.1)  TypeId:ver:2 id:22:IpimbData
Depth:2  XTC header: dmg:0  payload:28  extent:48  Src:Level:1  Source:DetInfo(CxiDg2.0:Ipimb.1)  TypeId:ver:1 id:31:IpmFex
Depth:2  XTC header: dmg:0  payload:32  extent:52  Src:Level:1  Source:DetInfo(CxiDg4.0:Ipimb.0)  TypeId:ver:2 id:22:IpimbData
Depth:2  XTC header: dmg:0  payload:28  extent:48  Src:Level:1  Source:DetInfo(CxiDg4.0:Ipimb.0)  TypeId:ver:1 id:31:IpmFex
Depth:2  XTC header: dmg:0  payload:32  extent:52  Src:Level:1  Source:DetInfo(CxiSc1.0:Ipimb.0)  TypeId:ver:2 id:22:IpimbData
Depth:2  XTC header: dmg:0  payload:28  extent:48  Src:Level:1  Source:DetInfo(CxiSc1.0:Ipimb.0)  TypeId:ver:1 id:31:IpmFex
Depth:2  XTC header: dmg:0  payload:32  extent:52  Src:Level:1  Source:DetInfo(CxiDg1.0:Ipimb.0)  TypeId:ver:2 id:22:IpimbData
Depth:2  XTC header: dmg:0  payload:28  extent:48  Src:Level:1  Source:DetInfo(CxiDg1.0:Ipimb.0)  TypeId:ver:1 id:31:IpmFex
Depth:2  XTC header: dmg:0  payload:32  extent:52  Src:Level:1  Source:DetInfo(CxiDg2.0:Ipimb.0)  TypeId:ver:2 id:22:IpimbData
Depth:2  XTC header: dmg:0  payload:28  extent:48  Src:Level:1  Source:DetInfo(CxiDg2.0:Ipimb.0)  TypeId:ver:1 id:31:IpmFex
Depth:1  XTC header: dmg:0  payload:0  extent:20  Src:Level:2  Segment:ProcInfo(172.21.26.26, pid=20346)  TypeId:ver:0 id:1:Xtc
Depth:1  XTC header: dmg:0  payload:132  extent:152  Src:Level:2  Segment:ProcInfo(172.21.26.102, pid=16207)  TypeId:ver:0 id:1:Xtc
Depth:2  XTC header: dmg:0  payload:32  extent:52  Src:Level:6  Reporter:BldInfo(FEEGasDetEnergy)  TypeId:ver:0 id:14:FEEGasDetEnergy
Depth:2  XTC header: dmg:0  payload:60  extent:80  Src:Level:6  Reporter:BldInfo(EBeam)  TypeId:ver:1 id:15:EBeamBld
Depth:1  XTC header: dmg:0  payload:614436  extent:614456  Src:Level:2  Segment:ProcInfo(172.21.26.26, pid=20340)  TypeId:ver:0 id:1:Xtc
Depth:2  XTC header: dmg:0  payload:614416  extent:614436  Src:Level:1  Source:DetInfo(CxiSc1.0:Tm6740.0)  TypeId:ver:1 id:2:Frame
Depth:1  XTC header: dmg:0  payload:36  extent:56  Src:Level:2  Segment:ProcInfo(172.21.26.25, pid=17284)  TypeId:ver:0 id:1:Xtc
Depth:2  XTC header: dmg:0  payload:16  extent:36  Src:Level:1  Source:DetInfo(NoDetector.0:Evr.0)  TypeId:ver:3 id:19:EvrData
Depth:1  XTC header: dmg:0  payload:0  extent:20  Src:Level:2  Segment:ProcInfo(172.21.26.27, pid=13359)  TypeId:ver:0 id:1:Xtc
Depth:1  XTC header: dmg:0  payload:4594084  extent:4594104  Src:Level:2  Segment:ProcInfo(172.21.26.139, pid=5224)  TypeId:ver:0 id:1:Xtc
Depth:2  XTC header: dmg:0  payload:4594064  extent:4594084  Src:Level:1  Source:DetInfo(CxiDs1.0:Cspad.0)  TypeId:ver:2 id:28:CspadElement
Depth:1  XTC header: dmg:0  payload:4594084  extent:4594104  Src:Level:2  Segment:ProcInfo(172.21.26.95, pid=3697)  TypeId:ver:0 id:1:Xtc
Depth:2  XTC header: dmg:0  payload:4594064  extent:4594084  Src:Level:1  Source:DetInfo(CxiDsd.0:Cspad.0)  TypeId:ver:2 id:28:CspadElement

Accumulate statistics for transitions

QCStatistics::printTransIdCounter()
Total number of known TransitionId : 13
Unknown         : 0         Reset           : 0         
Map             : 0         Unmap           : 0         
Configure       : 1         Unconfigure     : 0         
BeginRun        : 1         EndRun          : 1         
BeginCalibCycle : 1         EndCalibCycle   : 1         
Enable          : 1         Disable         : 1         
L1Accept        : 1589

In most cases the number of "Begin"-like transitions should be the same as the number of "End"-like transitions.

Check correctness of the datagram time sequence

In perfect world it should be 

t(Dg1) < t(Dg2) < t(Dg3) < ... < t(DgN)

search for datadrams with wrong time sequence, like

t(Dgi) > t(Dg[i+1])

Content of each record:

struct BrokenTimeRecord {
  unsigned       ndg;
  uint32_t       damage_value;
  Pds::ClockTime time_dg;
  Pds::ClockTime time_dg_prev;
};

Print summary for broken time records:

QCStatistics::printBrokenTimeRecords()
Total number of broken time records : 100
BrokenTimeRecord: Dgram:     1  t:1969-12-31 16:00:00.997  t-previous:16:00:00.000  damage:0
BrokenTimeRecord: Dgram:     2  t:2012-02-05 10:27:28.948  t-previous:10:27:28.997  damage:0
BrokenTimeRecord: Dgram:     3  t:2012-02-05 20:12:35.074  t-previous:20:12:35.948  damage:0
...

Size consistency of XTC containers

  • Check that the XTC container size does not exceed certain maximal value.
  • The total size of internal XTC containers should be consistent (and not exceed) with size of overlapping XTC container.
  • Accumulate XTC container size error records; content of each record: 
    struct SizeErrorRecord {
  unsigned ndg;
  unsigned depth;
  int      remaining;
  uint32_t damage_ext;
  uint32_t damage_int;
  Pds::TypeId::Type typeid_ext;
  Pds::TypeId::Type typeid_int;
};

Print in summary the XTC size error records:

QCStatistics::printXTCSizeErrorRecords()
Total number of size error records : 91
Size Error Record: Dgram:     1  depth:1  remaining:0  damage ext:0  damage int:0  name ext:Xtc  name int:Xtc
Size Error Record: Dgram:     1  depth:1  remaining:0  damage ext:0  damage int:0  name ext:Xtc  name int:Xtc
Size Error Record: Dgram:     1  depth:1  remaining:0  damage ext:0  damage int:0  name ext:Xtc  name int:Xtc

Accumulate statistics for data types

Count occurrence of each data type in XTC containers.

QCStatistics::printTypeIdCounter()
Total number of known TypeId : 58
Ind: TypeId name             : Entries Vers       payload                 Depth     Src:Level    Src:Names
                             :         min/max    min/max     : #zeros    min/max   min/max 
  0: Any                     : 2         0/0        0/0       : 2           2/2       6/6        BldInfo(FEEGasDetEnergy)
  1: Xtc                     : 14365     0/1        0/9816324 : 2619        0/2       1/3        ProcInfo(172.21.26.100, pid=18143)
                                                                                                 ProcInfo(172.21.26.25, pid=17284)
                                                                                                 ProcInfo(172.21.26.39, pid=6099)
                                                                                                 ProcInfo(172.21.26.26, pid=20340)
                                                                                                 ProcInfo(172.21.26.95, pid=3697)
                                                                                                 ProcInfo(172.21.26.26, pid=20346)
                                                                                                 ProcInfo(172.21.26.102, pid=16207)
                                                                                                 DetInfo(BldEb.0:NoDevice.0)
                                                                                                 ProcInfo(172.21.26.139, pid=5224)
                                                                                                 ProcInfo(172.21.26.27, pid=13359)
  2: Frame                   : 1589      1/1   614416/614416  : 0           2/2       1/1        DetInfo(CxiSc1.0:Tm6740.0)
  3: AcqWaveform             : 0     
  4: AcqConfig               : 0     
  5: TwoDGaussian            : 0     
  6: Opal1kConfig            : 0     
  7: FrameFexConfig          : 1         1/1       28/28      : 0           2/2       1/1        DetInfo(CxiSc1.0:Tm6740.0)
  8: EvrConfig               : 2         5/5      192/192     : 0           2/2       1/1        DetInfo(NoDetector.0:Evr.0)
  9: TM6740Config            : 1         2/2       12/12      : 0           2/2       1/1        DetInfo(CxiSc1.0:Tm6740.0)
 10: RunControlConfig        : 2         1/1       24/24      : 0           2/2       0/0        ProcInfo(0.0.0.0, pid=24604)
 11: pnCCDframe              : 0     
 12: pnCCDconfig             : 0     
 13: Epics                   : 20997     1/1       24/496     : 0           2/2       1/1        DetInfo(EpicsArch.0:NoDevice.0)
 14: FEEGasDetEnergy         : 1588      0/0       32/32      : 0           2/3       6/6        BldInfo(FEEGasDetEnergy)
 15: EBeamBld                : 1590      1/1       60/60      : 0           2/3       6/6        BldInfo(EBeam)
 16: PhaseCavity             : 0     
 17: PrincetonFrame          : 0     
 18: PrincetonConfig         : 0     
 19: EvrData                 : 1589      3/3       16/16      : 0           2/2       1/1        DetInfo(NoDetector.0:Evr.0)
 20: FrameFccdConfig         : 0     
 21: FccdConfig              : 0     
 22: IpimbData               : 7945      2/2       32/32      : 0           2/2       1/1        DetInfo(CxiDg2.0:Ipimb.1)
                                                                                                 DetInfo(CxiDg4.0:Ipimb.0)
                                                                                                 DetInfo(CxiSc1.0:Ipimb.0)
                                                                                                 DetInfo(CxiDg1.0:Ipimb.0)
                                                                                                 DetInfo(CxiDg2.0:Ipimb.0)
 23: IpimbConfig             : 5         2/2       60/60      : 0           2/2       1/1        DetInfo(CxiDg1.0:Ipimb.0)
                                                                                                 DetInfo(CxiDg2.0:Ipimb.0)
                                                                                                 DetInfo(CxiDg2.0:Ipimb.1)
                                                                                                 DetInfo(CxiDg4.0:Ipimb.0)
                                                                                                 DetInfo(CxiSc1.0:Ipimb.0)
 24: EncoderData             : 0     
 25: EncoderConfig           : 0     
 26: EvrIOConfig             : 0     
 27: PrincetonInfo           : 0     
 28: CspadElement            : 3178      1/2  4594064/4594064 : 0           2/2       1/1        DetInfo(CxiDs1.0:Cspad.0)
                                                                                                 DetInfo(CxiDsd.0:Cspad.0)
 29: CspadConfig             : 2         3/3   287844/287844  : 0           2/2       1/1        DetInfo(CxiDsd.0:Cspad.0)
                                                                                                 DetInfo(CxiDs1.0:Cspad.0)
 30: IpmFexConfig            : 5         2/2      520/520     : 0           2/2       1/1        DetInfo(CxiDg1.0:Ipimb.0)
                                                                                                 DetInfo(CxiDg2.0:Ipimb.0)
                                                                                                 DetInfo(CxiDg2.0:Ipimb.1)
                                                                                                 DetInfo(CxiDg4.0:Ipimb.0)
                                                                                                 DetInfo(CxiSc1.0:Ipimb.0)
 31: IpmFex                  : 7945      1/1       28/28      : 0           2/2       1/1        DetInfo(CxiDg2.0:Ipimb.1)
                                                                                                 DetInfo(CxiDg4.0:Ipimb.0)
                                                                                                 DetInfo(CxiSc1.0:Ipimb.0)
                                                                                                 DetInfo(CxiDg1.0:Ipimb.0)
                                                                                                 DetInfo(CxiDg2.0:Ipimb.0)
 32: DiodeFexConfig          : 0     
 33: DiodeFex                : 0     
 34: PimImageConfig          : 1         1/1        8/8       : 0           2/2       1/1        DetInfo(CxiSc1.0:Tm6740.0)
 35: SharedIpimb             : 0     
 36: AcqTDCConfig            : 0     
 37: AcqTDCData              : 0     
 38: Index                   : 0     
 39: XampsConfig             : 0     
 40: XampsElement            : 0     
 41: Cspad2x2Element         : 0     
 42: SharedPIM               : 0     
 43: Cspad2x2Config          : 0     
 44: FexampConfig            : 0     
 45: FexampElement           : 0     
 46: Gsc16aiConfig           : 0     
 47: Gsc16aiData             : 0     
 48: PhasicsConfig           : 0     
 49: TimepixConfig           : 0     
 50: TimepixData             : 0     
 51: CspadCompressedElement  : 0     
 52: OceanOpticsConfig       : 0     
 53: OceanOpticsData         : 0     
 54: EpicsConfig             : 0     
 55: FliConfig               : 0     
 56: FliFrame                : 0     
 57: QuartzConfig            : 0

Accumulate statistics of damages

  • Acumulate counters for
    • damage bits
    • damage data type
    • damage bits vs data type
  • Accumulate damage records; content of each record: 
    struct DamageRecord {
  unsigned ndg;
  unsigned depth;
  uint32_t damage_value;
  Pds::TypeId::Type type_id;
};

Summary table for damage bit counters

QCStatistics::printDamageBitCounter()   Damage counter
Damage bit:1  : DroppedContribution            6
Damage bit:12 : OutOfOrder                     0
Damage bit:13 : OutOfSynch                     0
Damage bit:14 : UserDefined                    4
Damage bit:15 : IncompleteContribution         9
Damage bit:16 : ContainsIncomplete            18

Summary table for damage bit vs data types counters

QCStatistics::printDamageCounter()
Total number of known TypeId : 58
Ind: TypeId name             : Entries  DmgTot  Bits:1       12      13      14      15      16      
  0: Any                     : 2             2       2       0       0       0       0       0
  1: Xtc                     : 14365        26       4       0       0       4       0      18
  2: Frame                   : 1589          0       0       0       0       0       0       0
  3: AcqWaveform             : 0      
  4: AcqConfig               : 0      
  5: TwoDGaussian            : 0      
  6: Opal1kConfig            : 0      
  7: FrameFexConfig          : 1             0       0       0       0       0       0       0
  8: EvrConfig               : 2             0       0       0       0       0       0       0
  9: TM6740Config            : 1             0       0       0       0       0       0       0
 10: RunControlConfig        : 2             0       0       0       0       0       0       0
 11: pnCCDframe              : 0      
 12: pnCCDconfig             : 0      
 13: Epics                   : 20997         0       0       0       0       0       0       0
 14: FEEGasDetEnergy         : 1588          0       0       0       0       0       0       0
 15: EBeamBld                : 1590          0       0       0       0       0       0       0
 16: PhaseCavity             : 0      
 17: PrincetonFrame          : 0      
 18: PrincetonConfig         : 0      
 19: EvrData                 : 1589          0       0       0       0       0       0       0
 20: FrameFccdConfig         : 0      
 21: FccdConfig              : 0      
 22: IpimbData               : 7945          0       0       0       0       0       0       0
 23: IpimbConfig             : 5             0       0       0       0       0       0       0
 24: EncoderData             : 0      
 25: EncoderConfig           : 0      
 26: EvrIOConfig             : 0      
 27: PrincetonInfo           : 0      
 28: CspadElement            : 3178          9       0       0       0       0       9       0
 29: CspadConfig             : 2             0       0       0       0       0       0       0
 30: IpmFexConfig            : 5             0       0       0       0       0       0       0
 31: IpmFex                  : 7945          0       0       0       0       0       0       0
 32: DiodeFexConfig          : 0      
 33: DiodeFex                : 0      
 34: PimImageConfig          : 1             0       0       0       0       0       0       0
 35: SharedIpimb             : 0      
 36: AcqTDCConfig            : 0      
 37: AcqTDCData              : 0      
 38: Index                   : 0      
 39: XampsConfig             : 0      
 40: XampsElement            : 0      
 41: Cspad2x2Element         : 0      
 42: SharedPIM               : 0      
 43: Cspad2x2Config          : 0      
 44: FexampConfig            : 0      
 45: FexampElement           : 0      
 46: Gsc16aiConfig           : 0      
 47: Gsc16aiData             : 0      
 48: PhasicsConfig           : 0      
 49: TimepixConfig           : 0      
 50: TimepixData             : 0      
 51: CspadCompressedElement  : 0      
 52: OceanOpticsConfig       : 0      
 53: OceanOpticsData         : 0      
 54: EpicsConfig             : 0      
 55: FliConfig               : 0      
 56: FliFrame                : 0      
 57: QuartzConfig            : 0

Print in summary all damage records

QCStatistics::printDamageRecords()
Total number of damage records : 33
Damage Record: Dgram:    72  depth:0  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:    72  depth:1  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:    72  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution
Damage Record: Dgram:   123  depth:0  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   123  depth:1  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   123  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution
Damage Record: Dgram:   272  depth:0  type:Xtc                       damage:     2  bit: 1: DroppedContribution
Damage Record: Dgram:   272  depth:1  type:Xtc                       damage:     2  bit: 1: DroppedContribution
Damage Record: Dgram:   272  depth:2  type:Any                       damage:     2  bit: 1: DroppedContribution
Damage Record: Dgram:   284  depth:0  type:Xtc                       damage: 81920  bit:14: UserDefined
                                                                                    bit:16: ContainsIncomplete
Damage Record: Dgram:   284  depth:1  type:Xtc                       damage: 81920  bit:14: UserDefined
                                                                                    bit:16: ContainsIncomplete
Damage Record: Dgram:   284  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution
Damage Record: Dgram:   332  depth:0  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   332  depth:1  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   332  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution
Damage Record: Dgram:   479  depth:0  type:Xtc                       damage:     2  bit: 1: DroppedContribution
Damage Record: Dgram:   479  depth:1  type:Xtc                       damage:     2  bit: 1: DroppedContribution
Damage Record: Dgram:   479  depth:2  type:Any                       damage:     2  bit: 1: DroppedContribution
Damage Record: Dgram:   700  depth:0  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   700  depth:1  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   700  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution
Damage Record: Dgram:   790  depth:0  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   790  depth:1  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   790  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution
Damage Record: Dgram:   923  depth:0  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   923  depth:1  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:   923  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution
Damage Record: Dgram:  1025  depth:0  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:  1025  depth:1  type:Xtc                       damage: 65536  bit:16: ContainsIncomplete
Damage Record: Dgram:  1025  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution
Damage Record: Dgram:  1305  depth:0  type:Xtc                       damage: 81920  bit:14: UserDefined
                                                                                    bit:16: ContainsIncomplete
Damage Record: Dgram:  1305  depth:1  type:Xtc                       damage: 81920  bit:14: UserDefined
                                                                                    bit:16: ContainsIncomplete
Damage Record: Dgram:  1305  depth:2  type:CspadElement              damage: 32768  bit:15: IncompleteContribution

Data quality check

  • psana-based analysis for selected types of data with known properties.
  • This is data-specific check, which can not be universal for all data types.

Indexing?

The same procedure of the loop over datagrams with iteration over XTC containers can be used for XTC data indexing..., but need to read all files for streams and chunks within run using the same algorithm like in psana.

Command line interface

xtcqc -h -l <logfile> -b <basedir> <fname1> [<fname2> <fname3> ...]

Optional parameters:
-h - help, showing usage of the command line parameters.
-l - log-file; if specified, then output will be re-directed to this file, otherwise on stdout.
-b - base-directory, i.e.: /reg/d/psdm/<INSTRUMENT>/<experiment>/xtc; if specified, than all xtc file names will be defined w.r.t. this directory.

Non-optional parameters:
<fname1> [<fname2> <fname3> ...] - all xtc file names should be listed without options.

Unpleasant feature of getopt:
The options -l and -b require the file name as a trailed optional parameter. If the optional parameter is mistakenly missed, then entire input will be corrupted.

Database for XTC Quality Check

Whenever the xtc file is processed, the results are saved in the database

 
/reg/d/psdm/<INSTRUMENT>/<experiment>/xtcqc/<xtc-like-file-name>.qc

Web-based control and access

Igor is going to produce web-based interface, like for translator, with functionality as follows.

  • Show list of files with processed/not-processed flag,
  • Control on job submission for quality check,
  • Browser for QC results of particular file/run.
  • No labels