2021.06.01 -- Data Calibration / Reconstruction Meeting

Agenda: 

1. Previous Action Items
2. 2019 Data Reconstruction / Calibration - All
3. Online Reconstruction and Monitoring Tools for 2021 Run - Jeremy
4. Proposed Changes to ReconstructedParticle - Norman
5. AOB

Attendees: Norman, Alic, Cameron, Jeremy, MattG, Maurik, Nathan, PF, Robert, Tongtong

Notes:

• Previous action items:
  • Cameron reported on studies he has conducted where he has dropped all SVT fits with times below -50ns, irrespective of whether a single or double pulse was fit to the waveform. This resulted in a loss of ~10% of tracks which had previously passed the analysis quality cuts, indicating roughly 10% of the tracks had at least one hit using a clearly unwanted fit. Work will continue to understand what pulse fits should be kept, but this proposal is clearly only removing "bad" hits/fits.
  • Currently we anticipate reconstructing the ~10% sample with the current set of Drivers. The SVT needs the raw data in order to determine the baseline calibrations and to do hit studies. However, we will need to reduce the output size for final production reconstruction of the full 2019 data set.
  • Reconstructing 10% of the data under current settings would result in ~150TB of disk-resident lcio files. HPS only has a guarantee of ~85TB on /cache, so we would be flushing the files we currently have on /cache and files would not be disk-resident for long, but files would be written to tape before being deleted. No issue brought up about flushing files currently on /cache. Size is expected to decrease after removing unneeded collections or objects.
  • We discussed whether we need 10% of the data, or whether a smaller fraction would be sufficient. It was suggested that the SVT group come up with a set of specific files that are needed for the baseline calibration and proceed with that.
• Jeremy gave an interactive demonstration of the online reconstruction and monitoring tools. The online reconstruction tools are documented at Online Reconstruction Tools.
• There was a long discussion on what runs are considered "good" for 2019. We decided to use 10000 to 10740 removing runs specifically marked as bad in the run db.

Action Items:

• MattG will look into whether we can prune any of the existing monitoring plots to streamline things for the upcoming run.
• We need to develop monitoring Drivers which only book and fill histograms for the online distributed monitoring.
• We need to revisit the list of "good" runs and define exactly what goes into that list. Maurik agreed to prepare a list of all runs marked as "bad" in the JLab run db.