2021.06.01 -- Data Calibration / Reconstruction Meeting

Agenda: 

1. Previous Action Items
2. 2019 Data Reconstruction / Calibration - All
3. Online Reconstruction and Monitoring Tools for 2021 Run - Jeremy
4. Proposed Changes to ReconstructedParticle - Norman
5. AOB

Attendees: Norman, Alic, Cameron, Jeremy, MattG, Maurik, Nathan, PF, Robert, Tongtong

Notes:

• Previous action items:
  • Cameron reported on studies he has conducted where he has dropped all SVT fits with times below -50ns, irrespective of whether a single or double pulse was fit to the waveform. This resulted in a loss of ~10% of tracks which had previously passed the analysis quality cuts, indicating roughly 10% of the tracks had at least one hit using a clearly unwanted fit. Work will continue to understand what pulse fits should be kept, but this proposal is clearly only removing "bad" hits/fits.
  • Currently we anticipate reconstructing the ~10% sample with the current set of Drivers.
    • The SVT needs the raw data in order to determine the baseline calibrations and to do hit studies.
    • The final comparisons of the SeedTracker/GBL and Kalman Filter track-finding efficiencies need additional statistics to investigate corners of phase space.
  • We will still need to reduce the output size for final production reconstruction of the full 2019 data set.
  • Reconstructing 10% of the data under current settings would result in ~150TB of disk-resident lcio files. HPS only has a guarantee of ~85TB on /cache, so we would be flushing the files we currently have on /cache and files would not be disk-resident for long, but files would be written to tape before being deleted. No issue brought up about flushing files currently on /cache. Size is expected to decrease after removing unneeded collections or objects.
  • We discussed whether we need 10% of the data, or whether a smaller fraction would be sufficient. It was suggested that the SVT group come up with a set of specific files that are needed for the baseline calibration and proceed with that.
• Jeremy gave an interactive demonstration of the online reconstruction and monitoring tools. The online reconstruction tools are documented at Online Reconstruction Tools.
• There was a long discussion on what runs are considered "good" for 2019. We decided to use 10000 to 10740 removing runs specifically marked as bad in the run db.
• We ran out of time to discuss proposed changes to the ReconstructedParticle. This is postponed to the next reconstruction meeting or perhaps analysis meeting.

Action Items:

• MattG will look into whether we can prune any of the existing monitoring plots to streamline things for the upcoming run.
• We need to develop monitoring Drivers which only book and fill histograms for the online distributed monitoring. This should go hand-in-hand with the work MattG is doing.
• We need to revisit the list of "good" runs and define exactly what goes into that list.