Notes on the Big Run and Monte Carlo Data Access
[prepared for short presentation on 19 March 2008]
What is the Big Run?
What *types* of MC datasets are typically generated?
What MC datasets have been generated?
What triggers the generation of new MC datasets?
What is scheduled for the near (pre-launch) future?
What about Pass6?
How does one stay up-to-date with regard to exactly what is available and planned?
Once I've figured out what data to use, how do I get my hands on it?
What is the Big Run?
A catch-all term encompassing much of all production Monte Carlo work since December 2007. There are many goals (which change with time!), including:
- produce a very large background dataset (8 full days)
- produce a 5-day background+full sky model for OpsSim2
- produce a 1-year interleaved background+full sky model dataset
What *types* of MC datasets are typically generated?
- Relatively frequent "standard" datasets (with selected, typical configurations)
- allGamma
- 100% gammas between 18 MeV and 562 GeV
- Normal trigger, NO OBF
- ~10M evts generated
- generated on upper hemisphere
- allMuon
- 100% muons
- NO trigger, NO OBF
- ~few M evts generated
- sample day background
- all background sources
***spanning one calendar day (0.2s runs every minute => 1440 runs)
- NO trigger, NO OBF
- background
- all background sources
- temporally contiguous runs (large statistics...~1 B evts generated)
- normal trigger, normal OBF
- Less frequent specialized datasets
- background + full sky model
- Interleaved background + full sky model
- GRBgrid (a GRB in every job)
- special trigger and/or OBF
- Unusual orientations of the s/c, incl. pointed observations
- special orbit files
What MC datasets have been generated?
- All production MC datasets are created as GLAST Pipeline "tasks" via batch jobs on the SLAC and Lyon compute farms.
- The production of essentially all MC datasets is documented on this web page, Service Challenge Monte Carlo Processing Summary
- Note that this Confluence page documents the production, but neither the code nor the complete configuration that goes into the production.
What triggers the generation of new MC datasets?
- A (significant) new GlastRelease (e.g., a new generation of classification tree analysis)
- Request from C&A group
- Request from other analysis or detector group
- Requests must (ultimately) be channeled through Richard Dubois
What is scheduled for the near (pre-launch) future?
- Will we do the 1-year interleaved background + full sky model run? Open question...
- See Richard's Big Run Checklist page for all the details
- Richard and Julie are your best contacts for Big Run plans (but they are both really busy, so first try to discover the answer yourself)
- Hopefully fixing the rough edges discovered during OpsSim2 data processing review
What about Pass6?
- There are as of this moment NO production MC datasets with Pass6 analysis
- As of yesterday morning still hammering out the finishing touches on the definition of Pass6 and the mechanism to reprocess some existing data
- Once Pass6 is up and running, one should expect all future MC datasets to use it
- With some luck, the opssim2 dataset (8 full days background + sky model) may be reprocessed by sometime next week...stay tuned.
How does one stay up-to-date with regard to exactly what is available and planned?
Once I've figured out what data to use, how do I get my hands on it?