Page History

Essential

dependencies:

•  Move away from afs disks
•  Build L1 locally, using container
•  Run executables using container
•  Use SLURM commands in xml
•  Move Migrate all L1 packages to gitgithub

Migration milestones:

•  CVS is broken
•  Singularity
•  Pipeline
•  Data migration

• Build L1 locally on containers
•  Run single task in S3DF pipeline
•  Update the L1Proc xml template
•  Run full L1Proc in S3DF with LPA

Sustainability and optimization

:

•  Get rid of SCons builds for L1 code
•  Upgrade python to v3 and above
•  Can we avoid crumbling altogether?
•  Optimize # of crumbs for new CPUs
•  Use flash for staging instead of xrootd?
•  import the locking script to slurm preempt

Open questions:

•  do we still need a wrapper for cleanup?
•   do we need isocRun for every executable?
•  what are the options and queues for slurm?SLURM queue options?
•  non-preemptable queue: --account fermi:L1
•  how does the scratch space work on S3DF?
•  FSW: need new staging dir, export, logging

Infrastructure needs:

•  CVS is going away
•  Singularity containers
•  Pipeline updates
•  Restore pipeline_wrapper
•  Fix the log viewer
•  Final rsync for u41
•  Fix the ingestor scripts

S3DF Cheatsheet:

• Install ssh config as described here
• to login from terminal: ssh slacd
• to ssh as glastraw, run kinit first
• NOTE: use terminal not XQuartz(?)
• afs was migrated to: /sdf/group/fermi/a/
• nfs was migrated to: /sdf/group/fermi/g/

• Singularity environment: see below

Non-L1  milestones:

•  Test full L1Proc in S3Df with LCI data
•  Update non-L1 xml (flagFT2, merge, setQual)
•  Update L1Proc build/test/update instructions

Additional (non L1) deliverables:

•  Migrate DQM/trending reports
•  Migrate cronjobs for DQM reports
•  Document L1Proc deployment