Infrastructure needs for L1Proc migration

Pipeline needs: Brian Van Klaveren 

• Add S3DF/S3DFDATA sites to DEV Pipeline, for L1 testing
• The PROD pipeline interface hangs when loading a task (example)
• Ability to delete tasks with runs from the pipeline admin (DEV and PROD)

SLURM questions:

• Document all SLURM options supported by the pipeline interface Brian Van Klaveren 
• What are all the available SLURM configurations (queue, memory, etc.) for S3DF? Yang, Wei 
• Do we need the analogous of a preempt queue in S3DF? What would that be? Richard Dubois 
  • TBD: currently only roma and milano partitions available during the test period. Worst case is we reserve 1k cores for L1 when the time comes.

CVS is broken: Brian Van Klaveren Kelly (Arrighi), Heather Joanne Bogart 

Trying to create a branch in svac/L1Pipeline with cvs tag -b, however:

• stag complains that my tag is not standard. But I’m not calling stag (L1Pipeline was never ported to SCons)
• /afs/slac.stanford.edu/g/glast/applications/install/@sys/usr/bin/stag disappeared from path (see comment below)
• stagging with -custom and -branch does not work (basically, stag won’t let me create a branch either)

Here are some options (porting the whole svac to git might be best for long-term support)

• disentangle cvs from stag (no idea what that would look like, and why cvs is still attached to stag on s3df)
• port svac to git, which honestly at this point would be my preferred option (no need to make it public tho)

Data migration issues: Richard Dubois 

• SCons symlinks were not migrated correctly: /afs/slac/g/glast/applications/install/@sys/usr/bin/scons was not copied to /sdf
  • RD: not sure the correct fix for the @sys, but it did get copied to /sdf/data/fermi/a/applications/SCons/2.1.0/bin/scons and can be used from there
  • MEM: this is an acceptable workaround for now. But really we should get rid of this whole SCons nightmare
• stag is also broken: /afs/slac.stanford.edu/g/glast/applications/install/@sys/usr/bin/stag was not installed correctly on /sdf
  • RD: stag is was linked to nfs and is here: /sdf/data/fermi/n/u05/stag/0.2.5/redhat6-x86_64-64bit/stag
  • MEM: this is also broken (just try calling it). The only working path is the afs one, but only works from rhel6-64
• xrootd stuff is also broken. See for example the PROD, DEV folders in here: /sdf/group/fermi/a/applications/xrootd(Wilko Kroeger )
  • MEM: this /sdf/group/fermi/a/applications/xrootd/dist/v3.1.1/i386_rhel60/bin/glast_wrapper.sh does not work (can't find fs)
     

Container needs: Yang, Wei 

• Need a container where the architecture is set explicitly to rhel6 instead of centos6 (architecture name confuses SCons)
• While we are at it, can we add CVS and vim to the new container?

Container quirks: Yang, Wei

• Singularity gets confused by symlinks (example: /sdf/home/m/monzani/fermi-user vs /sdf/group/fermi/user/monzani). Is that the expected behavior? For now, workaround is --bind /sdf:/sdf (ugly but workable)

• Surreal behavior with folder names: only difference is in the last line

#!/sdf/group/fermi/a/isoc/flightOps/rhel6_gcc44/ISOC_PROD/bin/shisoc python2.6

import os
import sys

L1Name = os.environ.get('L1_TASK_NAME') or "L1Proc"
L1Version = os.environ.get('PIPELINE_TASKVERSION') or os.environ.get('L1_TASK_VERSION') or "5.9"

BuildVolume = '/sdf/group/fermi/a/ground/releases/volume11'
L1BuildBase = os.environ.get('L1_BUILD_DIR') or os.path.join(BuildVolume, 'L1Proc')
L1Build = os.path.join(L1BuildBase, '5.9')

#!/sdf/group/fermi/a/isoc/flightOps/rhel6_gcc44/ISOC_PROD/bin/shisoc python2.6

import os
import sys

L1Name = os.environ.get('L1_TASK_NAME') or "L1Proc"
L1Version = os.environ.get('PIPELINE_TASKVERSION') or os.environ.get('L1_TASK_VERSION') or "5.9"

BuildVolume = '/sdf/group/fermi/a/ground/releases/volume11'
L1BuildBase = os.environ.get('L1_BUILD_DIR') or os.path.join(BuildVolume, 'L1Proc')
L1Build = os.path.join(L1BuildBase, L1Version)