Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Pipeline needs: Brian Van Klaveren 

  •  Add S3DF/S3DFDATA sites to DEV Pipeline, for L1 testing
  •  The PROD pipeline interface hangs when loading a task (example)
  •  Ability to delete tasks with runs from the pipeline admin (DEV and PROD)
  •  Can we pipe slurm_pilot into the standard logfile.txt for each task?
  •  Debugging DQM ingestion scriptsscripts#JAVA_HOME

SLURM questions:

  •  Document all SLURM options supported by the pipeline interface Brian Van Klaveren 
  •  What are all the available SLURM configurations (queue, memory, etc.) for S3DF? Yang, Wei 
  •  Do we need the analogous of a preempt queue in S3DF? What would that be? Richard Dubois 
    •  Current setting: --account fermi:L1

Data migration issues: Richard Dubois 

  •  SCons symlinks were not migrated correctly: /afs/slac/g/glast/applications/install/@sys/usr/bin/scons was not copied to /sdf
    •  RD: not sure the correct fix for the @sys, but it did get copied to /sdf/data/fermi/a/applications/SCons/2.1.0/bin/scons and can be used from there
    •  MEM: this is an acceptable workaround for now. But really we should get rid of this whole SCons nightmare
  •  stag is also broken: /afs/slac.stanford.edu/g/glast/applications/install/@sys/usr/bin/stag was not installed correctly on /sdf
    •  RD: stag is was linked to nfs and is here: /sdf/data/fermi/n/u05/stag/0.2.5/redhat6-x86_64-64bit/stag
    •  MEM: this is also broken (just try calling it). The only working path is the afs one, but only works from rhel6-64
  •  xrootd stuff is also broken. See for example the PROD, DEV folders in here: /sdf/group/fermi/a/applications/xrootd(Wilko Kroeger )
    •  MEM: this /sdf/group/fermi/a/applications/xrootd/dist/v3.1.1/i386_rhel60/bin/glast_wrapper.sh does not work (can't find fs)
       

Container needs: Yang, Wei 

  •  Need a container where the architecture is set explicitly to rhel6 instead of centos6 (architecture name confuses SCons)
  •  While we are at it, can we add CVS and vim to the new container?
  •  We need a fortran compiler as well, specifically g77 and f2py. From the Makefile of IGRF: G77 = /usr/bin/g77 and F2PY = f2py2.6
  •  Singularity gets confused by symlinks. Solution: more bind mounts. Script start_rhel6.sh just added to L1Proc for compilation

The following sections are obsolete


CVS is broken: Moved to git (completed on )

  • Documentation about stag (only works on rhel6-64). Documentation from Joanne on making branch tags with cvs
  • /afs/slac.stanford.edu/g/glast/applications/install/@sys/usr/bin/stag disappeared from path

Plan forward: move everything to git → All done, as of  


Container quirks: predictably, that went away on its own (checked  )

  • Surreal behavior with folder names: only difference is in the last line
Code Block
languagepy
titleThis code works:
#!/sdf/group/fermi/a/isoc/flightOps/rhel6_gcc44/ISOC_PROD/bin/shisoc python2.6

import os
import sys

L1Name = os.environ.get('L1_TASK_NAME') or "L1Proc"
L1Version = os.environ.get('PIPELINE_TASKVERSION') or os.environ.get('L1_TASK_VERSION') or "5.9"

BuildVolume = '/sdf/group/fermi/a/ground/releases/volume11'
L1BuildBase = os.environ.get('L1_BUILD_DIR') or os.path.join(BuildVolume, 'L1Proc')
L1Build = os.path.join(L1BuildBase, '5.9')
Code Block
languagepy
titleThis code does NOT work:
#!/sdf/group/fermi/a/isoc/flightOps/rhel6_gcc44/ISOC_PROD/bin/shisoc python2.6

import os
import sys

L1Name = os.environ.get('L1_TASK_NAME') or "L1Proc"
L1Version = os.environ.get('PIPELINE_TASKVERSION') or os.environ.get('L1_TASK_VERSION') or "5.9"

BuildVolume = '/sdf/group/fermi/a/ground/releases/volume11'
L1BuildBase = os.environ.get('L1_BUILD_DIR') or os.path.join(BuildVolume, 'L1Proc')
L1Build = os.path.join(L1BuildBase, L1Version)