Migrating ASP to S3DF

Deadlines

• November 1, 2023:  rhel6-64 logins disabled.
• March 31, 2024:  LSF license expires.

Resources

• Migration scripts, etc., are maintained in the asp_migration github repo.
• Locations of copies of pre-built Fermi software releases at S3DF:  /sdf/data/fermi/a/ground/releases
• s3df-migration slack channel in the Fermi-LAT workspace.
• Get Started on S3DF - Cheat Sheet
• Migrating L1Proc to S3DF
• ASP Installation Instructions
• Using RHEL6 Singularity Container

Non-Pipeline-II Services

• procmail-triggered processing of incoming GCN Notices
• cron job to ingest GCN notices into ASP db tables
• trscron job to prepare Monitored Source List light curves and sent them to GSFC using FastCopy
  • Executable: /afs/slac/g/glast/ground/links/data/ASP/makeDrpLcTables.sh

To Do List

• Update ASP CVS repository with most recent modifications.
• Convert the ASP and ASP-scons CVS areas to git repos.
  • Move these repos to the fermi-lat github org.
• Write up current installation instructions on Confluence.
• S3DF installation of GPLtools:  /sdf/data/fermi/a/ground/PipelineConfig/GPLTools/prod.
• S3DF location of ORACLE_HOME:  /afs/slac.stanford.edu/package/oracle/f/11.1.0/amd64_linux26/11.1.0 (This is accessible using  `singularity shell --bind /afs`).

• Using  /sdf/data/fermi/a/ground/releases/volume02/ScienceTools-09-32-05/ , numpy is missing libblas.so.3:

  >>> import numpy
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/sdf/data/fermi/a/ground/GLAST_EXT/redhat6-x86_64-64bit-gcc44/python/2.7.2-gl4/lib/python2.7/site-packages/numpy/__init__.py", line 170, in <module>
      from . import add_newdocs
    File "/sdf/data/fermi/a/ground/GLAST_EXT/redhat6-x86_64-64bit-gcc44/python/2.7.2-gl4/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
      from numpy.lib import add_newdoc
    File "/sdf/data/fermi/a/ground/GLAST_EXT/redhat6-x86_64-64bit-gcc44/python/2.7.2-gl4/lib/python2.7/site-packages/numpy/lib/__init__.py", line 18, in <module>
      from .polynomial import *
    File "/sdf/data/fermi/a/ground/GLAST_EXT/redhat6-x86_64-64bit-gcc44/python/2.7.2-gl4/lib/python2.7/site-packages/numpy/lib/polynomial.py", line 19, in <module>
      from numpy.linalg import eigvals, lstsq, inv
    File "/sdf/data/fermi/a/ground/GLAST_EXT/redhat6-x86_64-64bit-gcc44/python/2.7.2-gl4/lib/python2.7/site-packages/numpy/linalg/__init__.py", line 51, in <module>
      from .linalg import *
    File "/sdf/data/fermi/a/ground/GLAST_EXT/redhat6-x86_64-64bit-gcc44/python/2.7.2-gl4/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 29, in <module>
      from numpy.linalg import lapack_lite, _umath_linalg
  ImportError: libblas.so.3: cannot open shared object file: No such file or directory

  On centos7, libblas.so.3 lives in /lib64.  Do we need to install BLAS and LAPACK in the singularity image?

• Try building ASP on S3DF using /sdf/data/fermi/a/ground/releases/volume02/ScienceTools-09-32-05/
  • Find the S3DF equivalent functionality provided by the build tools and related applications in $GLASTROOT=/afs/slac.stanford.edu/g/glast.
• Replace trscron job for sending Monitored Source light curve data to GSFC.
• Update ASP to use an offline dev db implementation for testing.
• Hello, world pipelines on S3DF using legacy Fermi software releases.