You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

 

Transitioning to Git: 

Joanne foresees a large amount of work involved in transitioning to git due to the SLAC cvs structure involving large numbers of symlinked packages.  She is concerned with possible drift from glastrelease and ScienceTools, since both have overlapping tools.

We settled on keeping the release manager/cvs system the same for now, but have all commits go through git.  An automated script/utility would port any git changes to CVS and would be the only 'user' to have commit privileges on the CVS repository in order to prevent collisions.

 

Updates from Tuesday Morning:

Brian and Jim began process of porting CVS to git using cvstogit utility

-in principle you can restrict the groups that have write permission

-newer CVS to git seems to retain the history, based on svn to git utility

-main hurdle is permissions problem

-Need to restrict checkins to CVS to only git script

-Need to be able to allow users to access CVS users area without affecting/conflicting with git/cvs checkins

-submodules/details not sorted yet, just prepping for transition now by moving repo to SSD

 

Brian update:

-Took list and made CSV file for table on confluence, grep through repo for needed files

-Copied entire CVS repo locally to machine in order to ease transition

-Doesn't anticipate too many problems with this, but there are a few hiccups like author matching

-Org: fermi-lat

-Alex and I will be added to the fermi-lat team on github

-Test conversion of Likelihood (github.com/brianv0/Likelihood)

 

 

Externals:

 

Generally in favor of jettisoning most external packages.  Python can be provided by users, no need to carry our own versions around.

IRF generation require ROOT, nothing else does.  Can be removed from main FSSC ScienceTools for the public release, however, it would be good to keep it around for internal/developer use.

 

Updates from Tuesday Morning:

-Went through externals and determined statuses/necessity of each package

-Also determined whether or not we needed to provided packages or if they were available in conda

 

Other Notes About Science Tools:

  • Many pieces of analysis rely on ST being backwards-compatible.
  • Is the FSSC going to be doing more than just bug-fixes?
  • Will there be more development?

DOE Commitment:

  • 1.5 FTEs after 2018, down from 3 FTEs. Needs to be complete by end of FY ’18. Gradual ramp-up and transition desired.
  • Hardware purchasing? Server infrastructure, databases, etc.
  • IFC still funding hardware purchases
  • Hardware will come to SLAC
  • ORACLE server migration soon

Want to prepare for a long-term mission continuation.

Revise if appropriate and effort not excessive.

  

 

Data Pipeline:

 

  • Data arrives from GSFC
  • L0 ingest triggered by FastCopy post-processing script (into database)
  • Halfpipe keeps track of what has been delivered
  • Once tidy & ordered, hands off to L1
  • L1 registers in data catalog & sends FITS to GSFC

 

 

 

Data Distribution:

Get rid of FastCopy?

  • New version of FastCopy won't work with current hardware & vice versa
  • Use another transfer mechanism?
    • Would require changing all scripts downstream
    • Since everyone is having issues, may need to look at alternatives
  • New transfer mechanism would require many changes across the players
    • SSC, White Sands, MOC, GIOC, LISOC
    • FOT using VMs w/ SL5 to maintain

  

Astroserver vs. Data Server:

Do we need it?

  • What does AS do that DS doesn't?
    • Allows for multiple private data sets that can be checked before approving for release
    • Example: multiple P8 revisions 
  • How integrated is it into LAT team everyday life?
    • Limited subset of people
  • Gamma-ray bursts - not all event classes available at FSSC
    • Make available, but use SLAC authentication to access?
  • Method of maintaining some functionality but without a database?
  • Scripting interface available on the astroserver but only from SLAC
  • May be available via ssh tunneling
  • Remote authentication an option for the astroserver.

 Note: Astroserver web interface had 65 uses in last 2 months by 13 users. But most people likely use scripting.

 

Data Catalog: 

Data Catalog functionality is something that MUST be maintained. Need someone for this, definitely.

Data catalog issues that can arise: 

  • Changes to data file path /nfs/...
  • Trying to display too many folders in a browser. Not really an issue on command line. Command line only available onsite.
  • Need to log and get statistics on who is using these applications
  • MC data, intermediate data products for trending, etc.

Data server & astroserver may be combinable..?

  • Host everything in FITS format
  • Perhaps include merit data
  • No need to host instrument monitoring quantities
  • Create a "catalog-ready" data set that is queryable via python?

 

Mission Planning/Flight Operations

Actions:

  • Finalize and document list of needed permissions on the ISOC Mission Planning page
  • Get Fermi Sharepoint access for NASA people (SLAC windows account is not enough)
  • Robin/Elizabeth/Jerry to propose round robin schedule for weekly mission planning
  • document support cron scripts for SAA checks and planning product generation
  • document occasional mission-week-boundary LAT command scheduling problem
  • NASA planners to take over LCI calibrations planning, effective immediately
  • FSSC to consider having LAT operations workstation/tool installation

 

Mission Planning notes from Elizabeth:

  • fermilnx01 must be used for mission planning when doing anything that required file management 
  • Use ssh -XY when logging into SLAC
  • LAT planning typically performed on Monday. Requires products from the FOT on Monday:
    •  SAA predicts
    •  Fermi ephemeris

 

    • Script available to verify that products are good. Sends emails to Rob.
    • Info about scripts are in the confluence MP page.
    • Can see if products are good or bad by looking at link on confluence MP page.

 

  • No labels