Versions Compared

Old Version 5

changes.mady.by.user Kelly (Arrighi), Heather

Saved on

compared with

New Version Current

changes.mady.by.user Joanne Bogart

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

Minutes Kickoff Meeting October 20, 2008

Attendees:  Joanne Bogart, Emmanuel Cephas (EVO),  Richard Dubois, Tom Glanzman, Navid Golpayegani, Tony Johnson, Chuck Patterson,  Leon Rochester, Jon Vernaleo, Eric Winter (EVO)

SCons Status

Navid reports that the RHEL4 builds for ScienceTools are rolling.  The RHEL5 externals are ready, but the builds are stuck until SLAC Computing can reconfigure the two boxes they have provided.  Currently the issue is that AFS tokens are not being passed.  Wondering about how urgently we need to get these RHEL5 to fulfill the GSSC's request.  John V reports that GSSC has proceeded with its own builds and was mostly interested in learning about any potential problems with gcc4 compatibility.  They have had to include f2c to get the SciTools to build with gcc4.  Richard also asks that the RHEL5 builds be set up for CMT as well. 

The Windows version is ready to go.  Navid is working on compiling Qt for the Mac.  Qt is necessary for GoGui and RM.  The plan is to share as much code as possible.  The Mac Qt builds are horribly slow.  The config step too almost 12 hours.

OBF - Navid needs to talk to Tracy about three different version of libraries that have the same name.

Karen has created an initial version of the Release Manager web pages.  We need to schedule a meeting with Karen and other interested parties (Heather, Joanne, Navid, Jim, Richard, Chuck, Tony..) to discuss status.

Emmanuel reported concerning GR build status.  There are outstanding repackaging issues to be addressed for OBF, however in the meantime, Emmanuel is setting an rpath (runtime path) option for the linker to allow the builds to proceed.  Otherwise, SCons checks for all required libraries before the build and never actually gets to a compilation step.   Clearly this is not a long term solution, but allows testing to proceed.

GoGui

Joanne reported that there are 4 missing items:

1. Tagging Feature 
Joanne would like to understand how most users tag.  She (and I) primarily use rtag, but there are instances when the tag command is required.  Joanne has located and understands the existing MRvcmt code.  She should be able to translate it over to GoGui.

2. Windows
First step is to understand what SCons provides.  Joanne is stuck trying to get SCons to create a VS project.  It is expected that this will not provide all the required developer features we desire.  The anticipated course of action will be to follow the model used for CMT, where we have separate code that builds the Visual Studio project files for us.  Toby has some python code for this purpose and Tracy has previously worked with that code to force SCons to produce more useful project files.

3.  Override directories
Need to modify the way we use SCons to make override directories possible.  Some of Joanne's existing GoGui code assumes that this ability exists.  The plan is to support one over-riding directory.  Leon states he can live with that.  That override directory may contain any number of packages.  The fix will come in the SConstruct files, which while not pretty, will be invisible to most users.

4. Not holding us up but annoying..
There are several places where version numbers are used.  The SConscript files have a version number.  When GoGui checks out a package, there is no longer a version subdirectory. [Clarification: In the initial checkout of a container and its subpackages there is no sign of package version in the directory structure. It is in the SConscript file, however, and GoGui displays it. If you check out another copy of a package, either tagged or HEAD, into the same container, it must be assigned a directory which does include versioning, e.g. xmlBase-03-04-05. When it comes time to build, if either xmlBase or xmlBase-03-04-05 is selected, only the latter will be built because of the way the names are sorted. This is disconcerting if not downright misleading. GoGui will do a better job in a future release, probably by disabling build and clean on packages (like xmlBase in the situation described) which will not in fact be built or cleaned. Joanne]

GoGui is available on RHEL4 and for Windows.  It was suggested that we could have a GoGui demonstration on Wednesday.

64 Bit 

Navid reports that GSFC is moving to all 64 bit machines with compatibility libraries for 32 bits.   John mentioned that there are some SciTools failures that occur only on 64 bit machines.  Navid and John agree that so far, those issues have stemmed from missing compatibility libraries.  Richard notes that most of the tools are being run on 64 bit machines at SLAC, as the majority of machines area 64 bits in the farm.

Compilers

GR readiness for gcc4?  Dependent upon our migration to more recent external libraries such as Gaudi and G4.  There are no current plans to migrate either immediately

Visual Studio:  Toby reports that his group is taking care of their own needs concerning Visual Studio upgrades.  Leon expressed interest in the new VC++ that expands the debugger to allow viewing vector contents.  We should ask Tracy his opinion of upgrading.

ROOT

 Tom reports that there are three outstanding xrootd issues and one extra added by Tony:
https://jira.slac.stanford.edu/browse/GXR-46
https://jira.slac.stanford.edu/browse/GXR-45
https://jira.slac.stanford.edu/browse/GXR-44
https://jira.slac.stanford.edu/browse/GXR-43
He has been speaking to Andy, who suggests that he would prefer finding a way to more easily slide in newer versions of the xrootd client code into our ROOT library distributions.  At worst, we can always do what we did for v5.18.00c-gl1, where we obtained their updated xrootd client code and compiled it along with the rest of ROOT.

Skimmer Troubles

Current woes concerning skimming many (~10k) small files.  The memory use tends to grow with the number of files accessed.  Tony expressed concern over the rebuilding of indices, even in jobs where it is not necessary.  This may be impacting the memory use.  Perhaps we can flag runs in the data catalog to note whether or not they contain bad indices.  Heather will forward a message from Warren which explains when the fix to the merge code was deployed and which run was the first to use this fix.

FluxSvc

 Heather needs to talk to Tracy and Toby concerning FluxSvc and the status of the PointingInfo class:

GR currently uses FluxSvc v6r53, the latest tag is actually v6r54p2.

I'm inclined to move forward - however - the more recent tags include the PointingInfo class consolidation that was under discussion at the end of May.  At that time, I held off on the latest FluxSvc tags until it was sorted out how AnaTup's PtValsTool (I think) would be updated (or not)... as I recall we reverted PtValsTool to its original setup.  My question is... the changes in FluxSvc - which I believe includes a new class, IPointingInfo..  is that ok to pick up? I'm guessing we just don't actually use it?

Python

Troubles are brewing with full scipy and its requirement of lapack and blas.  Need to have a discussion with Toby and Jim.

Workbook

Chuck reports that his focus is on the iLat pages.  There have been concerns expressed about directing users to use specific versions of SciTools in conjuction with iLat.  Chuck suggests rolling out a tutorial that focuses on production versions of iLat and ScienceTools installed on SLAC public.  Heather would like to see the setup procedure for iLat be consistent for csh and bash.

The Data Access pages are also being updated.  The navigation bars have been laid out according to Tony's specifications.  However, at this time, iLat has taken priority.  Richard suggests that Chuck complete work on the standard point source analysis using iLat, then work on the data access pages, followed by creating the pages for extended sources.

How to update the workbook?  Chuck explained the working model used for the Science Tool pages.  The authors check pages out of CVS and check in the changes made using any HTML editor.  When they are complete, they notify Chuck who then incorporates them into the online version of the workbook.  The majority of the time spent by Chuck is making sure the format is consistent with the rest of the workbook.  Leon and Tom both asked how do we check out a specific page from the workbook?  Chuck offered to write up some directions.

heasoft versus astrotools

The current iLat instructions point at the ftools installed in the astrotools area of GLASTROOT at SLAC.  Heather does have a more recent version from heasoft 6.5 installed as well.  Tom and Tony (and possibly Jim) have been using that one.  It would be helpful to determine what addtional items should be included in the new heasoft area, and if we are agreed that this will be the location of future installation of ftools and associated utilities.

Wired

There have issues with excessive memory use.  Tony will try out the new GR with updated RootIo with some additional JO parameters to see if that helps alleviate the problem.  Tony also suggests shutting off the rebuilding of ROOT indices when it is not necessary - this may also be a memory hog.

There are a variety of other issues:

1. Relations files
Never got relation file reading working yet.  Should review the HepRepSvc interface.

2.  MC vesus real data
Currently two versions of the jobOptions are required to handle MC versus real data jobs.  It would be better to handle this automatically.

3. Merit Display
Is it possible to provide Instance Trees similar to FRED?  This would be one way to display the contents of merit in Wired, as has now been added to FRED by Leon.

FRED

Outstanding issue with RHEL4 and Fox library.  Currently, one cannot run FRED on RHEL4 with our distributed version of FRED and Fox.  Joanne has pursued this issue in the past.  Heather and Joanne will discuss this further during the week and see if there is an easy solution such as upgrading the version of fox.

Topic List

  • SCons
    • Windows support
    • MAC builds
    • end user scenario?
  • GoGui
    • Windows support (smile)
  • New Release Manager GoGui 
    • Talk to Karen about the current status and what else may be required.
  • RHEL5
  • RHEL4 builds
    • ST optimized builds?
    • Do we have an optimized version of G4 for GR?
      • Testing so we can try this out for the pipeline
  • LAT Workbook
    • Review of edit/update mechanism; can it be streamlined/simplified?
    • Any interesting tools to help keep pages current automatically?
    • How to best allow interested users to make contributions directly (e.g., corrections, updates)?
  • Externals
    • Reorganizing and using OBF in SCons
    • Moving to ROOT v5.20
    • Python
      • scipy and lapack and pointlike
  • Wired
    • Talk to Tony about remaining speed and memory issues
    • Can WIRED accomodate the HepRepSvc instance tree? (Needed for current implementation of merit in WIRED, TkrCluster, etc.)
  • Skimmer
    • Discuss outstanding JIRAs and future plans
    • Command-line skimmer issues (mainly, excessive memory use)
  • FluxSvc HEAD chat with Toby and Tracy
  • Overlays