Data Handling Weekly Meetings - 2009

Meetings are held via teleconference on Fridays, 10:30 Pacific time using the CalTech EVO system. We do not use video.
Please avoid an open mike, which may cause echoes. See the evo help.  For further information be sure to have a look at the VRVS Etiquette page.

Previous Years

• 2008
• 2007
• 2006
• 2005

October 16 2009

• Data Catalog update
  • Adding purge capability to line mode client
• AstroServer update
  • Splitting large output files to avoid out-of-memory problems
• XRoot/HPSS update Testing HPSS Prod2

September 18 2009

• PROD Oracle outage - 0900 9/24
• AFS token problems in batch
• Disk usage/tape backup
  • Level 1 Usage
  • Hpss usage
• Pipeline
  • Memory leak progress
    • Reclaiming Connections (image)
    • Reclaiming Connections (DEV message viewer)
  • Feature requests from L1

August 14 2009

• Status of L1 switch/reprocessing/astroserver
  • Dataset definitions
  • Astro Server
• Pipeline
  • Missing foreign keys
  • Pipeline web preferences issues
• Security patches - Tomcat servers switching to
  • Tomcat 5.5.27
  • Java 1.6.0_15
• Tony's vacation
  • Ernest expected back on Monday - so use confluence-admin mailing list for confluence/jira problems

July 31 2009

• Plans for release of public data
  • L1 switch to new version
  • Finalizing P105 reprocessing
  • Switch to new version of AstroServer
  • Names for data products
• AOB

June 26 2009

Note change of time 9am pacific time

• Astro server update
• Reprocessing status
  • Pass 6 vs Pass 7 and consequences on astro server etc.
• Reprocessing and Run Details discussion

June 12 2009

• Gliffy demo
• Update on wain020/wain OS upgrade
• Pass 7 related issues
  • Skimmer status
  • Astro server status
  • Left over reprocessing issues?
    • P100-MERIT and P100-FT1 tasks
• Pipeline requests

May 29 2009

• Google IO highlights
• Data Catalog contents cleanup
  • New problems page
• Post mortem of u18 problems from yesterday morning
• Status
  • Reprocessing
    • http://glast-ground.slac.stanford.edu/Pipeline-II/task.jsp?task=19821716
    • http://ganglia01.slac.stanford.edu:8080/ganglia/glast/?r=hour&s=descending&c=glast-xrootd
  • Xrootd/HPSS

May 8 2009

• Maven2 conversion
  • Project list
• Reprocessing readiness/status
  • Processing History tables and web pages
  • P100 output
  • Astro server readiness
  • P100 config
• SCons installer demo
• Speakers bureau web interface demo

April 24 2009

• Heads Up - Prod Oracle outage 4/29 - 2pm
• Astro Server performance update
• New glast-ground (aka fermi-ground)
• Reprocessing readiness/status
  • New meta-data in data catalog
  • Run History Web Interface
  • Data type for filtered Merit files

April 17 2009 (special reprocessing meeting)

• Discussion of file naming, file location, and dataCatalog registration
  • Pass7 reprocessing notes from Tom P100 (Pass7) Reprocessing Notes

April 10 2009

• Status of astro server upgrade
  • Planned new features
    • New database using HTM/partitions/new hardware
    • Decide between MySQL and Oracle
    • Handling of new p7 quanitities
    • Handling of new format FT1 and FT2 files
    • Adding availability of LS1 and 1-second FT2 files
    • Using new database hardware/software
    • Post-processing with gtmktime
    • Support for electron+positron files (perhaps)
  • Goal is to be ready for end-of-april pass 7 release
  • Need to start importing data from reprocessing by end-of-next-week
• Maven2 migration
  • Many projects now being switched to maven2/hudson/netbeans 6.5.1
  • New "archetype" makes it easy to start new projects
    • Karen is first guinea pig with "Speakers bureau app"
      • Karen has been making notes
• Crawler update
  • Understood problem with
    • New merit header
      • First use of RootObj<Int> - escaping <> was not handled properly
    • Recon files from L1 1.71
      • FileHeader object is actually missing (just a RootObject)
  • New crawler release coming
    • Will mark recon files as "HeaderError" but we can then change it to OK
    • Will recrawl all problem merit files
    • Will extract merit version from header
    • Will no longer use glast-rdr-test
• User disk web access
  • u33
• Pipeline status
  • New web front-end which does not use FTP
• Speakers bureau app
• Access to ops products

Mar 13 2009

• Glast (etc) passwords
• Oracle connections on machines up for >200 days
• Trending database disk usage statistics
• xrootd upgrade CCB
  • xrootd outage
• Data Access Issues
  • New astro server status
  • Outstanding Bugs
    • LONE-131@jira
    • DATASERV-151@jira
  • Standard data sets
  • Dealing with soon to arrive "bad" data
  • Reprocessing timeline

Feb 13 2009

• Confluence notes
  • New "Confluence Tips and Tricks", in particular hints on organizing confluence pages
    • See for example Data Handling Space
• Data Processing Stats
• Disk space update
  • Available disk space
    • New page lists "to be deleted" files
• Maven2 and Hudson status/demo
  • Documentation
    • For Developers
    • Generated
• Pipeline Status
• Party Time

Jan 30 2009

• Status of Pipeline upgrade
  • Latest version of pipeline server now in DEV
  • Latest version of pipeline front-end on tomcat03
    • Status
• Confluence upgrade
• Astro server upgrade
• xrootd update

Jan 16 2009

• Pipeline - new features demo
• Database password change warning
• Binary flash demo
• Status/Updates/AOB