July 19 2012- Astroserver and run quality
- Hardware changes (Tom)
- Update on NFS disk space upgrade
- Update on glastlnx server replacement
June 14 2012- Astroserver and run quality
- Hardware changes (Tom)
- Update on NFS disk space upgrade
- Update on glastlnx server replacement
May 10 2012- Hardware
- The plan to replace the sulkys.
- Ideas for replacing the glastlnx machines.
- How should we organize ourselves for planned computing outages?
- Astroserver and run quality
- Miscellanea
- Can we improve communication for pipeline work?
- New JIRA project for Fermi Data Handling
- Problems with the data catalog (see jira)
- Any idea what happened with robots.txt? do we need to password-protect the pipeline web interface?
Upcoming meetingsWe have now switched to a approximately bi-weekly schedule. The next meetings will be on: September 18 2009- PROD Oracle outage - 0900 9/24
- Disk tape usage
- Pipeline
August 14 2009- Status of L1 switch/reprocessing/astroserver
- Pipeline
- Missing foreign keys
- Pipeline web preferences issues
- Security patches – Tomcat servers switching to
- Tomcat 5.5.27
- Java 1.6.0_15
- Tony's vacation
- Ernest expected back on Monday – so use confluence-admin mailing list for confluence/jira problems
July 31 2009- Plans for release of public data
- AOB
June 26 2009Note change of time 9am pacific time - Astro server update
- Reprocessing status
- Pass 6 vs Pass 7 and consequences on astro server etc.
- Reprocessing and Run Details discussion
June 12 2009- Gliffy demo
- Update on wain020/wain OS upgrade
- Pass 7 related issues
- Pipeline requests
May 29 2009- Google IO highlights
- Data Catalog contents cleanup
- Post mortem of u18 problems from yesterday morning
- Status
May 8 2009April 24 2009 April 17 2009 (special reprocessing meeting)- Discussion of file naming, file location, and dataCatalog registration
April 10 2009- Status of astro server upgrade
- Planned new features
- New database using HTM/partitions/new hardware
- Decide between MySQL and Oracle
- Handling of new p7 quanitities
- Handling of new format FT1 and FT2 files
- Adding availability of LS1 and 1-second FT2 files
- Using new database hardware/software
- Post-processing with gtmktime
- Support for electron+positron files (perhaps)
- Goal is to be ready for end-of-april pass 7 release
- Need to start importing data from reprocessing by end-of-next-week
- Maven2 migration
- Many projects now being switched to maven2/hudson/netbeans 6.5.1
- New "archetype" makes it easy to start new projects
- Karen is first guinea pig with "Speakers bureau app"
- Karen has been making notes
- Crawler update
- Understood problem with
- New merit header
- First use of RootObj<Int> – escaping <> was not handled properly
- Recon files from L1 1.71
- FileHeader object is actually missing (just a RootObject)
- New crawler release coming
- Will mark recon files as "HeaderError" but we can then change it to OK
- Will recrawl all problem merit files
- Will extract merit version from header
- Will no longer use glast-rdr-test
- User disk web access
- Pipeline status
- Speakers bureau app
- Access to ops products
Mar 13 2009Feb 13 2009Jan 30 2009- Status of Pipeline upgrade
- Latest version of pipeline server now in DEV
- Latest version of pipeline front-end on tomcat03
- Confluence upgrade
- Astro server upgrade
- xrootd update
Jan 16 2009- Pipeline - new features demo
- Database password change warning
- Binary flash demo
- Status/Updates/AOB
Dec 12 2008Nov 21 2008- Highlights from
- Status reports/updates
- Release Manager II
- Resource Tracking
- Pipeline Release
- Improvements to RunQuality plugin (for Warren)
- Enhanced ability to send vars/file to batch jobs? (for Tom)
- Pipeline Front-End
- Remove use of anonymous FTP for file/log access
- Cleanup of data products
- Image Removed
- xrootd issues
- Data catalog improvements
- Download manager release
- Astro server release
- Merit Skimmer release
- Leap Second fixes
- Google Sky interface
- Confluence/JIRA release
- Group Manager release
Oct 31 2008Oct 24 2008- Upcoming outages
- Leap seconds planning
- There will be an new leap second at end of 2008
- Rob wants a list of all applications effected and whether we need to do anything to fix/test them
- Astro Server
- Disk/Oracle space
- I plan to turn on purging of L1 files older than 2 weeks today
Some plots from: http://glast-tomcat03.slac.stanford.edu:8080/xrootdiskspace-web/index.jsp Image Removed Image Removed Image Removed Oct 10 2008Sep 26 2008- Database cleanup status
- Pipeline 1.3.1 release planned for next week
- Pipeline performance continues to be good
Image Removed Image Removed - Contingency ISOC planning
- Especially as it may impact web tools
- xrootd tape backup plans
- New tape group mappings
- /glast/MC -> glastMC
- /glast/Data/Flight -> glastflight
- /glast/* -> glast
- Files to be backed up
- Cron job now writing list to ~glast/backupfiles/level1backupfiles.txt (7128 files)
No Format |
---|
select filesizebytes,path
from datasetgroup g
join verdataset d on (d.datasetgroup = g.datasetgroup)
join datasetversion dv on (d.latestversion=dv.datasetversion)
join verdatasetlocation l on (dv.masterlocation = l.datasetlocation)
join run r on (r.runid=l.runmin)
where g.datasetlogicalfolder=39684091 and l.datasetsite='SLAC_XROOT'
and (l1procstatus in ('Complete' , 'Incomplete') or sysdate-lastModified>'7 00:00:00')
and datasetdatatype in ('DIGI', 'RECON', 'MERIT', 'SVAC', 'GCR')
order by datasetdatatype, runmin;
|
- Wilko leaves Oct 6, back Nov 5. Has been working with Tofigh, so Tofigh can help provide emergency support.
Sep 5, 2008- Pipeline release plans
- Reprocessing readiness
- xrootd update
- Miscellaneous
- Apps being moved from tomcat07 (glastlnx09) to tomcat06 (glastlnx13).
- This will cause some nagios messages until nagios monitoring is updated
- This is to free memory on glastlnx09 for astro server database
- confluence 2.9 is now set up for testing at http://confluence-new.slac.stanford.edu
- jira-new coming soon
8/8/08- Infrastructure Shifts
- Recent/Upcoming release plans
- Nagios overview - Emmanuel
Aug 1, 2008Now that the rush to get things ready for LEO is subsiding we should go over what things didn't get done before, and what needs to be improved, cleaned up, etc. - Near term plans for
- Pipeline
- Data catalog
- Linemode download manager
- Ability to delete files from data catalog
- Ability to purge old versions of (L1) files
- In both cases data catalog will mark files to be deleted, then crawler will actually delete the files (from xrootd)
- Skimmer
- Upgrade web front-end to use v6 backend
- Understand issues with skimming real data
- Miscellaneous improvements
- Xrootd
Jul 17, 2008- Reprocessing discussion
- Line-mode datacat - new features
- Infrastructure shifts discussion
Jun 20, 2008Jun 13, 2008- Final decisions about infrastructure shifts
- Request from Ian/Arash for oracle outage
- Updates
- AOB
May 30, 2008- Run Quality Update
- Database/Performance issues after pipeline upgrade
- AOB
May 23, 2008- Nagios Overview/Status
- xrootd update
- Run Quality Update
- runQuality object to be added to scriptlet environment
- runQuality.setL1RunStatus(long run, String status)
- One of Running, Complete, InComplete, InProgress, Running, Unset
- runQuality.setReviewStatus(long run, String status)
- One of Not ready, Waiting review, (Reviewed, Unset)
- Datacat versions update
- AOB
May 16, 2008May 9, 2008- Data Catalog line-mode client update
- Xrootd Update (New fileserver, new xrootd release, glastlnx22, etc)
- AOB
May 2, 2008- Stress test update
- Pipeline II progress, release plans
- Xrootd Update (New fileserver, new xrootd release, etc)
- AOB
April 18, 2008- Launch update
- Stress test issues
- Run Catalog Discussion
- Questions
- Who registers the run?
- What is the relationship to Bryson's downlink tables
- How do we know if run is complete?
- Are runs assigned a status before the run is complete?
- Is rerunning a top level stream a "reprocessing"
- Can original run status change as a result of reprocessing
- If so do we need a history?
- What should the web interface look like
- Initial Schema proposal
- xrootd status
April 11, 2008- xrootd tape achiving status
- AOB
April 4, 2008- Oracle Database Update
- Pipeline II release
- Infrastructure "How-to-fix" documentation
- Expert Shifts
- AOB
March 28, 2008- Oracle Database Update
- Pipeline Update
- Upcoming CCB action requests
- Infrastructure "Expert Shifts"
March 21, 2008- Tomcat servers update
- Oracle
- Oracle server/stress testing update
- Oracle wallet/roles/materialized views discussion
- Data catalog
- Recent CCB action requests
- Upcoming CCB action requests
- Pipeline client rollback fix (1.0.2)
- Pipeline front-end fixes (message viewer etc) (2.5.1)
- Group manager fix to import data from campus MySQL (1.7) and export list of users for xrootd
- Put oracle wallet into production for Pipeline Server (1.0.1), Pipeline Client (1.0.3?), Datacat client (1.1.1)
February 29, 2008- Fallout from yesterday's outage
- Pipeline deadlock update
- xrootd update
- Preparation for opssim2
- Plans for Change Control Board (CCB)
- Oracle passwords
- ASP web display update
February 22, 2008February 15, 2008- Shift Database/Web app
- Using AFS tokens to log in to glast account (postponed to later)
- New Crawler
- OpsSim2 readiness issues
- xroot and old file versions
- Schedule for OS/IP upgrade on glastlnx*
- Effects of elimination of services
February 1, 2008 January 18, 2008 January 11, 2008 |