Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Area

Problem

Comments

Resolution

Monitoring

Problem loading alarm files : javax.servlet.jsp.el.ELException: An error
occurred while evaluating function "dmapp:getAlarms"

?

?

Crawler

Slow crawl? No data files for stress test runs.

Crawler had been reporting all files as missing all day. All files were recrawled and then appeared to be OK.

?

Monitoring

Monitoring data was being sent to DEV

Application was poked and then seemed to work

?

L1

Recon EOR timed out in 80418003.257755648.15606093 with
/afs/slac/g/glast/isoc/flightOps/rhel3_gcc32/ISOC_PROD/bin/isoc: line 33:
/afs/slac/g/glast/isoc/flightOps/volumes/vol5/isoc_rpm/rhel3_gcc32_install_20080227/etc/rpmenv.sh:
Connection timed out Are we overwhelming isoc/flightOps afs?

?

?

Monitoring

DQM stuck

?

?

Pipeline

Mail backlog

Processing of e-mail from L1Proc is slow. This seems to be caused by contention for locking the top level stream (a problem not seen in the much simpler MC task we used for testing)

PII-306@JIRA PII-305@JIRA PII-318@JIRA

...

Area

Problem

Comments

Resolution

Tomcat

Web applications were all down

tomcat01 had run out of permgen space

GG-23@JIRA


10:00am (Daniel Flath):

Wiki Markup
The new stored procedures (See April 21 note) came with new queries.  Until Oracle has run the new queries long enough to gather execution-plan statistics, it cannot recommend a more efficient execution-plan.  Two of the queries that are most heavily executed during dependency evaluation were identified by the \[Grid Control\] top SQL activity report.  Oracle is recommending execution-plan improvements for both.  The revised plan estimates indicate that we may benefit significantly by implementing the changes.

 I have implemented the explain plans, shown here, and Warren will submit another run to L1 to see if the changes made a difference, and then iterate over this process again if there are further recommendations from Oracle.