OpsProb Summary

Monitoring

Comparing Histograms

It is not currently possible to compare the same histogram from different runs

Login

Login in GLAST data portal

One user was not registered in the glast user database, several users had missing SLAC username's

Fixed

Tomcat

alarms summary doesn't work

Connection broken between IIS and server. Not a crash

The tomcat server was restarted. A page has been created for tracking similar outages http://confluence.slac.stanford.edu/x/qQFV

Data Monitoring

Alarm summary still not working

Same as above

ASP

overlapping bars in ASP flares

A data representation issue.

Fixed

Glast Ground

Menu item for shifts had not been updated since Ops Sim 1

Fixed

Data Monitoring

Problems with Alarm documentation

No readout errors in MC

Data Access

skimming on single runids gives bad files

It is currently not easy to skim OpsSim2 data because of multiple versions of the same files.

Data Access

Problems using download manager with CAL files

Cal files are getting negative file sizes. Download managed chokes on these.

Several JIRA's created: GDCB-49@JIRA DATASERV-104@JIRA

Data Monitoring

plot not showing the value that triggered the alert?

Trending plots are accumulated by default. Usability issue.

Data Monitoring

slow making plots

Big data request, many users, db overloaded

Data Monitoring

Data processing files inaccessible

Connection broken between IIS and server. Not a crash.

The server was restarted and the error recorded at http://confluence.slac.stanford.edu/x/qQFV

Data Monitoring

Data Quality Monitoring Page stalled

Same as above.

Data Monitoring

missing plots

Not sure which plots

Data Monitoring

Monitoring seems to be down again...

Due to missing data

GDM-28@JIRA

ASP

Problem with GCN notices

Documentation

Who is on shift in Confluence

Link in confluence page was broken

Fixed

ASP Data Viewer

ASP Start time and End time

Bug

Fixed

ASP Data Viewer

Can not access page directly

User had to login. Usability issue.

SM-28@JIRA

Data Monitoring

TowerCalLayerCalColumnCalXFace plots not being displayed

Some users cannot see plots.

Under investigation.

ASP

Source Monitoring

User was using wrong tool

User instructed to use http://glast-ground.slac.stanford.edu/ASPDataViewer SM-20@JIRA

Pipeline

GLAST PIpeline II PAge

Request to make it easier to identfy failed runs

JIRA created: PFE-156@JIRA

Data Monitoring

Missing plots

Recon_ReconAcdPhaMips_PMTA_Zoom_TH1_AcdTile metadata type was incorrect (seems to have changed during the day?)

Fixed

Data Monitoring

Ingestion problem

Some data not getting ingested. Some data seems duplicated.

Under investigation.

Data Monitoring

Processing errors

Caused by problems in trending ingestion code

Fixed?

JIRA

Exception on JIRA homepage

"Bad file descriptor" in Lucene search engine

JIRA (and confluence) server was restarted

Data Monitoring

Trending Configuration "Max Number of bins"

Setting Max Number of bins=10 has no effect

L1Proc

Fast mon crash for run 258280320

L1Proc

Processing error for run 258292096

bug in gcrCalib that occasionally makes recon segfault.

Fixed by rollback

Logging

Logs for 80306007

Log reports were missing from data processing page after processing complete

The application that posts messages to the database from the central message file was having a problem and had to be restarted

Cross Trending

Scatter plots

Scatter plots for the cross trending are not always displayed

Under investigation

Data Access

Problem accessing root files from noric

Fixed by using root 5.18 ?

Pipeline

pipeline mail processing

Mail delivery to pipeline server sometimes slow

Under investigation

Data Monitoring

Incorrect/Slow plot display

A bug in selection of histograms for vector quantities has been found and fixed (but not deployed). Some progress has been made on understanding poor performance of plot displays, but problem remains under investigation