Page History

...

• Analyze
  • There is a group of analysis scripts, that pick up zipped data; do their analysis, aggregate data and prepare web reports.
  • The first script to be executed is analyze-hourly.pl which takes as input data the output of getdata.pl and from this aggregaytes the data to by day and writes the latest  to the /nfs/slac/g/net/pinger/pingerreports/hep/<metric>/ directory with the file name <metric><size><by><yyyy><mm>-<dd>.txt.gz. The analyze-hourly.pl script is run daily from the trscrontab on pinger and by default analyzes the data gathered for yesterday. 
    Example output filename:

    Code Block
    /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2006-09-28.txt.gz

    By default the above file is created once thus the directory appears as:

    Code Block

    57cottrell@pinger:~>ls -l /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05* -rw-rw-r-- 1 pinger iepm 492144 May 2 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-01.txt.gz -rw-rw-r-- 1 pinger iepm 545968 May 3 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-02.txt.gz -rw-rw-r-- 1 pinger iepm 561661 May 4 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-03.txt.gz -rw-rw-r-- 1 pinger iepm 566550 May 5 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-04.txt.gz -rw-rw-r-- 1 pinger iepm 537127 May 6 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-05.txt.gz -rw-rw-r-- 1 pinger iepm 538830 May 7 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-06.txt.gz -rw-rw-r-- 1 pinger iepm 488360 May 8 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-07.txt.gz -rw-rw-r-- 1 pinger iepm 499020 May 9 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-08.txt.gz -rw-rw-r-- 1 pinger iepm 563840 May 10 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-09.txt.gz -rw-rw-r-- 1 pinger iepm 583454 May 11 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-10.txt.gz -rw-rw-r-- 1 cottrell iepm 577949 May 12 22:08 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-11.txt.gz -rw-rw-r-- 1 cottrell iepm 102 May 12 17:25 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-12.txt.gz

    Example output format (there are 24 hourly lines like this per day/per host pair):

    Code Block
    icfamon.dl.ac.uk lns62.lns.cornell.edu 108.871 ... 109.620 icfamon.dl.ac.uk lns62.lns.cornell.edu

  • The remaining analyze scripts (analyze-daily.pl, analyze-monthly.pl, analyze-allmonths.pl, and analyze-allyears.pl) take as input the data from analyze-hourly.pl, analyze-daily.pl, analyze-daily.pl, and analyze-allmonthly.pl and create files of the form:  

    Code Block
    /nfs/slac/g/net/pinger/pingerreports/hep/<metric>-<size>-by-<site|node>(-<YYYY>?)(-<mm>?)(-<dd>?).txt.gz

  • More information can be found on running getdata.pl and the analyze scripts to recover missing data.
• PRM
  • /afs/slac/package/pinger/prm/prm.pl
  • The script prm.pl, is used to create csv files for the analyzed data which can then be used by PingER motion charts or PingER Executive Plots.
  • The script requires a configuration file, which contains entries for all the reports prm should create. These entries are of the form:

    Code Block
    <metric name> <monitoring site> <country||continent> <tick> (<filter>?)

  • It gets its data from pingtable.pl either by calling it locally or by accessing it via the web.
  • The reports created by prm are available online.
  • The reports are used by the metric motion plots, the metric maps and the executive plots.

Data

/nfs/slac/g/net/pinger/pinger_mon_data/ping-<YYYY><MM>.txt contains data from 2005-2010./nfs/slac/g/net/pinger/pinger2/data/ping<YYYY>-<MM>.txt contains data from 2009-2012./nfs/slac/g/net/pinger/pingerdata/ contains data from 1997-2003/nfs/slac/g/net/pinger/pingerdata/hep/data/ contains data 1997-2007

Anomalies

We have spotted anomalies between the values reported by:

...