Introduction

This page describes, how the data collected by the SLAC PingER site, ends up as information, which is used by pingtable, motion charts, intensity maps and several other applications. Various scripts are used in turn to generate the data. The scripts in their turn are driven by a trscrontab that executes on pinger@pinger.slac.stanford.edu.

Scripts

• TimePing:** /afs/slac/package/pinger/timeping.pl
  • Represents the old script to perform ping measurements and store raw data.
  • Stores raw data on nfs at:

    /nfs/slac/g/net/pinger/pinger_mon_data/ping-<YYYY>-<MM>.txt

• PingER2:
  • /afs/slac/package/pinger/pinger2/share/pinger/pinger2.pl
  • Represents the new Tool to perform ping measurements and store raw data.

  • [http://confluence.slac.stanford.edu/display/IEPM/PingER2+at+SLAC]

  • Stores raw data on nfs at:

    /nfs/slac/g/net/pinger/pinger2/data/ping-<YYYY>-<MM>.txt
    e.g.
    /nfs/slac/g/net/pinger/pinger2/data/ping-2011-02.txt

• Ping_Data
  • /afs/slac/g/www/cgi-wrap-bin/net/offsite_mon/ping_data/ping_data.pl
  • Represents the web interface(cgi script) used to fetch data from remote monitoring node.
  • zips the data before transfer.
• GetData
  • /afs/slac/package/pinger/getdata.pl

  • queries ping_data[http://slac.stanford.edu/cgi-wrap/ping_data.pl] to fetch data.

  • Stores old files as zipped File at

    /nfs/slac/g/net/pinger/pingerdata/hep/data/<host>/ping-<YYYY>-<MMM>.txt.gz

  • See Restoring PingER data for more details.
• Analyze
  • There is a group of analysis scripts, that pick up zipped data; do their analysis, aggregate data and prepare web reports.
  • The first script to be executed is analyze-hourly.pl which takes as input data the output of getdata.pl and from this aggregaytes the data to by day and writes the latest  to the /nfs/slac/g/net/pinger/pingerreports/hep/<metric>/ directory with the file name <metric><size><by><yyyy><mm>-<dd>.txt.gz,
    Example output filename:

    /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2006-09-28.txt.gz
    

    Example output format (there are 24 hourly lines like this per day/per host pair):

    icfamon.dl.ac.uk lns62.lns.cornell.edu 108.871 ... 109.620 icfamon.dl.ac.uk lns62.lns.cornell.edu
    

  • The remaining analyze scripts (analyze-daily.pl, analyze-monthly.pl, analyze-allmonths.pl, and analyze-allyears.pl) take as input the data from analyze-hourly.pl ancd create files of the form:  

    /nfs/slac/g/net/pinger/pingerreports/hep/<metric>-<size>-by-<site|node>(-<YYYY>?)(-<mm>?)(-<dd>?).txt.gz

  • More information can be found on running getdata.pl and the analyze scripts to recover missing data.

IMPROTANT:    /nfs/slac/g/net/pinger/pingerreports/hep/<metric> contains the  recent years data however /nfs/slac/g/pinger/pingerdata/hep/data contains old data(from 1998 to 2007) for nodes.

• PRM
  • /afs/slac/package/pinger/prm/prm.pl
  • The script prm.pl, is used to create csv files for the analyzed data which can then be used by PingER motion charts or PingER Executive Plots.
  • The script requires a configuration file, which contains entries for all the reports prm should create. These entries are of the form:

    <metric name>    <monitoring site>    <country||continent>    <tick>    (<filter>?)

  • It gets its data from pingtable.pl either by calling it locally or by accessing it via the web.
  • The reports created by prm are available online.
  • The reports are used by the metric motion plots, the metric maps and the executive plots.

Anomalies

We have spotted anomalies between the values reported by:

• table.pl and pingtable.pl
• and pingtable.pl at SLAC and pingtable.pl at SEECS

They are discussed and explained in the Anomalies report.