Page History

...

• Analyze
  • There is a group of analysis scripts, that pick up zipped data; do their analysis, aggregate data and prepare web reports.

  • The first script to be executed is wrap-analyze-hourly.pl which takes as input data the output of getdata.pl and from this aggregates the data to by day and writes the latest  to the /nfs/slac/g/net/pinger/pingerreports/hep/<metric>/ directory with the file name <metric><size><by><yyyy><mm>-<dd>.txt.gz. The analyze-hourly.pl script is run daily from the trscrontab on pinger and by default analyzes the data gathered for yesterday. 
    Example output filename for the minimum_rtt metric:

    Code Block
    /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2006-09-28.txt.gz

    By default the above file is created once thus the directory appears as:

    Code Block

    57cottrell@pinger:~>ls -l /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05* -rw-rw-r-- 1 pinger iepm 492144 May 2 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-01.txt.gz -rw-rw-r-- 1 pinger iepm 545968 May 3 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-02.txt.gz -rw-rw-r-- 1 pinger iepm 561661 May 4 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-03.txt.gz -rw-rw-r-- 1 pinger iepm 566550 May 5 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-04.txt.gz -rw-rw-r-- 1 pinger iepm 537127 May 6 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-05.txt.gz -rw-rw-r-- 1 pinger iepm 538830 May 7 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-06.txt.gz -rw-rw-r-- 1 pinger iepm 488360 May 8 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-07.txt.gz -rw-rw-r-- 1 pinger iepm 499020 May 9 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-08.txt.gz -rw-rw-r-- 1 pinger iepm 563840 May 10 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-09.txt.gz -rw-rw-r-- 1 pinger iepm 583454 May 11 02:17 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-10.txt.gz -rw-rw-r-- 1 cottrell iepm 577949 May 12 22:08 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-11.txt.gz -rw-rw-r-- 1 cottrell iepm 102 May 12 17:25 /nfs/slac/g/net/pinger/pingerreports/hep/minimum_rtt/minimum_rtt-100-by-node-2011-05-12.txt.gz

    Example output format. There Following the 1st line in the file there is 1 line like the following per day/per host pair.  Between the initial and final src_name and tgt_name tokens there are 24 tokens one for each hour of the day, missing data is identified by a dot followed by a space (. ), e.g.:

    Code Block
    icfamon.dl.ac.uk lns62.lns.cornell.edu 108.871 . . . . . . . . 108.892 . . . . . . . . . . . . . 109.620 icfamon.dl.ac.uk lns62.lns.cornell.edu

    
    The first line in the file contains a label for each of the time slots (e.g. hours):

    Code Block
    0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

  • The remaining analyze scripts (wrap-analyze-daily.pl, wrap-analyze-monthly.pl, wrap-analyze-allmonths.pl, and wrap-analyze-allyears.pl) take as input the data from wrap-analyze-hourly.pl, wrap-analyze-daily.pl, and wrap-analyze-allmonths.pl and create files of the form:  

    Code Block
    /nfs/slac/g/net/pinger/pingerreports/hep/<metric>-<size>-by-<site|node>(-<YYYY>?)(-<mm>?)(-<dd>?).txt.gz /nfs/slac/g/net/pinger/pingerreports/hep/<metric>-<size>-by-<site|node>-<60|120|365>days.txt.gz /nfs/slac/g/net/pinger/pingerreports/hep/<metric>-<size>-by-<site|node>-<allmonths|allyears>.txt.gz

  • There are ~ 16 metrics:

    <option value="MOS">Mean Opinion Score</option>

    <option value="alpha">Directivity</option>

    <option value="average_rtt" selected>Average Round Trip Time</option>

    <option value="conditional_loss_probability">Conditional Loss Probability</option>

    <option value="duplicate_packets">Duplicate Packets</option>

    <option value="ipdv">Inter-Packet Delay Variation</option>

    <option value="iqr">Inter-Quartile Range</option>

    <option value="maximum_rtt">Maximum Round Trip Time</option>

    <option value="minimum_packet_loss">Minimum Packet Loss</option>

    <option value="minimum_rtt">Minimum Round Trip Time</option>

    <option value="out_of_order_packets">Out of Order Packets</option>

    <option value="packet_loss">Packet Loss</option>

    <option value="throughput">TCP Throughput (kbits/s)</option>

    <option value="unpredictability">Ping Unpredictability</option>

    <option value="unreachability">Ping Unreachability</option>

    <option value="zero_packet_loss_frequency">Zero Packet Loss Frequency</option>

    Information on these can be found at http://www.slac.stanford.edu/comp/net/wan-mon/tutorial.html.

     

  • More information can be found on running getdata.pl and the analyze scripts to recover missing data.
• PRM
  • /afs/slac/package/pinger/prm/prm.pl
  • The script prm.pl, is used to create csv files for the analyzed data which can then be used by PingER motion charts or PingER Executive Plots.

  • The script requires a configuration file, which contains entries for all the reports prm should create. These entries are of the form:

    Code Block
    <metric name> <monitoring site> <country||continent> <tick> (<filter>?)

  • It gets its data from pingtable.pl either by calling it locally or by accessing it via the web.
  • The reports created by prm are available online.
  • The reports are used by the metric motion plots, the metric maps and the executive plots.

...