Motivation

The PingER metrics motion charts are primarily used to visualize the trends in the Internet end-to-end performance statistics measured to over 160 countries from the 40+ PingER monitoring nodes spread worldwide. Having gathered data since 1998, the charts enable the users to study the trends, step changes, significant improvements/degradations with the help of these 4-dimensional charts. The charts are based on Google Motion Charts.

The different sets of charts (w.r.t. regions) enables the users to study the progress made by countries in comparison to their neighbors as well as the world in general.

Below are some examples:

Observations made from Stanford Linear Accelerator Center (SLAC), USA in 2008.

View of the world
from SLAC in Aug 2008
(min RTT vs norm. throughput
& Internet penetration)

View of the world
from SLAC in Sep 2007
(min RTT vs jitter, HDI
& Internet users)

View of the world
from SLAC in 2008
(packet loss vs min RTT,
region & throughput)

Observations made from SLAC 2019 

View of world from SLAC June 2019 (Norm throughput against min RTT, bubble size=population)View of World from SLAC June 2019 (IDI index vs Packet loss)View of world from SLAC June 2019 (Norm throughput vs IDI, color=minimum RTT) 
 

 

Observations made from CERN, Geneva, Switzerland:

View of South Asia
from CERN in Aug 2008
(avg RTT vs norm. throughput, 
packet loss & Internet
penetration)

View of the Balkans
from CERN in Sep 2008
(min RTT vs jitter,
norm. throughput &
Internet users)

View of Africa
from CERN in Sep 2008
(min RTT vs norm. throughput,
Internet penetration &
unreachability)

 

Usage (also see the video)

By default, PingER metrics motion chart shows a motion scatter/bubble plot of the ping (100 Byte) average RTT in msec. ( x ) versus the normalized derived TCP throughput in kbits/sec ( y ). Each bubble represents a country. By default the bubbles are colored by region and their sizes are proportional to the Internet users. The motion is the time. The time granularity is by year. Click on the play button to initiate the motion.

  • To change the metric represented by the size of the bubble use the 'size' combo box to the right of the chart.
  • Click on the pull down box of an axis' label to change the metric that is displayed on an axis or in a table. Other PingER metrics that are available include minimum RTT, jitter and unreachability.
  • Move the mouse over a bubble to identify the values for the bubble.
  • Linear or logarithmic scales may be selected using the combo box along the axis.
  • Move the mouse over a coloured legend symbol to blink the balloons associated with the symbol's color.
  • To follow the changes in detail simply move the slider bar with the mouse.
  • One can select a bubble by clicking on it to provide a label for the bubble that persists with the motion. The trails check box leaves a trail to follow the motion of selected bubbles. N.B. selected bubbles determine how far the data to be displayed goes back in time.
  • By clicking the links at the right top of the page, one may select the vantage point, the observed region and the granularity of the measurements.

Implementation Details

HTML output

To create a motion chart, the data must be populated in a data structure (as stated by Google Visualization API and copied below):

<html>
  <head>
    <script type="text/javascript" src="http://www.google.com/jsapi"></script>
    <script type="text/javascript">
      google.load("visualization", "1", {packages:["motionchart"]});
      google.setOnLoadCallback(drawChart);
      function drawChart() {
        var data = new google.visualization.DataTable();

              data.addColumn('string', 'Country');
              data.addColumn('date', 'Month');
              data.addColumn('string', 'Region');
              data.addColumn('number', 'Average RTT (ms)');
              data.addColumn('number', 'Normalized Throughput (Kbps)');
              data.addColumn('number', 'Internet Users (#)');
              data.addColumn('number', 'Internet Penetration (%)');
              data.addColumn('number', 'Population (#)');
              data.addColumn('number', 'Minimum RTT (ms)');
              data.addColumn('number', 'Packet Loss (%)');
              data.addColumn('number', 'Unreachability (%)');
              data.addColumn('number', 'Jitter (ms)');
              data.addColumn('number', 'DOI Index');
              data.addColumn('number', 'HDI Index');
 
              data.addRows(1);
              data.setValue(0, 0, "Pakistan");
              data.setValue(0, 1, new Date(2007,02,01));
              data.setValue(0, 2, 'South Asia');
              data.setValue(0, 3, 421.346);
              data.setValue(0, 4, 1568.95315647059);
              data.setValue(0, 5, 7500000);
              data.setValue(0, 6, 4.7);
              data.setValue(0, 7, 172800051);
              data.setValue(0, 8, 311.482);
              data.setValue(0, 9, 1.331);
              data.setValue(0, 10, 15.591);
              data.setValue(0, 11, 19.906);
              data.setValue(0, 12, 0.26);
              data.setValue(0, 13, 0.551);

              data.addRows(1);
              data.setValue(1, 0, "Sri Lanka");
              data.setValue(1, 1, new Date(2007,02,01));
              data.setValue(1, 2, 'South Asia');
              data.setValue(1, 3, 418.664);
              data.setValue(1, 4, 2760.35944717647);
              data.setValue(1, 5, 280000);
              data.setValue(1, 6, 1.4);
              data.setValue(1, 7, 21128773);
              data.setValue(1, 8, 270.494);
              data.setValue(1, 9, 0.304);
              data.setValue(1, 10, 8.871);
              data.setValue(1, 11, 42.093);
              data.setValue(1, 12, 0.33);
              data.setValue(1, 13, 0.743); 

 
              data.addRows(1);
              data.setValue(2, 0, "Pakistan");
              data.setValue(2, 1, new Date(2007,03,01));
              data.setValue(2, 2, 'South Asia');
              data.setValue(2, 3, 397.897);
              data.setValue(2, 4, 1394.21896911765);
              data.setValue(2, 5, 7500000);
              data.setValue(2, 6, 4.7);
              data.setValue(2, 7, 172800051);
              data.setValue(2, 8, 305.533);
              data.setValue(2, 9, 1.248);
              data.setValue(2, 10, 12.917);
              data.setValue(2, 11, 31.064);
              data.setValue(2, 12, 0.26);
              data.setValue(2, 13, 0.551);

              data.addRows(1);
              data.setValue(3, 0, "Sri Lanka");
              data.setValue(3, 1, new Date(2007,03,01));
              data.setValue(3, 2, 'South Asia');
              data.setValue(3, 3, 357.202);
              data.setValue(3, 4, 4634.87959694118);
              data.setValue(3, 5, 280000);
              data.setValue(3, 6, 1.4);
              data.setValue(3, 7, 21128773);
              data.setValue(3, 8, 274.092);
              data.setValue(3, 9, 0.144);
              data.setValue(3, 10, 36.319);
              data.setValue(3, 11, 11.889);
              data.setValue(3, 12, 0.33);
              data.setValue(3, 13, 0.743);

 
              data.addRows(1);
              data.setValue(4, 0, "Pakistan");
              data.setValue(4, 1, new Date(2007,04,01));
              data.setValue(4, 2, 'South Asia');
              data.setValue(4, 3, 372.313);
              data.setValue(4, 4, 1918.28198552422);
              data.setValue(4, 5, 7500000);
              data.setValue(4, 6, 4.7);
              data.setValue(4, 7, 172800051);
              data.setValue(4, 8, 310.145);
              data.setValue(4, 9, 0.969);
              data.setValue(4, 10, 13.844);
              data.setValue(4, 11, 24.146);
              data.setValue(4, 12, 0.26);
              data.setValue(4, 13, 0.551);

              data.addRows(1);
              data.setValue(5, 0, "Sri Lanka");
              data.setValue(5, 1, new Date(2007,04,01));
              data.setValue(5, 2, 'South Asia');
              data.setValue(5, 3, 397.434);
              data.setValue(5, 4, 2450.70159022405);
              data.setValue(5, 5, 280000);
              data.setValue(5, 6, 1.4);
              data.setValue(5, 7, 21128773);
              data.setValue(5, 8, 298.214);
              data.setValue(5, 9, 0.410);
              data.setValue(5, 10, 7.527);
              data.setValue(5, 11, 6.979);
              data.setValue(5, 12, 0.33);
              data.setValue(5, 13, 0.743);

        var chart = new google.visualization.MotionChart(document.getElementById('chart_div'));
        chart.draw(data, {width: 850, height:500});
      }
    </script>
  </head>

  <body>
    <div id="chart_div" style="width: 850px; height: 500px;"></div>
  </body>
</html>  

The data must comply with the formatting requirements (as stated by Google Visualization API and copied below):

  • The first column must be of type 'string' and contain the entity names (e.g., "Pakistan", "Sri Lanka" in the example above).
  • The second column must contain time values. Time can be expressed in a few different ways:
    • Years. Column type: 'number'. Example: 2008.
    • Month, day and year. Column type should be 'date' and the values should be javascript Date instances.
    • Week numbers. Column type should be 'string' and the values should have the pattern YYYYWww, which conforms to ISO 8601. Example: '2008W03'.
    • Quarters. Column type should be 'string' and the values should have the pattern YYYYQq, which conforms to ISO 8601. Example: '2008Q3'.
  • Subsequent columns can be of type 'number' or 'string'. Number columns will show up in the drop down menus for X, Y, Color and Size axes. String columns will only appear in the drop down menu for Color. 

Note that the data must be loaded in a chronologically ascending order. See the code above as an example. Also note that the order in which the columns are organized, defines the default metrics with which the axes are labeled. The first number column defines the label for the x-axis where as the second number column defines the default label for the y-axis. By changing the order of the number columns, the default labels for the axes can be changed.

Relevant files 

Following are the files involved in the process of generating the HTML page:

Application:

File path:/afs/slac/package/pinger/motion-chart/create-motion-chart-page.pl

The script that defines the input arguments to generate the relevant .html files:

File path: /afs/slac/package/pinger/motion-chart/create-motion-chart-pages.pl

Demographics:

File path: /afs/slac/package/pinger/motion-chart/demographics.csv
 
File format:
#Country, Year, Population, Internet Users, Internet Penetration, DOI, HDI
United States, 2008, 303824646, 213364918, 70.7, 0.62, 0.951

 PingER data (obtained from prmout) Average RTT, Jitter (IPDV), Minimum RTT, Normalized Throughput, Packet Loss and Unreachability as seen from CERN, SLAC and ICTP:

The following files are automatically updated on a daily basis by the pinger@pinger.slac.stanford.edu trscrontab.
File formats:
allyearly: #allyearly,?,1998,1999,2000,2001,2002,2003,2004,2005,2006,2007,2008
monthly: #monthly,?,Oct2008,Sep2008,Aug2008, ... Jan2007,Dec2006,Nov2006
last60days:  #last60days,?,08Oct27,08Oct26,08Oct25,08Oct24, ... 08Aug31,08Aug30,08Aug29

 
File paths:
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-CH.CERN.N2-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-CH.CERN.N2-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-CH.CERN.N2-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-EDU.SLAC.STANFORD.N3-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-EDU.SLAC.STANFORD.N3-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-EDU.SLAC.STANFORD.N3-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-IT.ICTP.N1-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-IT.ICTP.N1-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/average_rtt-IT.ICTP.N1-country-monthly.csv

/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-CH.CERN.N2-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-CH.CERN.N2-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-CH.CERN.N2-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-EDU.SLAC.STANFORD.PINGER-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-EDU.SLAC.STANFORD.PINGER-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-IT.ICTP.N1-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-IT.ICTP.N1-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/ipdv-IT.ICTP.N1-country-monthly.csv

/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-CH.CERN.N2-country-allyearly.cvs
/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-CH.CERN.N2-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-CH.CERN.N2-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-EDU.SLAC.STANFORD.PINGER-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-EDU.SLAC.STANFORD.PINGER-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-IT.ICTP.N1-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-IT.ICTP.N1-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/minimum_rtt-IT.ICTP.N1-country-monthly.csv

/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-CH.CERN.N2-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-CH.CERN.N2-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-CH.CERN.N2-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-IT.ICTP.N1-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-IT.ICTP.N1-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-IT.ICTP.N1-country-monthly.csv

/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-CH.CERN.N2-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-CH.CERN.N2-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-CH.CERN.N2-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-EDU.SLAC.STANFORD.PINGER-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-EDU.SLAC.STANFORD.PINGER-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-IT.ICTP.N1-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-IT.ICTP.N1-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/packet_loss-IT.ICTP.N1-country-monthly.csv

/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-CH.CERN.N2-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-CH.CERN.N2-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-CH.CERN.N2-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-EDU.SLAC.STANFORD.PINGER-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-EDU.SLAC.STANFORD.PINGER-country-monthly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-IT.ICTP.N1-country-allyearly.csv
/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-IT.ICTP.N1-country-last60days.csv
/afs/slac/g/www/www-iepm/pinger/prmout/unreachability-IT.ICTP.N1-country-monthly.csv

Country to region mapping:

The following file is automatically updated on a daily basis by the pinger@pinger.slac.stanford.edu trscrontab.
File path: /afs/slac.stanford.edu/www/comp/net/mon/countries.tsv
 
File format: Country Name        Region Name
Note that the country name and region name are separated by a tab.

#==================================================================================================
#This is file /afs/slac.stanford.edu/www/comp/net/mon/countries.tsv
#Current as of: Sat Jun 29 00:15:05 2019
#Written by Script: /afs/slac/package/pinger/write_country_list.
#Written on: lnxcron.slac.stanford.edu
#Executing Directory: /afs/slac.stanford.edu/u/sf/pinger
#User: pinger
#Usually run via trscron from pinger@pinger
#Data extracted from the Guthrie NODEDETAILS Oracle database.
#==================================================================================================
Afghanistan     South Asia
Albania Balkans
Algeria Africa
Andorra Europe
Angola  Africa
Argentina       Latin America
Armenia Central Asia
Australia       Oceania
Austria Europe
Azerbaijan      Central Asia
Bahamas Latin America
Bahrain Middle East
Bangladesh      South Asia
Belarus Europe
Belgium Europe
Benin   Africa
Bhutan  South Asia
Bolivia Latin America
Bosnia Herzegovina      Balkans
...

Implementation

 Following is the pseudo code for implementation:

1. Load the country to region mapping to the hash-table "%region" which uses the country
    names (as defined by PingER) as the index.
    e.g. $region{'Kenya'} = Africa and
           $region{'Korea, Rep'} = East Asia.

 
2. Load the PingER data to the data structure "%data" which uses PingER metrics as the index.

    Here each entry stores a reference to a unique hash-table defined to store the PingER data
    related to a country. Understandably the second hash-table uses country names as the index.
    Each entry of the hash-table stores a reference to another hash-table used for storing
    performance measurements for the country. Here timestamps are used as indices.
    e.g. $data{'minrtt'}->{'Kenya'}->{'1998'},
           $data{'packetloss'}->{'United States'}->{'Oct2008'} and
           $data{'ipdv'}->{'Pakistan'}->{'08Oct27'}
    The metrics used are: minrtt, avgrtt, nthroughput, ipdv, packetloss and unreachability.

    An important factor to consider here is the file format. The yearly data from prmout is listed
    in an ascending order from left to right, where as for monthly and daily data, the
    measurements are listed in a descending order. Thus while loading, the iterators need to
    be defined accordingly.

 
3. Load the demographics to the data structure "%data" which uses labels as the index.
    Here each entry stores a reference to a unique hash-table defined to store the demographics
    of a country. The second hash-table uses country names as the index. Each entry of
    the hash-table stores a reference to another hash-table used for storing statistics
    for the country. Here timestamps are used as indices.
    e.g. $data{'population'}->{'Kenya'}->{'1998'},
           $data{'doi'}->{'United States'}->{'Oct2008'} and
           $data{'inetpenetration'}->{'Pakistan'}->{'08Oct27'}

    The metrics used are: population, inetusers, inetpenetration, doi and hdi.

4. Generate the .html file in compliance with the Google Visualization API constraints.
    Here the data is loaded in the following format (code in Javascript):
              data.addColumn('string', 'Country');
              data.addColumn('date', 'Month');
              data.addColumn('string', 'Region');
              data.addColumn('number', 'Average RTT (ms)');
              data.addColumn('number', 'Normalized Throughput (Kbps)');
              data.addColumn('number', 'Internet Users (#)');
              data.addColumn('number', 'Internet Penetration (%)');
              data.addColumn('number', 'Population (#)');
              data.addColumn('number', 'Minimum RTT (ms)');
              data.addColumn('number', 'Packet Loss (%)');
              data.addColumn('number', 'Unreachability (%)');
              data.addColumn('number', 'Jitter (ms)');
              data.addColumn('number', 'DOI Index');
              data.addColumn('number', 'HDI Index');
 
    N.B. the data must be uploaded in a chronologically ascending order.
 
5. Create .html files for each vantage point: CERN Switzerland (CH.CERN.N2),
    SLAC USA (EDU.SLAC.STANFORD.N3), ICTP Italy (IT.ICTP.N1)
    for each region: Africa, Balkans, Central Asia, East Asia, Europe, Latin America,
    Middle East, North America, Oceania, S.E. Asia, South Asia, World, for each
    granularity: Yearly (since 1998), Monthly (last 20 months) and Daily (last 3 weeks).

Updating data files

The PingER data files are generated using the prm scripts. There are some trscronjobs running under pinger@pinger.slac.stanford.edu.  They appear as:

################################################################################
#prm.pl runs monthly to create the executive plots to be accessesd via:
#http://www-iepm.slac.stanford.edu/pinger/prmout/
pinger;600 30 02 1 * * /afs/slac.stanford.edu/package/pinger/prm/prm.pl \
        -c /afs/slac.stanford.edu/package/pinger/prm/prm-allmonthly.conf

#And yearly (Jan 5th at 4:30am) to get the yearly data.
pinger;600 30 04 05 1 * /afs/slac.stanford.edu/package/pinger/prm/prm.pl \
        -c /afs/slac.stanford.edu/package/pinger/prm/prm-allyearly.conf

#It also runs each morning to update the last 21 days information.
pinger;600 30 00 * * * /afs/slac.stanford.edu/package/pinger/prm/prm.pl \
        -c /afs/slac.stanford.edu/package/pinger/prm/prm-last60days.conf

#Create the motion bubble plots:
pinger;10 00 08 * * * /afs/slac/package/pinger/motion-chart/create-motion-chart-pages.pl

On the other hand, the demographics need to be updated manually. The file containing the coalesced statistics is demographics.csv. However to update the records, the steps listed below must be followed. Files relevant to the process are:

Flat file listing the statistics:
/afs/slac/package/pinger/motion-chart/demographics.csv
The file looks like:
#Country, Year, population, inetusers, inetpenetration, doi, hdi, cpi, idi
Afghanistan, 1998, 22912814, 0, 0.1, 0, , 0, 0
...
Afghanistan, 2019, 37209007, 0, 11.45, 0, 0.498, 0, 1.95


The script to generate csv files (each reflecting records for a single statistic):
/afs/slac/package/pinger/motion-chart/generate-csv-from-demographics.pl*

 The script to generate demographics.csv from the independent csv files:
/afs/slac/package/pinger/motion-chart/generate-demographics-from-csv.pl*

The intermediate flat files listing each statistic separately are:

CPI: /afs/slac/package/pinger/motion-chart/cpi.csv #Corruptions Perceptions Index from Transparency International
Appears as:
Country, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019
Afghanistan, , , , , 0.373, 0.383, 0.398, 0.408, 0.417, 0.429, 0.437, 0.453, 0.463, 0.471, 0.482, 0.487, 0.491, 0.493, 0.494, 0.498, 0, 0.498

DOI: /afs/slac/package/pinger/motion-chart/doi.csv #Digital Opportunity Index from ITU
Appears as:
Country, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019
Algeria, 0.39, 0.39, 0.39, 0.39, 0.39, 0.39, 0.39, 0.39, 0.42, 0.42, 0.42, 0.42, 0.42, 0, 0, 0, 0, 0, 0, 0, 0, 0

HDI: /afs/slac/package/pinger/motion-chart/hdi.csv #Human Development Index from UNDP
Appears as:
Country, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019
Afghanistan, , , , , 0.373, 0.383, 0.398, 0.408, 0.417, 0.429, 0.437, 0.453, 0.463, 0.471, 0.482, 0.487, 0.491, 0.493, 0.494, 0.498, 0, 0.498

IDI: /afs/slac/package/pinger/motion-chart/idi.csv #ICT Development Index from ITU
Internet Penetration: /afs/slac/package/pinger/motion-chart/inetpenetration.csv
Appears as:
Country, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019
Afghanistan, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.71, 1.71, 1.71, 1.71, 1.71, 1.71, 1.71, 1.95, 1.95, 1.95

Internet Users: /afs/slac/package/pinger/motion-chart/inetusers.csv
Appears as:
Country, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019
Afghanistan, 0, 0, 0, 1000, 1000, 20000, 25000, 300000, 535000, 500000, 500000, 1000000, 1000000, 0, 0, 0, 0, 0, 0, 0, 0, 0

Population: /afs/slac/package/pinger/motion-chart/pop.csv
Appears as:
Country, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019
Afghanistan, 22912814, 23646128, 23898198, 23997412, 25268405, 27060359, 28513677, 29928987, 31056997, 31889923, 32738376, 28396000, 29121286, 30
019922, 30918557, 31817193, 32715829, 33614464, 34513100, 35411736, 36310371, 37209007

Note that the format of these files is such that each row represents statistics of a
country and the columns list the annual measurements. e.g. the hdi.csv has the
following format:
    #Country, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009
    Albania, 0.801, 0.801, 0.801, 0.801, 0.801, 0.801, 0.801, 0.801, 0.801, 0.801, 0.801, 0.801
    Algeria, 0.733, 0.733, 0.733, 0.733, 0.733, 0.733, 0.733, 0.733, 0.733, 0.733, 0.733, 0.733
    AmericanSamoa, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
 
Note that the header of the intermediate csv files must be of the form:
Country, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009

Updating the records

:

1. Create a backup of the demographics.csv. This is just to avoid loss of information
    (in case of any error).
    (command: cp demographics.csv demographics.csv.back)
2. Execute the following command to generate the separate .csv files:
    ./generate-csv-from-demographics.pl
    This will generate the required files (cpi.csv, doi.csv, hdi.csv, inetpenetration.csv,
    inetusers.csv and pop.csv).
3. These separate csv files maintain records for a single statistic. Any update required
    to the demographics data should be made to these csv files. For example to add records
    for the year 2010, open each csv (for the separate statistic) in MS Excel and add
    another column. The header must state the correct year (note the file format). Similarly,
    to add countries to the records, add a row with the annual measurements separated by
    commas.
4. Once all the csv files have been updated, execute the script to generate the coalesced
    demographics statistics.
   ./generate-demographics-from-csv.pl
   This command would read the separate .csv files and generate an updated demographics.csv
   file.

There are certain conditions that must be met:

  1. All the files (the *.csv and *.pl) must be in the same folder.
  2. Both the scripts (.pl) are strongly dependent on the format of .csv files being correct. 
  3. The order of execution must follow the steps listed above (i.e generate the separate csv files, edit them and then generate the new demographics.csv)
  4. The country names used here must be the same as those used by PingER. The source of the data presented here is mentioned in the section 'Miscellaneous Details'.

Both the scripts generate-csv-from-demographics.pl and generate-demographics-from-csv.pl take options as input parameters for customization.

Adding a new statistic would require simple changes(/additions) to the scripts below. The code is documented to facilitate this process.

/afs/slac/package/pinger/motion-chart/generate-csv-from-demographics.pl*
/afs/slac/package/pinger/motion-chart/generate-demographics-from-csv.pl*

Generating up-to-date PingER Metric Motion Charts

To generate up to date .html files of the PingER metric motion charts execute the following script:

/afs/slac/package/pinger/motion-chart/create-motion-chart-pages.pl

The charts generated (listed below) may then viewed publicly via SLAC's web server at https://www-iepm.slac.stanford.edu/pinger/prmout/

/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-IT.ICTP.N1-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-CH.CERN.N2-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-CH.CERN.N2-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-CH.CERN.N2-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.N3-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.N3-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.N3-monthly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-IT.ICTP.N1-allyearly.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-IT.ICTP.N1-last60days.html
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-IT.ICTP.N1-monthly.html

Miscellaneous Details

  • Tool: The interactive graph was generated using the Google Motion Chart Widget.
  • Data: The data presented here was collected by the PingER project, processed by prmout. The same data is available in tabular format. The statistics of Population were acquired from the U.S. Census Bureau, the country to region mapping was obtained by the geographical database maintained by the PingER project, where as the Internet usage statistics were taken from Internet World Stats. Please note that while all the statistics were acquired over several years (i.e. since 1998), the Internet usage statistics were documented in Nov. 2007.
  • Loading: The time to load and render the web page is largely determined speed of the link, the file length (about a MByte that is determined by the number of metrics and frequency of data points and the number of metrics) and the speed of the client rendering.
  • Metrics: Average RTT (ms), Normalized Throughput (Kbps), Internet Users (#), Internet Penetration (%), Population (#), Minimum RTT (ms), Packet Loss (%), Unreachability (%), Jitter (ms), DOI Index, HDI Index,
  • Authors: Umar Kalim & Les Cottrell 10/10/2008, version 1.3. Idea champion: Shahryar Khan.

Problems

Sometimes instead of balloons, stars or asterisks are displayed.   Try a complete refresh (CTRL-Refresh).

Due to unavailability of data for population and internet users for 2011, the static version of demographics.csv was generated. Instead of generating the demographics.csv from the script generate-demographics-from-csv.pl use the static version of demographics.csv.

Internet users and population have to be integer values. If these are decimal values, the HTML page chart fails to generate even though create-motion-chart-pages.pl runs fine.

The text appears but no chart. You probably need to enable flash.

Anomalous values of throughput

There are often anomalously high or low values or dramatic changes in throughput as one moves from year to year. This can typically be observed by looking at the motion metrics normalized throughput. There can be several causes:

  • A possibility is that the country has moved from using Geo Stationary Satellite (GEOS) to terrestrial links. This will dramatically reduce the minimum RTT from over 450ms to typically well under 350ms, and hence increase the throughput since it goes as 1/RTT. For example: see the change in Cuba between the year 2012 and the following years. This is due to the change from using a GEOS to a terrestrial link via Venezuela.
  • If we have very few (e.g. 1) target hosts in a country then maybe the target behaves strangely. For example, see Cape Verde from 2015 to 2019. it is seen the throughput drops from 5,531 Kbps to 395 Kbps. The main cause is the dramatic increase in loss from 0.2% to 15.5%. Note the throughput goes as 1/sqrt(loss). To see whether the cause is the anomalous behavior of a single target (in the case of Cape Verde the only target was www.cmsf.cv) if possible we add more targets in the country (e.g. in July 2019 we added more targets in Cape Verde). 

  • Another example is Iran where the loss rose from < 1% in 2015-2017  to over 7% in 2018 and 2019.
  • If we have multiple hosts then we can look at whether some targets are behaving anomalously (usually large RTTs, high losses) some of the time (e.g. as seen monthly)  and if so filter them out of the summations.
    • It may also be that the target has a low RTT or loss. The cause of the low RTT may be that the target is no longer physically located in the country but is a proxy closer by (e.g. in the US).  In this case, we replace the target. A low loss of 0.0000% may be since all the pings sent have been returned with no loss.  For a 365 day year we monitor each target with 10 (or up to 30 if there are losses) every 30 minutes, i.e. ~175K pings (or if 1 ping is lost a 0.0006% loss). If we monitor for a month then the loss of a single packet gives a loss of ~0.007%.
    • If there are big differences between targets in a country then we may need to add multiple hosts to get a quorum. For example, for Iran in 2019 there were huge differences in losses for the two targets monitored there (irissf.ir with losses of 0.1% - 0.8% and www.iut.ac.ir with losses of 7% -22% ).

Tar'ing files

[cottrell@pinger ~]$ tar -cvf umar.tar /afs/slac/package/pinger/motion-chart
tar: Removing leading `/' from member names

/afs/slac/package/pinger/motion-chart/

/afs/slac/package/pinger/motion-chart/generate-demographics-from-csv.pl

/afs/slac/package/pinger/motion-chart/generate-csv-from-demographics.pl

/afs/slac/package/pinger/motion-chart/inetusers.csv

/afs/slac/package/pinger/motion-chart/population.csv

/afs/slac/package/pinger/motion-chart/demographics.csv

/afs/slac/package/pinger/motion-chart/log.stdout

/afs/slac/package/pinger/motion-chart/backup-2009/

/afs/slac/package/pinger/motion-chart/backup-2009/generate-csv-from-demographics.pl

/afs/slac/package/pinger/motion-chart/backup-2009/generate-demographics-from-csv.pl

/afs/slac/package/pinger/motion-chart/backup-2009/inetusers.csv

/afs/slac/package/pinger/motion-chart/backup-2009/demographics.csv

/afs/slac/package/pinger/motion-chart/backup-2009/inetpenetration.csv

/afs/slac/package/pinger/motion-chart/backup-2009/idi.csv

/afs/slac/package/pinger/motion-chart/backup-2009/hdi.csv

/afs/slac/package/pinger/motion-chart/backup-2009/cpi.csv

/afs/slac/package/pinger/motion-chart/backup-2009/doi.csv

/afs/slac/package/pinger/motion-chart/backup-2009/create-motion-chart-pages.pl

/afs/slac/package/pinger/motion-chart/backup-2009/pop.csv

/afs/slac/package/pinger/motion-chart/backup-2009/create-motion-chart-page.pl

/afs/slac/package/pinger/motion-chart/backup-2009/log.conf

/afs/slac/package/pinger/motion-chart/backup-2008/

/afs/slac/package/pinger/motion-chart/backup-2008/population.csv

/afs/slac/package/pinger/motion-chart/backup-2008/population.csv.old2

/afs/slac/package/pinger/motion-chart/backup-2008/demographics.csv

/afs/slac/package/pinger/motion-chart/backup-2008/population.csv.old3

/afs/slac/package/pinger/motion-chart/backup-2008/population.csv.old4

/afs/slac/package/pinger/motion-chart/backup-2008/create-motion-chart-page.pl.bak

/afs/slac/package/pinger/motion-chart/backup-2008/create-motion-chart-page.pl.old2

/afs/slac/package/pinger/motion-chart/backup-2008/create-motion-chart-pages.pl

/afs/slac/package/pinger/motion-chart/backup-2008/population-old.csv

/afs/slac/package/pinger/motion-chart/backup-2008/population.csv.old

/afs/slac/package/pinger/motion-chart/backup-2008/create-motion-chart-page.pl

/afs/slac/package/pinger/motion-chart/backup-2008/generate-pinger-metrics-chart.sh

/afs/slac/package/pinger/motion-chart/backup-2008/create-motion-chart-page.pl.old

/afs/slac/package/pinger/motion-chart/backup-2008/create-motion-chart-pages.pl.bak

/afs/slac/package/pinger/motion-chart/generate-csv-from-demographics.v1.pl.bak

/afs/slac/package/pinger/motion-chart/generate-demographics-from-csv.v1.pl.bak

/afs/slac/package/pinger/motion-chart/inetpenetration.csv

/afs/slac/package/pinger/motion-chart/.demographics.csv.swp

/afs/slac/package/pinger/motion-chart/idi.csv

/afs/slac/package/pinger/motion-chart/hdi.csv

/afs/slac/package/pinger/motion-chart/cpi.csv

/afs/slac/package/pinger/motion-chart/doi.csv

/afs/slac/package/pinger/motion-chart/log.conf.bk

/afs/slac/package/pinger/motion-chart/.create-motion-chart-page.pl.swp

/afs/slac/package/pinger/motion-chart/create-motion-chart-pages.pl

/afs/slac/package/pinger/motion-chart/.population.csv.swp

/afs/slac/package/pinger/motion-chart/demographics.csv.bk

/afs/slac/package/pinger/motion-chart/pop.csv

/afs/slac/package/pinger/motion-chart/create-motion-chart-page.pl

/afs/slac/package/pinger/motion-chart/log.conf

/afs/slac/package/pinger/motion-chart/logs/

/afs/slac/package/pinger/motion-chart/logs/log.stdout

/afs/slac/package/pinger/motion-chart/logs/archives/

/afs/slac/package/pinger/motion-chart/logs/archives/motion_chart.log.2009-10-09.tar.gz

/afs/slac/package/pinger/motion-chart/logs/archives/motion_chart.log.2009-10-11.tar.gz

/afs/slac/package/pinger/motion-chart/logs/archives/motion_chart.log.2009-10-13.tar.gz

/afs/slac/package/pinger/motion-chart/logs/archives/motion_chart.log.2009-10-12.tar.gz

/afs/slac/package/pinger/motion-chart/logs/archives/motion_chart.log.2009-10-10.tar.gz

/afs/slac/package/pinger/motion-chart/logs/manage-logs.pl

/afs/slac/package/pinger/motion-chart/backup-2010/

/afs/slac/package/pinger/motion-chart/backup-2010/inetusers.csv

/afs/slac/package/pinger/motion-chart/backup-2010/demographics.csv

/afs/slac/package/pinger/motion-chart/backup-2010/inetpenetration.csv

/afs/slac/package/pinger/motion-chart/backup-2010/idi.csv

/afs/slac/package/pinger/motion-chart/backup-2010/hdi.csv

/afs/slac/package/pinger/motion-chart/backup-2010/cpi.csv

/afs/slac/package/pinger/motion-chart/backup-2010/doi.csv

/afs/slac/package/pinger/motion-chart/backup-2010/create-motion-chart-pages.pl

/afs/slac/package/pinger/motion-chart/backup-2010/pop.csv

/afs/slac/package/pinger/motion-chart/backup-2010/create-motion-chart-page.pl

/afs/slac/package/pinger/motion-chart/.create-motion-chart-page.pl.swo

/afs/slac/package/pinger/motion-chart/idi-20100809.csv

/afs/slac/package/pinger/motion-chart/pop-20190629.csv

/afs/slac/package/pinger/motion-chart/demographics-20190630.csv

/afs/slac/package/pinger/motion-chart/junk

/afs/slac/package/pinger/motion-chart/ir.csv

/afs/slac/package/pinger/motion-chart/ye.csv

/afs/slac/package/pinger/motion-chart/.generate-demographics-from-csv.pl.swp

/afs/slac/package/pinger/motion-chart/demographics-20190713.csv

/afs/slac/package/pinger/motion-chart/demographics-20190714.csv

/afs/slac/package/pinger/motion-chart/demographics-20190714-new.csv

[cottrell@pinger ~]$ ls -l umar.tar 

-rw-r--r-- 1 cottrell sf 7884800 Sep 23 14:59 umar.tar
 

Or with compression which reduces the tarball from 7,884,800 Bytes to 1,509,685 Bytes

[cottrell@pinger ~]$ tar -czvf umar.tar.gz /afs/slac/package/pinger/motion-chart


tar: Removing leading `/' from member names

/afs/slac/package/pinger/motion-chart/

/afs/slac/package/pinger/motion-chart/generate-demographics-from-csv.pl


...
[cottrell@pinger ~]$ ls -l  umar.tar.gz 

-rw-r--r-- 1 cottrell sf 1509685 Sep 23 15:07 umar.tar.gz

In addition one will also need to access:

The anonymous ftp space for user cottrell is at /afs/slac/public/users/cottrell, thus all we need to do is cp umar.tar.gz /afs/slac/public/users/cottrell

[cottrell@pinger ~]$ cp umar.tar.gz /afs/slac/public/users/cottrell/

[cottrell@pinger ~]$ ls -l /afs/slac/public/users/cottrell/umar.tar.gz 

-rw-r--r-- 1 cottrell sf 1509685 Sep 23 15:51 /afs/slac/public/users/cottrell/umar.tar.gz

One can access the file from the web via ftp://ftp.slac.stanford.edu/users/cottrell

Maintaining Logs

Log4perl has been used to create logs and alert the administrators (currently kalim & cottrell) of any anomaly.

The logging process maintains the logs by using two appenders (sinks):

  1. The first appender is used to highlight critical errors -- as a result of these errors, the charts fail to appear. If such an error occurs an email describing the error is sent to the administrators.
  2. The second appender maintains detailed information while the charts are generated.

The configuration file for the logging process is located at:

/afs/slac/package/pinger/motion-chart/log.conf

The log itself is stored at:

 /afs/slac/package/pinger/motion-chart/logs/motion_chart.log

The configuration file is described below:

# Define the root logger and the filters. Set the trigger level to INFO
log4perl.logger = INFO, AppInfo, AppError

# Filter to match level ERROR. Use this filter for the appender
# which sends emails about critical errors
log4perl.filter.MatchError = Log::Log4perl::Filter::LevelMatch
log4perl.filter.MatchError.LevelToMatch  = ERROR
log4perl.filter.MatchError.AcceptOnMatch = true

# Filter to match level INFO. Use this filter for the appender
# maintaining detailed logs.
log4perl.filter.MatchInfo  = Log::Log4perl::Filter::LevelMatch
log4perl.filter.MatchInfo.LevelToMatch  = INFO
log4perl.filter.MatchInfo.AcceptOnMatch = true

# Define the Error appender
log4perl.appender.AppError         = Log::Dispatch::Email::MailSend
log4perl.appender.AppError.to      = kalim@slac.stanford.edu, cottrell@slac.stanford.edu
log4perl.appender.AppError.subject = [Missing data in Motion Charts] see /afs/slac/package/pinger/motion-chart/logs/motion_chart.log
log4perl.appender.AppError.layout = \
    Log::Log4perl::Layout::PatternLayout
log4perl.appender.AppError.layout.ConversionPattern = %d %p (%L) %m %n
# Inform about critical errors only
log4perl.appender.AppError.Threshold = ERROR
# log4perl.appender.AppError.Filter   = MatchError

# Information appender
log4perl.appender.AppInfo = Log::Log4perl::Appender::File
log4perl.appender.AppInfo.filename    = logs/motion_chart.log
log4perl.appender.AppInfo.mode = write
log4perl.appender.AppInfo.layout = \
    Log::Log4perl::Layout::PatternLayout
log4perl.appender.AppInfo.layout.ConversionPattern = %d %p %L %m %n
# log4perl.appender.AppInfo.Filter   = MatchInfo

The log maintaining detailed information may grow to a large size. To avoid this problem I wrote a script which rotates the logs on daily basis while maintaining archives of the last five days.

The script is located at:

/afs/slac/package/pinger/motion-chart/logs/maintain-logs.pl

The archives at stored at:

 /afs/slac/package/pinger/motion-chart/logs/archives/

The script maintain-logs is executed daily as a cron-job

Updates

July 2019

Updated several of the metrics, in particular the PingER metrics (minimum and average RTT, jitter, loss, unreachability, throughput), plus IDI, HDI, Penetration and Population. the following were not updated and hence terminate in 2010: DOI, CPI, Users.

October 2019

The frame for the chart was being displayed but with no contents.

Umar pointed out

Please change line 307, which currently lists:
      <script type='text/javascript' src='https://www.google.com/jsapi'></script>
to be: 
<script type='text/javascript' src='https://www.gstatic.com/charts/loader.js'></script>. N.b. do not use " instead of '.
Determined the change was needed in /afs/slac/package/pinger/motion-chart/create-motion-chart-page.pl (nb no 's' in page).
Backup copy kept in ~cottrell/bin/afs/slac/package/pinger/motion-chart/create-motion-chart-pages-20191012.pl
Reran create-motion-chart-pages.pl (nb this one has an 's' in it).

Sat Oct 12 10:48:28 2019 create-motion-chart-pages.pl: finished, took 239 secs.

I followed the instructions for Enabling Flash in Chrome at https://support.digication.com/hc/en-us/articles/115003963468-Enabling-Flash-for-Google-Chrome-Windows-Macintosh-2.0.0 is installed in this browser

Chrome No change.

  •  Viewed source and made sure change had taken effect
  • Looked in Developer tools, noted the following which may be of interest
    • pinger-metrics-motion-chart.html:51508 A parser-blocking, cross site (i.e. different eTLD+1) script, https://ssl.google-analytics.com/ga.js, is invoked via document.write. The network request for this script MAY be blocked by the browser in this or a future page load due to poor network connectivity. If blocked in this page load, it will be confirmed in a subsequent console message. See https://www.chromestatus.com/feature/5718547946799104 for more details.
  • (anonymous) @ pinger-metrics-motion-chart.html:51508
  • pinger-metrics-motion-chart.html:1 Cross-origin plugin content from must have a visible size larger than 400 x 300 pixels, or it will be blocked. Invisible content is always blocked.

Firefox No change

Umar pointed the problem was related to having no data for Normalized throughput. If one chooses another metric for the y axis default then the content is displayed.

  • Normalized throughput is automatically updated by prm.pl via trscrontab on pinger@pinger.slac.stanford.edu

    pinger;1300   30 07 04 1 * /afs/slac.stanford.edu/package/pinger/prm/prm.pl -lp true -c /afs/slac.stanford.edu/package/pinger/prm/prm-allyearly-continent.conf #takes 11 mins on PingER 11/24/2012
  • Ran it manually to be sure and looked at the date updated: -rw-rw-rw- 1 pinger sf 1143 Oct 14 10:18 nthroughput-EDU.SLAC.STANFORD.PINGER-continent-allyearly.csv
  • Reran create-motion-chart-pages.pl
  • No change.
  • Looked at the file https://www-iepm.slac.stanford.edu/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-continent-allmonthly.csv (created by prm.pl using the configuration file /afs/slac.stanford.edu/package/pinger/prm/prm-allyearly-continent.conf) and found it contained all periods ('.'). The configuration file looks OK. Reordered prm-allyearly-continent.conf and re-ran prm.pl. On the other hand the file https://www-iepm.slac.stanford.edu/pinger/prmout/throughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv appears OK.
  • Executing prm.pl with the above .conf file it appears data is available in for example https://www.slac.stanford.edu/cgi-bin/pingtable.pl?file=throughput&by=by-node&size=100&tick=allyearly&from=EDU.SLAC.STANFORD.PINGER&to=Africa&ex=none&only=all&ipv=all&dataset=hep&percentage=any, as well as for the equivalent files=minimum_rtt and file=packet_loss.
  • Fixed missing single quotes (') in mon-lib.pl sub wget.  Now https://www-iepm.slac.stanford.edu/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-continent-allyearly.csv has data.
  • Look at source of pinger-metrics-motion-char.html, Normalized throughput is metric_index 4 in data.setValue(datapoint_index,metric_index,region), however it does not show up.
  • Look at: /afs/slac/package/pinger/motion-chart/create-motion-chart-page.pl -r Europe -f EDU.SLAC.STANFORD.PINGER -t allyearly. The normalized throughput file is /afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv. However, this file (AKA https://www-iepm.slac.stanford.edu/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv) has no data, e.g.

    allyearly,?,1998,1999,2000,2001,2002,2003,2004,2005,2006,2007,2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018,2019

    EDU.SLAC.STANFORD.PINGER-to-Afghanistan,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,

    EDU.SLAC.STANFORD.PINGER-to-Albania,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,

  • Re-ran /afs/slac.stanford.edu/package/pinger/prm/prm.pl -lp true -c /afs/slac.stanford.edu/package/pinger/prm/prm-allyearly-slac.conf to yield https://www-iepm.slac.stanford.edu/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv AKA /afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv which has data in it.

    • Reran ~cottrell/bin/create-motion-chart-pages.pl, still no Normalized throughput data in https://www-iepm.slac.stanford.edu/pinger/pinger-metrics-motion-chart.html
    • There is data in /afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv AKA https://www-iepm.slac.stanford.edu/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv
    • Looking in /afs/slac/package/pinger/motion-chart/create-motion-chart-page.pl -r Europe -f EDU.SLAC.STANFORD.PINGER -t allyearly it opens (around line 968) the file /afs/slac/g/www/www-iepm/pinger/prmout/nthroughput-EDU.SLAC.STANFORD.PINGER-country-allyearly.csv and data is found there.
    • Reran /afs/slac/package/pinger/motion-chart/create-motion-chart-pages.pl, still no Normalized throughput data there.

    • Normalized throughput is metric_index 4 in data.setValue(datapoint_index,metric_index,region). It is added around line 444 in /afs/slac/package/pinger/motion-chart/create-motion-chart-page.pl. Using perl -d /afs/slac/package/pinger/motion-chart/create-motion-chart-page.pl -r Europe -f EDU.SLAC.STANFORD.PINGER -t and halting at

      442:	            unless (!defined($data{"nthroughput"}->{$country}->{$interval})) {
443:	                if (   (($data{"nthroughput"}->{$country}->{$interval}) ne ".")
444 	                    && (($data{"nthroughput"}->{$country}->{$interval}) ne "")
445 	                    && (($data{"nthroughput"}->{$country}->{$interval}) != 0)){
446:	                  print OUTFILE "              data.setValue($j, 4, "
447 	                      . $data{"nthroughput"}->{$country}->{$interval} . ");\n";
448 	                }
449 	            }
DB<15> p "data.setValue($j, 4, $data{'nthroughput'}->{$country}->{$interval});\n"
data.setValue(693, 4, 12240.9);
And it is written to OUTFILE /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.PINGER-allyearly.html
Looking in /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.PINGER-allyearly.html we see lines of the form:
              data.addRows(1);
              data.setValue(5, 0, 'Denmark');
              data.setValue(5, 1, 1998);
              data.setValue(5, 2, 'Europe');
              data.setValue(5, 3, 210.241);
              data.setValue(5, 4, 2919.71);    #Note the metric_index=4, which is for nthroughput
              data.setValue(5, 16, 360.566)
The metric_index of 4 is deduced from:
var data = new google.visualization.DataTable();
              data.addColumn('string', 'Country');
              data.addColumn('number', 'Year');
              data.addColumn('string', 'Region');
              data.addColumn('number', 'Average RTT (ms)');
              data.addColumn('number', 'Normalized Throughput (Kbps)');
              data.addColumn('number', 'Population (#)');
Then we ran 
time /afs/slac/package/pinger/motion-chart/create-motion-chart-pages.pl
It took 225 seconds.
However when we look at 
/afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart.html
we see:
data.addRows(1);
              data.setValue(39, 0, 'Denmark');
              data.setValue(39, 1, 1998);
              data.setValue(39, 2, 'Europe');
              data.setValue(39, 3, 210.241);
              data.setValue(39, 16, 360.566)
We also verified that /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.PINGER-allyearly.html had not changed So the new data is not getting between create-motion-chart-page.pl to create-motion-chart-pages.pl
Also:
-bash-4.1$ ls -l /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 pinger sf 465357 Oct 16 12:44 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.PINGER-allyearly.html
Where as the file being displayed by
bash-4.1$ ls -l /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart.html
lrwxr-xr-x 1 pinger sf 103 Jun 29 21:54 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart.html -> /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.PINGER-allyearly.html
-bash-4.1$ ls -l /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 pinger sf 2152879 Oct 14 10:22 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.PINGER-allyearly.html
Which is unexpected since I thought re-running create-motion-chart-pages.pl would have updated the above file to today (Oct 16). I am unclear what
updated the file on Oct 14. 


The possible files are:
bash-4.1$ ls -l /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart*EDU*allyearly*
-rw-rw-r-- 1 kalim  sg  420174 Jun 29 18:17 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf  588037 Oct 16 12:43 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Africa-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg   97918 Jun 29 18:17 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf  137858 Oct 16 12:43 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Balkans-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg  101016 Jun 29 18:17 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf  135565 Oct 16 12:44 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Central_Asia-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg   65743 Jun 29 18:18 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf   73439 Oct 16 12:44 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-East_Asia-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg  334304 Jun 29 18:18 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf  465357 Oct 16 12:44 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Europe-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg  224179 Jun 29 18:18 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf  310831 Oct 16 12:45 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Latin_America-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg  141526 Jun 29 18:18 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf  209985 Oct 16 12:45 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Middle_East-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg   50387 Jun 29 18:18 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf   66114 Oct 16 12:45 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-North_America-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg   54170 Jun 29 18:19 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf   75277 Oct 16 12:46 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-Oceania-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg  103053 Jun 29 18:19 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf  149525 Oct 16 12:46 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-S.E._Asia-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg   87208 Jun 29 18:19 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf  123603 Oct 16 12:46 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-South_Asia-EDU.SLAC.STANFORD.PINGER-allyearly.html
-rw-rw-r-- 1 kalim  sg 1588320 Jun 29 18:19 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.N3-allyearly.html
-rw-rw-r-- 1 pinger sf 2152879 Oct 14 10:22 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.PINGER-allyearly.html
lrwxr-xr-x 1 pinger sf      64 Jun 29 21:51 /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart-all-EDU.SLAC.STANFORD.SLAC-allyearly.html -> /afs/slac/g/www/www-iepm/pinger/pinger-metrics-motion-chart.html
Also below are the crontab entries:
#Create the motion bubble plots, create-motion-chart-pages.pl takes ~ 4 mins:
pinger;30 00 18 * * * /afs/slac/package/pinger/motion-chart/create-motion-chart-pages.pl # Takes ~5 minutes
pinger;02 05 11 * * * /afs/slac/package/pinger/motion-chart/logs/manage-logs.pl -n 5 > /afs/slac/package/pinger/motion-chart/logs/log.stdout

 

 


 

 

 






 

 

 

 

 

  • No labels