Page History

Purpose

PingER (Ping End-to-end Reporting) is the name given to the Internet End-to-end Performance Measurement (IEPM) project to monitor end-to-end performance of Internet links. The project now involves hundreds of sites in many countries all over the world and we are actively seeking new partners for this project.

• Brochure, Examples of Use,  Publication,  Tutorial , What is PingER (from ICTP/Trieste)

Case Studies

• , Submission to Silicon Valley TechAwards 2012.

Examples of Use

See here.

Case Studies

PingER Validation

...

• Development Indices and pingER

Presentations on Case Studies

• PingER
• Derived Throughput
  • Throughput versus Loss
  • Ookla Speedtest.net Throughput
  • ZDnet Australia's Speedtest
  • Ping minRTT vs HTTP minRTT validation
• Also see https
• Sub-Sahara Region,
• ICFA/SCIC report on Africa,
• Powerpoint presentation on Sub-Saharan Africa
• Internet Connectivity in Africa, presented by Les Cottrell at the Internet and Grids in Africa: An Asset fro African Scientists for the Benefit of African Society, 10-12 December 2007, Montpellier France.
• Internet View of the Digital Divide, Especially for Sub-Saharan Africa, presented by Les Cottrell at the IHY Africa Workshop 11-16 November 2007, Addis Ababab, Ethiopia.
• http://www.slacakamai.stanford.educom/grpus/scsen/net/talk07/icfa-mexico07.pptQuantifying the Digital Divide: Latin America, S. Asia, Africa, presented by Les Cottrell at the ICFA Workshop on Digital Divide Issues for Global e Science, Mexico City Oct 24-27, 2007
• Quantifying the Digital Divide: A Scientific Overview of Network Connectivity and Grid Infrastructure in South Asian Countries, S. Khan, R. L. Cottrell, U Kalim, A. Ali, CHEP07, Victoria Canada. Also SLAC-PUB 12581. Poster

PingER Validation

• our-thinking/state-of-the-internet-report/state-of-the-internet-connectivity-visualization.jsp which goes back quarter by quarter to 2007 with Akamai download speeds by country with a linear y scale. The full report is at: https://content.akamai.com/pg8228-q4-2016-soti-connectivity-report.html
   Image AddedImage Added
• Broadband tariffs see here
  Image Added 

PingER Analysis

• PingER data Analysis
• Monitoring Data Format
• Anomalous Hosts
• CERN View
• Conditional Port Probability
• Duplicate Packets
• MOS
• Derivation of Directness of Connection
• Derived Throughput
  • Throughput versus Loss
  • Ookla Speedtest.net Throughput
  • ZDnet Australia's Speedtest
• Daily Data Validation Management Down Sites, Collection Status

PingER Operations

• Downloading Pinger
• Requirements for Monitoring Host, Requirements for Remote Host
• Data: Meta database (NODEDETAILS), NODEDETAILS hints,  NODEDETAILS schema, NODEDETAILS in Perl, NODETAILS in XML,  Raw Data Format; Adding a new node to NODEDETAILS .
• PingER FAQ
• Management:
  • Getting an account at SLAC, access privileges etc.
  • Data Processing Workflow,
    • Archiving PingER data
    • Retrieving PingER data
  • Adding metrics to PingER,
  • Finding target hosts,
  • Duties, 
  • Gathering delays,
  • Maintaining collection & analysis,
  • Restoring data,
  • Usage Information for Scripts
  • Daily Data Validation Management
  • Down Sites, Collection Status
• Mapping of file space
• Choice of PingER packet size
• Pakistani hosts with new PingER2 to not lose <BeaconList> March 2012.

• Moving Pinger January 2022

PingER Deployment

• PingER deployment 2008
• Shockwave Map, Google Map, Node count Intensity map, Node count GeoMap, Interactive Java Map (VIPER)
• Beacons , Beacon Expansion
• Monitoring Hosts, Remote Hosts
• Regions, Hosts per Country by REgionRegion
• Status of Pakistani monitoring hosts
• South East Asian Hosts (from UNIMAS)
• History of PingER Deployment
• Finding PingER remote hosts
• PingER at DESY

PingER Databases

Report on converting PingER flat files to a SQL database, Ghulam Nabi, M. Farhan Maqsoos, Bilal Naseer.

• NODEDETAILS
• Countries and Regions
• Database Passwords

PingER Analysis

• Monitoring Data Format

Toolbox

• Database Synchronization
• PingER-DB Implementation details
• PingER Schema Queries
• PingER DB usage
• Comparison of the speed of the new perfSONAR schema database versus the old flat files

PingER Data

• PingER Data Flow at SLAC
• Archiving Analyzed Pinger Data by tar for Retrieval by Anonymous FTP

Toolbox

• Add new metrics to PingER
• PingER Data Explorer
• PingER PingER Host Searcher
• PingER Route Visualizer
• PingER Visualization(VIPER)
• PingER Smokeping Integration
• PingER Metrics Motion Chart
• PingER Metrics Intensity Maps
• PingER Node Count Intensity & GeoMaps
• PingER Monitored Node Status
• Deploying table.pl
• Updating ping_data.pl and traceroute.pl

Porting PingER Archive to NUST

• PingER Meta-Database
• Web interface to manage the PingER Meta-Database
• Synchronization of the SLAC and NUST databases
• Scripts for fetching data from monitoring sites and pinger management
• Analysis and Implementation of Relational Archive Site for PingER and CBG Integration with TULIP
• Pinger Archive Site

PingER2

Installation

http://www.rnoc.gatech.edu/pinger2/, also Johari has created a shell script to automate the installation of pinger package in Ubuntu/Fedora/Centos Linux distro.  The finalized installation script for pinger software for CentOS/Fedora based distribution is available from the following url (past the step by step tutorial): http://pinger.unimas.my/pinger/install-tutorial.php. Kashif assisted in testing the script.

http://confluence.slac.stanford.edu/display/IEPM/PingER2+at+SLAC

Raja Asad Khan, October 2012

PingER2 Data Flow

...

& Access

• PingER2 data flow

...

• PingER data flow at SLAC
• Accessing PingER Data

Work in Progress

1. Automating the creation of offsite.nodes directly from nodedetails.
2. Automating the creation of beacons.txt directly from nodedetails.
3. Document procedures for editing existing nodes.

ePingER

This is a project to provide low cost, low power PingER monitoring or remote stations.

...

Future Projects

TULIP

• CAIDA RTT vs. distance results
• TULIP database backup
• Laundering Landmarks
• Incorrect geolocation results
• TULIP Task List

The following sections provide documentation and instruction on how to do any and everything in Pinger. This is a work in progress as new tools are documented as developed and old code revamped.

Progress Reports

...

• ePingER on Android phone
• ePingER in Zambia
• ePingER in Malaysia
• ePingER at SLAC

PingER Futures

• FAQ

TULIP

Tulip Web Site
Old TULIP Web site
Tulip-CBG project report, by Ghulam Nabi, M. Farhan, Maqsood, Buialal Naseer, 2011
TULIP Analysis of alpha as a function of RTT and Region for Europe, North America and Pakistan, by Raja Asad Khan, October 2012.
Discussion on using International gateways to provide alpha between regions.

Visual Traceroute (VTrace)

PingER LOD 

This is a project to provide Linked Open Data access to PingER data. 

• PingER LOD Home page
• PingER LOD project Wiki documentation.
• PingER LOD publications
• Proposed project by Ibrahim at UM
• PingER and Hadoop proposed by Ridzuan, also see the original proposal

PingER Warehouse

• PingER Data Warehouse using Big Data with Cloudera on Nebula
• Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance, T. M. S. Barbosa, R. F. Souza, S. M. S. Cruz, M. L. Campos and R. L. Cottrell. SLAC PUB, submitted to NETAPPS 2015. 
• Survey on Big Data Indexing strategies, Fatima Bintu Adama, Adib Habbal, Suhaidi Hassan, R. Les Cottrell. Bebo White, Ibrahim Abdullahi.

PingER and BigData

Leveraging PingER big data via  modified pingtable.

PingER and perfSONAR

At one time perfSONAR supported PingER. This was discontinued October 2014.

Comparison of PingER and perfSONAR.

Future PingER Projects

Progress Reports

PingER Papers and Presentations

Pinger Meeting notes