Introduction
To facilitate the analysis and presentation of the PingER data by others we made the historical hourly data available via anonymous FTP.
- The initial audiences were:
- Chin Fang for part of the Zettar demonstration of file transfer at the Super Computing 2014
- Renan Souza for the UFRJ Linked Open data project
- Ibrahim Abaker of UM for his Big Data project
- This hourly data was chosen rather than the raw data since it is cleaner having been through several filters.
Volume of data
There are roughly 100,000 files. The volumes of the files are shown below.
| Uncompressed Volume of files per Year for all metrics | Uncompressed Volume for 3 metrics |
|---|---|
 |  |
Compression
The compression ratio is about 5:1.
| Frequency of files by size from compressed & uncompressed data | Cumulative & yearly compressed data volumes |
|---|---|
 |  |
Missing data
There are two types of missing data:
- Given the number of days, number of metrics we expect 6209 files in a year. However typically some files are missing (e.g. data may not have been gathered that day, the gathering host may have been down, the data may have been corrupted etc.)
- The monitoring host may not have been working or the host being monitored may not have responded to the pings. In this case since there is no value for the metric for the monitoring-remote host pair, then a dot is recorded.
| Missing files/year by metric | Total number of dots per year |
|---|---|
 |  |
Spreadsheets
Hourly file data analysis from Renan