Attendees:
Joao, Bebo, Les (SLAC), Maria, Marcos, Antonio, Renan, Claudio (Brazil)
...
Introduction to PingER if necessary, have they looked at?, n.b. Maria, Les, Joao, Bebo, Antonio are already familiar with PingER
- http://www-iepm.slac.stanford.edu/pinger/
- http://www.slac.stanford.edu/comp/net/wan-mon/tutorial.html
- http://www-iepm.slac.stanford.edu/paperwork/ieee/ieee.pdf
- PingER Papers and Presentations
- Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance, Thiago Barbosa, Renan Souza, Sergio Serra, R. Les Cottrell presented at NetApps2015, Dec 2015
...
Hourly data is accessible via FTP at ftp://ftp.slac.stanford.edu/users/cottrell/ as tar files by metric. This is updated daily. See Archiving PingER data by tar for retrieval by anonymous ftp for how to access the complete set. Also can just few (up to 60) days in case you missed some days (see PingER Public FTP Archive daily update). Could use this to populate one's own warehouse.
...
What are interests and for whom:
- Warehouse
- Maria suggests two undergraduate students put up an OLAP application on top of Joao's Data Warehouse.
- Maria already has a an ETL (Extraction Transforming Loading) process for a small Data Warehouse they had at UFRJ (done Cristiane). This could be adapted to maintain the SLAC Data Warehoues from the FTP hourly PingER data.
- One of Maria's students (Rafaelo) has a cluster of four Impala/Hadoop nodes. He can use SPARQL to make analytical queries (not sure if data is PingER or other)
- Renan interested in speeding up queries from the Warehouse
- Semantic web
- Bebo, Renan
- Visualization & analysis
- Marcos, Claudio and Antonio
- One thing they are looking at is real-time visualization, which needs fast response. From Renan's work a couple of years ago response time to OLAP queries using Impala of 100 million tuples was ~ 30 seconds.
- Marcos, Claudio and Antonio
Next meeting
Thursday 8/1219/2016 4:00pm SLAC, 8:00pm Rio.
Atendees: Les (SLAC), Bebo (SLAC), João (SLAC), Guto (Rio), Renan (Rio), Maria Luiza (Rio)
Some slides about the meeting: PingER Vis.pdf
Also look at meeting Brazil in early September after Joao returns.
...