You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Attendees:

Joao, Bebo Les (SLAC), Maria, Marcos, Antonio, Renan, Claudio (Brazil)

Admin

Contacts see Contacts for Brazil.

Topics

Introduction to PingER if necessary, have they looked at?

There appears to be considerable interest in the Brazilians to develop new ways to visualize the PingER data. This would continue after Joao leaves at the end of August. This would use a warehouse to access the data via a Hadoop cluster etc.  Initially this would use the warehouse that Joao has installed at SLAC.  It could also be a warehouse elsewhere since Joao now keeps the FTP hourly data up to date daily and it is possible to just get the new data (see https://confluence.slac.stanford.edu/display/IEPM/Public+FTP+daily+update)

We are thinking of a brain-storming session with the Brazilians, (cc'd) Bebo, Joao and myself. This should be soon, i.e. before Joao leaves. We were thinking of this coming Thursday, would you be available, even better if could be at SLAC. Since they are 4 hours ahead it could be daytime.

We are thinking of an extension to pingtable.pl to make the menu much more powerful, e.g. provide the standard ticks(like we do today)  plus the ability to select times and dates, allow selection of multiple metrics, allow selection of multiple from and to, plus allow selection of multiple visualization (time series, bar charts, maps etc.)  One place to look for graphs that we need today but require exporting to Excel etc to create would be the yearly PingER status reports (e.g http://www.slac.stanford.edu/xorg/icfa/icfa-net-paper-jan15/report-jan15.docx).  Other visualizations would be to make comparisons versus economic (HDI, IDI, corruption ...) or development indices (e.g. correlation plots a la http://www-iepm.slac.stanford.edu/pinger/pinger-metrics-motion-chart.html) or even versus other data available via the semantic web

Accessing data

Hourly data is accessible via FTP at ftp://ftp.slac.stanford.edu/users/cottrell/ as tar files by metric. This is updated daily. See Archiving PingER data by tar for retrieval by anonymous ftp for how to access the complete set. Also can just few (up to 60) days in case you missed some days (see Public FTP daily update). Could use this to populate one's own warehouse.

How do we move forward

Program that supported Renan, Thiago and Joao has finished.

Is SLAC warehouse accessible from Brazil?

  • Use SLAC warehouse
    • Can we keep it running?
    • Need to get into production, document and get performance timings
  • Create own warehouse?

What resources are Brazil thinking of engaging?

  • Students, BS, MS, PhD
  • Courses

What are interests and for whom:

  • Warehouse
  • Semantic web
  • Visualization & analysis

Next meeting

8/12/2016 4:00pm SLAC, 8:00pm Rio.

Also look at meeting Brazil in early September after Joao returns.

 

 

 

 

  • No labels