...
Wednesday Aug 20th, 2014 9:00pm Pacific Standard Time, Thursday Aug 21st 2014 9:00am Pakistan time, Thursday Aug 21st, 2014 12:00 noon Malaysian time, Thursday Aug 21st, 2014 1:00am Rio Standard Time.
Attendees
Invitees:
Anjum-, Hassaan Khaliq, Kashif*, Raja*, Johari*, Nara, Adnan, Abdullah, Badrul, Ridzuan, Ibrahim*, Hanan, Saqib*, Adib, Les*, Renan, Bebo
...
Anjum reports (6/23/2014) that "the proposal for conference has been submitted for approval and Pinger has been added in the agenda. Travel expenses for Les and Bebo have also been included in the conference proposal. We are awaiting the proposal approval. 8/16/2014: The faculty management at UM just changed and many matters required urgent attention. Abdullah will be able to update us soon once he gets a chance to see the Vice Chancellor. Once the approval is given the venue for the conference can be at UM or UUM.
As discussed earlier, the only twist here is that Pinger will be seen as a case study for big data. This is good in a sense that people interested in doing research in the domain of big data can deploy pinger monitoring nodes at their respective universities/organizations and in return, play around with the data. We agreed that it looked like the 25th would be a good day for the PingER workshop. Les should be able to make it from Burkina Faso, and Bebo should be able to get back to the US for Thanksgiving. There would be back to back presentation on how PingER gathers, archives data, what data there is, the data types, how to access etc. by Les followed by Bebo on Google Tools for Big data.- Anjum suggested putting together a paper on metrics provided by PingER for Sigmetrix. The due date is in November.
Renan
Luiza has proposed three approaches to provide big data analysis/mining of PingER multidimensional data:
- Conventional. Utilization of Pentaho environment to handle big multidimensional data, which enables utilization of enhanced user interfaces.
- Linked Data. Benchmarking of more sophisticated Triple Stores than the one we use today at PingER LOD (Sesame). Preferably, we should analyze parallel and distributed solutions. CumulusRDF is an example.
- Utilization of Greenplum (http://en.wikipedia.org/wiki/Greenplum). This is an intensive high performance database from EMC with many features such as caching. It is partly from the EMC acquisition of Pivotal. There is also a DBMS called Grindplan that explores lots of features using Pivotal.
- Renan is investigating an alternative to Hadoop, which utilizes a Scientific Workflow Management System and makes use of Map/Reduce paradigm to help both querying and provenance of the Linked Data (RDF) data.
- Ibrahim is investigating an approach that utilizes Hadoop Map/Reduce in a Key/Value store with PingER data in RDF.
Following the last meeting Les made available via FTP examples of PingER data. There are two types:
- Raw data as gathered daily from all the monitoring hosts. This data is ie measured at 30 minute intervals and is quite dirty.
- Analyzed data by metric. This has been cleaned up. Les recommends UFRJ uses the cleaned up data.,
The instructions for the data were also sent as well as size estimates and information on how PingER data has been used.
Following the last meeting Les made available via FTP examples of PingER data. There are two types:
- Raw data as gathered daily from all the monitoring hosts. This data is ie measured at 30 minute intervals and is quite dirty.
- Analyzed data by metric. This has been cleaned up. Les recommends UFRJ uses the cleaned up data.,
The instructions for the data were also sent as well as size estimates and information on how PingER data has been used.
Maria and Renan are advancing in some approaches to deal with PingER data, making it easier to be analyzed and integrated. In particular they have been busy studying and evaluating alternatives, analyzing results from the latest benchmarks on NoSQL (including RDF and graph based storages) database management, distributed processing and mediated solutions over relational databases, and also other experiments with multidimensional analyses on Linked Data. The new students involved are now understanding better the scenario and they have been interacting with Renan regularly.
They have separated the tasks into 2:
- Quantitative analysis on PingER data
- They want to know how PingER has grown, since 1998 until today and how it might be in the next years. By doing this, we may focus on more suitable technologies that deal with scenarios that have a similar profile with PingER.
- Two students are working on this.
- Approaches to handle PingER current data
- Conventional approach – Utilization of Cassandra as back-end database to provide easy crossing of parameters to get PingER data.
- One student is working on this.
- Distributed and parallel approach – Utilization of a data warehouse on top of a distributed file system to provide low latency response to complex queries (like the ones we were not able to do on my previous work). Additionally, how Scientific Workflow Management Systems may help in the ETL process of transforming PingER so it can easily be stored on the data warehouse.
- Renan is working on this.
- Pure RDF approach – Good ways of modeling and natively storing RDF data.
- Maria-Luiza is working on this.
- NoSQL approaches – How other NoSQL DBMS may be adequate for PingER multidimensional data.
- Two students are evaluating existing NoSQL solutions for multidimensional scenarios (such as PingER)
- Key-Value storages for PingER data in RDF
- This is Ibrahim’s work.
- Conventional approach – Utilization of Cassandra as back-end database to provide easy crossing of parameters to get PingER data.
In the end, they want to compare all these approaches.
Maria and Renan are advancing in some approaches to deal with PingER data, making it easier to be analyzed and integrated. In particular they have been busy studying and evaluating alternatives, analyzing results from the latest benchmarks on NoSQL (including RDF and graph based storages) database management, distributed processing and mediated solutions over relational databases, and also other experiments with multidimensional analyses on Linked Data. The new students involved are now understanding better the scenario and they have been interacting with Renan regularly, They are planning to meet up this week to discuss some updates and achievements so far.
UM
The ping server at http://pinger.fsktm.um.edu.my/cgi-bin/traceroute.pl?target=www.slac.stanford.edu&function=ping gives ping server busy at the moment. Please try again later. Some one with access to the web servers should look at that (e.g. review the web logs). Maybe it is being hit with a lot of requests simultaneously. If they are coming from SLAC we may want to look at reflector.cgi.
...