...
- Aqsa and Saba are working together. Their goal is focusing on visualization of PingER historical data using warehouse. The idea is to develop a warehouse in UAF university and make it publicly available. They are 50-60% done with setting up a Hadoop cluster with 3 nodes, 1 master, 2 slaves. She is currently working on importing the PingER data into hdfs on the cluster. they have run some Impala queries on the data and are working on visualization
- Topic: visualization on pingER data (email from Aqsa and Response from Renan)I have studied the google charts as visualization tools but here are some points need to be discussed.1. The idea of applying visualization on Data warehouse (Impala query results) does not seem to be so useful because Data warehouse contains static data and visualization charts will also remains static and need to be updated with the time.
Yes, it needs to be updated with the time. My suggestion is to transform PingER data into data to be inserted into the data warehouse. Myself and some other Brazilian students have developed codes to do this. Such process should occur at least once a day to keep the data warehouse updated daily. This has never been done by any of us.
2. Google charts API cannot integrate with Impala As Impala is hadoop distributed Big Data supported database Google can only integrate with flat files or flat databases like Mysql.
If Google charts API can only read flat files (e.g., CSV files), it is trivial to save a database query result as a CSV flat file that would be consumed by Google charts. Can Google charts generate a plot dynamically after reading a just-created CSV file?
- Tahseen is working on Handling Missing Data in PingER by using Artificial Intelligence. He has executed some Java queries and looking at parallel scientific workflow management using SciCumulus.
- Sara Masood is working on a PingER Internet Performance Monitoring Agent On Android Device. There has been a discussion to pin down more closely what is needed, i.e. a PingER Measurement Agent running on an Android making measurements to the Beacons, and the ability to upload the data nightly. We held a skype group meeting with Topher, Sara, Saqib, Bebo and Les to move forward. See 20160331 Meeting on PingER/Android, also see Discussion on porting PingER to Android. She has what she needs for now.
...
Saqib's old supervisor agreed to appoint a master student to take of PingER in UTM. Saqib has emailed 3/9/2016, no progress 4/6/2016, no progress 5/18/2016..
UM
MYREN
...
/2016. Any progress 6/22/2016.
UM
MYREN
Email 6/15/2016 to Fitzi, pinger.fsksm.utm.my is down
...
.
NUST
Emailed Dr Hassaan 6/15/2016 asking for update and 6/18/2016.
Dr Hassaan Khaliq contacted and said "We shall look into the issues with them" 5/13/2016.
Last update 4/6/2016.
Hassaan's student (Mian Anas Abrar <anas.abrar@gmail.com>) reports: "Sir haven't disabled any of them which i told earlier and actually can't even get back to them because of their wrong contact numbers or their numbers not responding since then. I have tried contacting Dr.Les earlier few time as per your instructions but don't know why i wasn't able to email him. rest the status is actually same nodes being down". We believe we have solved the mail problem.
...
Strikethrough working again in JuneDr Hassaan Khaliq contacted and said "We shall look into the issues with them" 5/13/2016.
PingER at SLAC
Working on the following hosts to be able to gather data
...