Versions Compared

Old Version 16

changes.mady.by.user Les Cottrell

Saved on

compared with

New Version 17

changes.mady.by.user Les Cottrell

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Hassaan Khaliq?, Muhammad Anas Abrar? (SEECS); Saqib+, Aqsa? (UAF); Johari?, Adnan Khan (UNIMAS);  Badrul,  Ridzuan, Ibrahim (UM); (UTM); Adib?+, Fatima (UUM); Fizi Jalil (MYREN);  Les+, Bebo+, Joao (SLAC).

...

  • slac-pinger/pinger created by topherwhite.  PingER project https://github.com/slac-pinger/pinger
  • Now we have it we can share with Amity to check it works.
    • Bebo reports that someone from Amity requested to be a member of the Android group.

UUM

From Adib:

  • "Since we're still waiting on an account, at SLAC, I asked the student to explore another idea (which is quite relevant) proposed by Prof. Bebo “Creative Adib asked a student (Jaafrau)  to explore  “Creative visualization of PingER data, including rich interaction;”. He has already started looking at the possible attractive/interactive design, at the same time he is working to improve his programming skills."

  • There is another student, (Jaafrau) has a design.  Looking at visualization.

  •   He has come up with a good design, yet the development part needs a lot of work. Adib will ask him to upload his interface and share the link.

  • Adib has a Adib has a student from Thailand who is working with Adib at UUM who  may be interested in installaing a PIngER MA in Thailand. Adib will talk next week after exams to see if can get permissions. 8/3/2016: No update yet, as they have a long holiday in Thailand. Will try to contact him.

  • Adib has written a case study on the Internet performance in ASEAN countries and its implication on the Socio-Economic Indexes. It has been shared with Dr.Les and Prof. Bebo  who have commented. We are looking for where to publish.

...

pinger.unimas.my is working again as of ~ June 18th, No update 5/18/2016also teh traceroute server is working.

Student started last month on March  on the ISO., any progress 8/1/2016?
Adnan has not been assigned any tasks. He is interested in writing papers and will be discussing with Johari on the way forward.Will revisit traceroute server. No progress 3/12/2016
Update PingER Malaysia site, hope to start in April.
Get back to RPi2 at Datacenter no progress 3/12/216

...

  • Aqsa has completed her research work on "visualization on pingER data" and now working on to publish a conference paper. Here are few details of the research work.

    • The query results can be exported as CSV file. I use the CSV file of Query results from Impala to draw Line and Bar charts by using Google API.

    • She has created a Data warehouse on pingER data. First we transform the pingER text files into CSV files. Then i upload these CSV files on HDFS and populate Impala Tables and queries.

    • Line and Bar charts are created on a webpage running or executed by localhost server. it can be updated as the query results varies.

  • Aqsa and Saba are working together. Their goal is focusing on visualization of PingER historical data using warehouse.  The idea is to develop a warehouse in UAF university and make it publicly available. 

  • Aqsa has has put together an  abstract of a conference paper and submitted to "The 3rd IEEE/ACM International Conference on Big Data Science, Engineering and Applications (BDSEA 2016)" (http://computing.derby.ac.uk/bdcat2016/).  "Applying Big Data Warehousing and Visualization Techniques on pingER Data",  Aqsa Hameed, Dr. Saqib Ali, Dr. Les Cottrell and Bebo White, submitted to BDSEA 2016. Authors can access it via:   https://easychair.org/conferences/?conf=bdsea2016  

  • Aqsa and Saba are working together. Their goal is focusing on visualization of PingER historical data using warehouse.  The idea is to develop a warehouse in UAF university and make it publicly available. They are 50-60% done with setting up a Hadoop cluster with 3 nodes, 1 master, 2 slaves. She is currently working on importing the PingER data into hdfs on the  cluster. they have run some Impala queries on the data and are working on visualization
    • Topic: visualization on pingER data (email from Aqsa and Response from Renan)
    I have studied the google charts as visualization tools but here are some points need to be discussed.
    1. The idea of applying visualization on Data warehouse (Impala query results) does not seem to be so useful because Data warehouse contains static data and visualization charts will also remains static and need to be updated with the time.
    Yes, it needs to be updated with the time. My suggestion is to transform PingER data into data to be inserted into the data warehouse. Myself and some other Brazilian students have developed codes to do this. Such process should occur at least once a day to keep the data warehouse updated daily. This has never been done by any of us.
    2. Google charts API cannot integrate with Impala As Impala is hadoop distributed Big Data supported database Google can only integrate with flat files or flat databases like Mysql. If Google charts API can only read flat files (e.g., CSV files), it is trivial to save a database query result as a CSV flat file that would be consumed by Google charts.  Can Google charts generate a plot dynamically after reading a just-created CSV file? 
    Is using a different data visualization library (e.g., D3 https://d3js.org/ ) an option? 

  • Aqsa and  team members are working on creating Data warehouse and we are very close to complete this. Here are some updates.

    • Tehseen qureshi has transformed the pingER text files into binaries and soon he will be able to get CSV files.

    • Saba is working on defining a 4 node cluster.
    • Aqsa has uploaded some sample CSV files on HDFS and run Impala queries as i will get the actual CSV from tehseen these steps are also will be completed

  • Visualization Status 

    • Aqsa has 

    drawn a line chart and bar chart on the data of sample CSV file and i am exploring some more charts to be drawn by using Google API's.

  • Cottrell and Bebo White, submitted to BDSEA 2016. Authors can access it via:   https://easychair.org/conferences/?conf=bdsea2016  

  • Sara Masood - pingER  monitoring host on android . 
    • She is using perl-android-apk Interpreter to run perl on android. It includes execution of 3 perl scripts:
      • getdata.pl, ping_data.pl, pinger2.pl

    • First step is to install pre-requisite i.e. lynx, perl, XML::Simple, make etc. However, she is having permission denied issue when she executes "make install” command at path usr/local/bin because it’s a read only file system. She is  searching for the solution but maybe needs to change the path to /data/local/tmp.

    • Step 2 Is to execute ping _data and cgi-bin, traceroute.pl <http://traceroute.pl/> 

    • Step 3: To execute pinger2.pl <http://pinger2.pl/>  that will start gathering data

...

Saqib will proceed to install Pinger in UAF Pakistan 6/22/2016, any update 8/3/2016?

UTM 

Saqib's old supervisor  agreed to appoint a master student to take of PingER in UTM. Saqib has emailed  3/9/2016, no progress 4/6/2016, no progress 5/18/2016. 

...

2.     For the M.Sc. student in UTM we have to request again to Prof Hanan and Prof Asri and we need support from Prof Johari.

UM

MYREN

NUST

Asked for update on the following 7/30/2016:

...

  • Currently it does not respond to ping pingerqta.pern.edu.pk

  • 274cottrell@pinger:~$ping 121.52.157.157
    PING 121.52.157.157 (121.52.157.157) 56(84) bytes of data.
    ^C
    --- 121.52.157.157 ping statistics ---
    6 packets transmitted, 0 received, 100% packet loss, time 5625ms

  • http://121.52.157.148, responds with the default Apache web page
  • http://121.52.157.148//cgi-bin/ping_data.pl? responds (albeit very slowly) with Get Ping Data from sharedw228.b2bhosting.net  Also there is data. 
    • This is a funny hostname. It comes from the system response to the perl commands:

      • use Net::Domain qw(hostname hostfqdn hostdomain);
        my $hostname = hostfqdn();

  • The following needs to reflect the new IP address at 121.52.157.148. It is very easy to change the IP address once the Measurement Agent is up and running and responding, in fact the PingER meta data at SLAC was changed in February 2015. However the DNS still points to:
    • Output from: /bin/cat /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost 121.52.157.157 pingerqta.pern.edu.pk
     

...