| Wiki Markup |
|---|
{html}
This outline is based on the <a href="https://confluence.slac.stanford.edu/display/IEPM/Project+Plan">Project Plan</a>.
<h3> Project Progress </h3>
<ol>
<li>
<b>Ontology</b>
<ol>
<li>Goal
<ul>
<li>
To define the vocabulary used by PingER as well as its terms, concepts, taxonomy, and relations between each of the terms.
</li>
</ul>
</li>
<li>Progress
<ul>
<li>
We are basing our ontology on the one proposed by the project MOMENT <a href="https://svn.fp7-moment.eu/svn/moment/public/Ontology/">[1]</a> described by the article <i>Measurement Ontology for IP traffic (MOI); IP traffic measurement ontologies architecture</i>, by the European Telecommunications Standards Institute (ETSI) <a href="http://www.etsi.org/deliver/etsi_gs/MOI/001_099/003/01.01.01_60/gs_moi003v010101p.pdf">[2]</a>.
</li>
<li>
More terms and relations were added and edited on the based ontology. More specifically, metrics were properly added (Directivity, Conditional Loss Probability, Duplicate Packets, Inter-Packet Delay Variation, Out of Order Packets, TCP Throughput, Ping Unpredictability, Ping Unreachability, and Zero Packet Loss Frequency). Statistical measurements were also added (Inter-quartile Range, Median, 75%, 90%, and 95%). Information about the nodes were also added based on the Confluence page specifications <a href="https://confluence.slac.stanford.edu/display/IEPM/PingER+NODEDETAILS">[3]</a> (Site Name, Nick Name, Full Name, Contact for the host, Comments, Trace Server, App User, Ping Size, Ping Server, URL, Project Type, and GMT).
</li>
<li>
Other existing ontologies are also being linked to our ontology:
<ul type="circle">
<li>The Food and Agriculture Organization of the United Nations Geopolitical (FAO) <a href="http://www.fao.org/countryprofiles/geoinfo/geopolitical/">[4]</a> to support the definitions of country, continent, and region. </li>
<li>Friend of a Friend (FOAF) <a href="http://www.foaf-project.org/">[5]</a> to support the description of the Contact for the host.</li>
<li>DBPedia <a href="http://dbpedia.org/About">[6]</a>, the ontology and the resources, will be used to provide more variety of information about geographic locations or any other thing Wikipedia has that can be related to PingER. So it should be possible to make very specific and sophisticated queries.</li>
</ul>
</li>
</ul>
</li>
<li>To do
<ul>
<li>
Test and verify if the ontology works as desired. It is likely that small modifications will be needed. Verify if the instances created with the ontology are consistent.
</li>
</ul>
</li>
<li>Dates
<ul>
<li>
We started building the ontology on May 29, 2013.
</li>
</ul>
</li>
</ol>
</li>
<br/>
<li>
<b>RDF Repository</b>
<ol>
<li>Goal
<ul>
<li>
To establish a good environment for the RDF <a href="http://www.w3.org/RDF/">[7]</a> repository.
</li>
</ul>
</li>
<li>Progress
<ul>
<li>
After analyzing the existing technologies for RDF Repository <a href="http://www.bioontology.org/wiki/images/6/6a/Triple_Stores.pdf">[8]</a>, Jena SDB <a href="http://jena.apache.org/documentation/sdb/">[9]</a> backed with MySQL was chosen to be the storage for all the RDF data.
</li>
<li>
Some instances for the classes of the ontology were created for testing and loaded (via command line) into the repository.
</li>
<li>
Programmatically loaded data into the repository using Jena Api. Successfully inserting either an entire RDF file or a single triple statement into the database.
</li>
<li>
A SPARQL Endpoint <a href="http://www.w3.org/TR/rdf-sparql-query/">[10]</a><a href="http://semanticweb.org/wiki/SPARQL_endpoint">[11]</a> was set up locally (on the developer's machine) and can be used to query the repository. Java Web with Tomcat Servlets is being used for the web server.
</li>
</ul>
</li>
<li>To do
<ul>
<li>
Cross data with multiple data sets. Use SPARQL Queries that link multiple data sets (FOAF, DBPedia, FAO, etc) with our data set. Run these queries on the local (developer’s machine) SPARQL Endpoint.
</li>
</ul>
</li>
<li>Dates
<ul>
<li>
Started on June 5, 2013.
</li>
</ul>
</li>
</ol>
</li>
<br/>
<li>
<b>Generate RDF Data</b>
<ol>
<li>Goal
<ul>
<li>
Generate RDF data from PingER data using the ontology built.
</li>
</ul>
</li>
<li>Progress
</li>
<li>To do
<ul>
<li>
Analyze and study the data from PingER. Find out a good way to write the data in RDF format.
</li>
<li>
Analyze the raw data. Verify the standards and patterns and check if we can use good technologies to extract the data from the raw data, transform it to RDF format, and load them into the RDF repository. Do some search about Kettle/Pentaho ETL tools <a href="http://en.wikipedia.org/wiki/Extract,_transform,_load">[12]</a><a href="http://kettle.pentaho.com/">[13]</a>.
</li>
</ul>
</li>
<li>Dates
</li>
</ol>
</li>
</ol>
<h6>References</h6>
<p>
[1] Project MOMENT Ontologies. Retrieved from <a href="https://svn.fp7-moment.eu/svn/moment/public/Ontology/">https://svn.fp7-moment.eu/svn/moment/public/Ontology/</a> on June 5, 2013.
</p>
<p>
[2] ETSI. Measurement Ontology for IP traffic (MOI); IP traffic measurement ontologies architecture. Retrieved from <a href="http://www.etsi.org/deliver/etsi_gs/MOI/001_099/003/01.01.01_60/gs_moi003v010101p.pdf">http://www.etsi.org/deliver/etsi_gs/MOI/001_099/003/01.01.01_60/gs_moi003v010101p.pdf</a> on June 5, 2013.
</p>
<p>
[3] PingER Node Details. Retrieved from <a href="https://confluence.slac.stanford.edu/display/IEPM/PingER+NODEDETAILS">https://confluence.slac.stanford.edu/display/IEPM/PingER+NODEDETAILS</a> on June 5, 2013.
</p>
<p>
[4] Food And Agriculture Organization Of The United Nations Ontology. Retrieved from <a href="http://www.fao.org/countryprofiles/geoinfo/geopolitical/">http://www.fao.org/countryprofiles/geoinfo/geopolitical/</a> on June 5, 2013.
</p>
<p>
[5] Friend of a Friend Ontology. Retrieved from <a href="http://www.foaf-project.org/">http://www.foaf-project.org/</a> on June 5, 2013.
</p>
<p>
[6] DBPedia. Retrieved from <a href="http://dbpedia.org/About">http://dbpedia.org/About</a> on June 5, 2013.
</p>
<p>
[7] Resource Description Framework (RDF). Retrieved from <a href="http://www.w3.org/RDF/">http://www.w3.org/RDF/</a> on June 5, 2013.
</p>
<p>
[8] Comparison of Triple Stores<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->1. <!--[endif]-->Introduction<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]--><o:p> </o:p></p>
<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->2. <!--[endif]-->Ontology<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->a. <!--[endif]-->Goal<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:59.55pt;text-indent:-59.55pt;
mso-text-indent-alt:-2.85pt;mso-list:l12 level3 lfo2;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->To define the vocabulary used by PingER as well
as its terms, concepts, taxonomy, and relations with each other.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->b. <!--[endif]-->Progress<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->We are basing our ontology on the one proposed
by the project MOMENT[1] described by the article <i>Measurement Ontology for IP traffic (MOI); IP traffic measurement
ontologies architecture</i>, by the European Telecommunications Standards
Institute (ETSI) [2].<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->More terms and relations were added and edited
on the based ontology. More specifically, metrics were properly added
(Directivity, Conditional Loss Probability, Duplicate Packets, Inter-Packet
Delay Variation, Out of Order Packets, TCP Throughput, Ping Unpredictability,
Ping Unreachability, and Zero Packet Loss Frequency). Statistical measurements
were also added (Inter-quartile Range, Median, 90%, 95%, and 75%). Information
about the nodes were also added based on the Confluence specifications [3]
(Site Name, Nick Name, Full Name, Contact for the host, Comments, Trace Server,
App User, Ping Size, Ping Server, URL, Project Type, and GMT).<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Other existing ontologies are also being linked
to our ontology: <o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:92.7pt;text-indent:-18.0pt;
mso-list:l9 level1 lfo3;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Freebase [ref], an open large graph database.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:92.7pt;text-indent:-18.0pt;
mso-list:l9 level1 lfo3;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->DBPedia [6], the ontology and the resources,
will be used to provide more information about any geographic location or any
other thing Wikipedia has that can be connected to the PingER. So it should be
possible to make very specific queries.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->c. <!--[endif]-->To do<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->After doing very complex <i>mashups</i> [ref] using the ontology, we need to verify if the ontology
needs any adjustments.<o:p></o:p></p>
<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->3. <!--[endif]-->RDF Repository<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo4;tab-stops:36.0pt"><!--[if !supportLists]-->a. <!--[endif]-->Goal<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->To establish a good environment for the RDF
repository.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->b. <!--[endif]-->Progress<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->We analyzed the existing technologies to make it
possible. There are well-known Triple Storages such as Jena, Sesame, and
Virtuoso [7]. According to [7], Virtuoso
struggles to load large datasets (>1M triples). Hence, we first decided to
try Jena SDB (with MySQL) and then Sesame Native.<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>i.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Jena SDB results<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->After establishing Jena SDB 1.3.6 with a native
Java API for storing and querying a relational database MySQL [ref], [ref], we
were able to load a reasonable large amount of data to test. <o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->More specifically, data of all nodes and all
yearly measurement data were loaded. However, querying the dataset was very
disappointing. Some very simple queries, such as listing the value of a given metric
in a given year for a given pair of sites took 7 minutes to run. We only had
less than 100K triples. <o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Since we would have much more than 100K triples
and 7 minutes is way too long for such a simple query, we decided to change the
RDF repository. In addition, the results of the articles [7] and [8] show that
there are better alternatives. Therefore, Jena SDB may work very well for smaller
datasets, which is not PingER’s case.<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>ii.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Sesame Native results<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Using the results of [7] and [8], we decided to
try Open RDF Sesame Native 2.7.2. This solution requires a Java Webserver as an
interface with the loading program and the RDF repository itself. We are using
the latest Tomcat version, Tomcat 7 [ref]. The physical layer implementation of
the repository provides customization of indexes, which it is said to significantly
enhance the performance of the queries execution [ref].<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->We are using the indexes <i>spoc,sopc,psoc,posc,opsc,ospc. </i>The order of the indexes for [s]ubject,
[o]bject, and [p]redicate defines the pattern of a search. For example, the
index <i>spoc </i>optimizes a search in
which the subject is the first field of the statement (a statement is composed
by subject, predicate, and object). More specifically, a query that would take
advantage of this index would be: list all instances of metrics (the subject to
be searched for) that are of the “type” “TCP Throughput”. “Type” (the
predicate) and “TCP Throughput” (the object) are given and we are looking for
the metric (the subject). The [c]ontext is a fourth field in “triple”
statements and is used as named graphs in Sesame. Our solution does not
consider contexts, though.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->However, multiple indexes increases the used
disk space (to store the indexes) and it also takes longer to load the data to
maintain the indexes.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Note: [7] points out that the largest dataset
known to be loaded into Sesame Native has only 50M triples. If we extrapolate
this number, there is no guarantee that the performance will remain
satisfactory.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Finally, the exactly same query that took 7
minutes to run, in solution (i) – Jena SDB, took less than 5 seconds using solution
(ii) – Open RDF Sesame Native. The circumstances tested were the same in both
environments, i.e., same computer, same dataset, and same query.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Therefore, we decided to migrate the project to
use Open RDF Sesame Native as RDF Repository.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->c. <!--[endif]-->To do<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:2.0cm">Run very complex queries to test
the performance.<o:p></o:p></p>
<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->4. <!--[endif]-->Accessing the RDF Repository<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo6;tab-stops:36.0pt"><!--[if !supportLists]-->a. <!--[endif]-->Goal<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Establish an easy way to access the RDF data.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->b. <!--[endif]-->Progress<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->We are using Java Tomcat Webserver to host a Sparql
Endpoint [ref]. The HTML page has a text area in which the user will be able to
write Sparql 1.1 queries [ref] to access the RDF repository. By default, the
results are showed in HTML tables, in a JSP page. However, there will be a
combo box to choose in which format the results will be shown. Available
formats will be triples in CSV, JSON, and XML-RDF.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->c. <!--[endif]-->To do<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Use CSS and JavaScript to make it prettier.<o:p></o:p></p>
<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->5. <!--[endif]-->Loading the RDF repository<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo7;tab-stops:36.0pt"><!--[if !supportLists]-->a. <!--[endif]-->Goal<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Generate RDF data using external datasets and PingER
data.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->b. <!--[endif]-->Progress<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->The process of generating RDF data and
populating the RDF repository is divided into subsections:<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo8;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>i.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Set up the prefixes<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Following the RDF standard, all resources are
uniquely identified by an URI [ref]. In order to write less and to provide a
better organization of the statements, it is common to use namespaces
(prefixes) instead of writing absolute URI. For example, it is common to use
the namespace <b>rdfs </b>for the W3C
rdf-schema (<a href="http://www.w3.org/TR/rdf-schema/">http://www.w3.org/TR/rdf-schema/</a>).<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l5 level1 lfo15;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Frequency: If the repository is totally empty,
i.e., it has just been created, the program should first insert the prefixes
into the database. In other words, this should be done only once.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l5 level1 lfo15;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Time to load: Less than 1 second.<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>ii.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Instantiate continents<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->The continents are internally instantiated in
the memory of the program so there is no HTTP GET [ref].<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l0 level1 lfo14;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Frequency: Only once.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l0 level1 lfo14;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Time to load: Less than 1 second.<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>iii.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Instantiate countries<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->The program uses a HTTP GET to access Geonames
API [ref] to retrieve a JSON with data of all countries. Then, for each entry
of the JSON, the program instantiates a country in RDF and loads into the
repository.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Frequency: Only once.<br>
Time to load: Less than 3 minutes.<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>iv.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Generate Node Details JSON<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->A HTTP GET is done on PingER data to retrieve
the nodes and their information (IP, Nickname, Site Name, Latitude, Longitude, etc.).
A JSON is then generated (and written into a file) to be used by the program.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l3 level1 lfo13;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Frequency: Same as the generation of the %NODE_DETAILS
(about each 4 hours).<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l3 level1 lfo13;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Time to generate JSON: Less than 4 seconds.<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>v.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Instantiate Towns and States<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->For each entry in the Node Details JSON, the
program runs HTTP GETs on the Geonames API to try to find the nearest town
(with at least 1000 habitants) and city (with at least 15000 habitants) based
on the latitude and longitude of the site. The state where the town is located (if
applicable) is also instantiated and linked to the town. The program finally inserts
the instantiated data into the repository.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Note: The program also tries to link the
existing found town with other known RDF datasets (DBPedia [ref] and Freebase
[ref]).<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l10 level1 lfo12;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Frequency: <i>Probably</i>
the same as (iv).<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l10 level1 lfo12;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Time to load: ~30 minutes.<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>vi.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Instantiate Schools<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->For each entry in the Node Details JSON, the
program runs HTTP GETs on the DBPedia Sparql Endpoint to try to find a school
whose name is similar to the site’s Full Name, on Node Details JSON.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->If found, an instance of a school is inserted
into the repository. Information about school includes endowment, number of
students (undergrad and postgrad), faculty size, etc.) <o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Note: The program also tries to link the
existing found school with Freebase.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Note2: DBPedia (just like Wikipedia) is not
complete. Thus, for example, information that is available for a very famous
and well-known university may not be available for another university that is
not as famous.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l10 level1 lfo12;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Frequency: <i>Probably</i>
the same as (iv).<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l10 level1 lfo12;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Time to load: ~30 minutes.<o:p></o:p></p>
<p class="ListSubSubtitle" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>vii.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Instantiate Nodes<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->After instantiating Towns, States, Schools,
Continents, and Countries, we can finally instantiate the PingER nodes.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="text-indent:-3.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Again, for each entry of the Node Details JSON,
the program instantiates a Node linking it with its respective Town, School (if
it is one), Country, State (if applicable), and Continent.<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l8 level1 lfo16;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Frequency: <i>Probably</i>
the same as (iv).<o:p></o:p></p>
<p class="ListParagraphLevel2" style="margin-left:121.05pt;text-indent:-18.0pt;
mso-list:l8 level1 lfo16;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Time to load: Less than 3 minutes.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->c. <!--[endif]-->To do<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Make better (more precise and informative)
annotations of how long each step above takes to run. Maybe space utilized
should also be included.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Complex tests should be performed.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Study other forms of retrieving information about
schools.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Study ways to optimize this process. Maybe
parallelizing? <o:p></o:p></p>
<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->6. <!--[endif]-->Loading the RDF Repository
(PingER Measurements)<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo17;tab-stops:36.0pt"><!--[if !supportLists]-->a. <!--[endif]-->Goal<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Although this step remains in the same line of
the previous one (4), it can be separated into a totally different context –
the measurements context, just as a matter of better organization and
understanding. The goal is to generate RDF data from PingER dataset and load it
into the repository.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->b. <!--[endif]-->Progress<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->After loading the repository with the data
specified in the previous step (4), the program needs to load PingER measurement
data.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->The first step in this process is to generate
the Monitoring-Monitored [ref] JSON. The program executes a HTTP GET in <a href="http://www-wanmon.slac.stanford.edu/cgi-wrap/dbprac.pl?monalias=all">http://www-wanmon.slac.stanford.edu/cgi-wrap/dbprac.pl?monalias=all</a>
to retrieve all monitoring nodes. Then, for each monitoring node, another HTTP
GET is executed in <a href="http://www-wanmon.slac.stanford.edu/cgi-wrap/dbprac.pl?monalias=MONITORING_NODE&find=1">http://www-wanmon.slac.stanford.edu/cgi-wrap/dbprac.pl?monalias=<span style="color:windowtext;text-decoration:none;text-underline:none"> </span>EDU.SLAC.STANFORD.N3&find=1</a>,
where the value of <i>monalias </i>is a
given monitoring node, to retrieve the monitored nodes by that monitoring node.
The Monitoring-Monitored JSON is then generated and written into a file.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:92.7pt;text-indent:-18.0pt;
mso-list:l2 level1 lfo21;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Frequency of generating this JSON: <i>Probably</i> the same as (4.iv)<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:92.7pt;text-indent:-18.0pt;
mso-list:l2 level1 lfo21;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Time to generate: Less than 2 minutes.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Having the JSON, the instantiating process
happens according to this approach: For each monitoring node (entry of the
JSON), for each metric, for each packet size, for each time parameter, the
program executes a HTTP GET in the Pingtable [ref] Tab Separated Values (TSV)
file specified by crossing all these parameters. A TSV URL example is of the
form <o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]--><a href="http://www-wanmon.slac.stanford.edu/cgi-wrap/pingtable.pl?format=tsv&file=average_rtt&by=by-node&size=100&tick=allyearly&from=EDU.SLAC.STANFORD.N3&to=WORLD&ex=none&only=all&dataset=hep&percentage=any">http://www-wanmon.slac.stanford.edu/cgi-wrap/pingtable.pl?format=tsv&file=average_rtt&by=by-node&size=100&tick=allyearly&from=EDU.SLAC.STANFORD.N3&to=WORLD&ex=none&only=all&dataset=hep&percentage=any</a><o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Where the parameters:<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:59.55pt;text-indent:-2.85pt;
mso-list:l4 level3 lfo18;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->from – is the monitoring node that pings other
monitored nodes.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:59.55pt;text-indent:-2.85pt;
mso-list:l4 level3 lfo18;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->tick – represents the time aggregation. PingER
has data from 1998 to 2013. At this moment, the project is considering only the
following tick parameters:<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:88.8pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l7 level1 lfo20;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->allyearly<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:88.8pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l7 level1 lfo20;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->allmonthly<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:88.8pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l7 level1 lfo20;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->last365days<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:59.55pt;text-indent:-2.85pt;
mso-list:l4 level3 lfo18;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->size – is the packet size. At this moment, the
project is considering only packets of size 100 bytes.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-left:59.55pt;text-indent:-2.85pt;
mso-list:l4 level3 lfo18;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-family:Symbol;mso-fareast-font-family:Symbol;mso-bidi-font-family:
Symbol">·<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->file – is the network measurement metric. At
this moment, the project is considering only the following metrics:<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Mean Opinion Scores<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Directivity<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Average Round Trip Time<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Conditional Loss Probability<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Duplicate Packets<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Inter Packet Delay Variation<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Minimum Round Trip Delay<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Packet Loss<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->TCP Throughput<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Unreachability<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:92.7pt;margin-bottom:.0001pt;text-indent:-18.0pt;mso-list:l11 level1 lfo19;
tab-stops:36.0pt"><!--[if !supportLists]-->-<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Zero Packet Loss Frequency<o:p></o:p></p>
<p class="ListParagraphLevel1" style="margin-top:0cm;margin-right:0cm;margin-bottom:
0cm;margin-left:59.55pt;margin-bottom:.0001pt;text-indent:-2.85pt;mso-list:
none;tab-stops:36.0pt"><o:p> </o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Note: This process is totally independent of the
previous step (4). Hence it can be independently parallelized. However, if this
step is executed before the previous, the measurement information regarding the
nodes will point to broken links, which is not a big problem and does not
prevent loading measurement data. The broken links will be automatically
repaired when the nodes are successfully instantiated (section 4.vii). <o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->c. <!--[endif]-->Performance Evaluation<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="mso-fareast-font-family:Calibri;mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:Calibri;mso-bidi-theme-font:minor-latin;color:red"><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span></span><!--[endif]-->Last 365 days: For each monitoring node, for
each metric, it is taking around 1 hour to load the data into the repository.
Hence, for 80 monitoring, for the 11 metrics, it is going to take approximately
880 hours (36 days). <span style="color:red">Thus, impracticable amount of time.<o:p></o:p></span></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->d. <!--[endif]-->To do<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->It is taking a huge amount of time to load the entire
data. We must optimize this process.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Measure time and space taken to load the
repository.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Complex tests.<o:p></o:p></p>
<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->7. <!--[endif]-->Rich visualization of the data<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo9;tab-stops:36.0pt"><!--[if !supportLists]-->a. <!--[endif]-->Goal<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Provide smart and useful visualization of PingER
data in RDF format.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->b. <!--[endif]-->Progress<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->We studied to possibility of using 3 APIs:<o:p></o:p></p>
<p class="ListSubSubtitleCxSpFirst" style="margin-left:98.35pt;mso-add-space:
auto;text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo11;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>i.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Google Maps JavaScript API v3 [ref]<o:p></o:p></p>
<p class="ListSubSubtitleCxSpMiddle" style="margin-left:98.35pt;mso-add-space:
auto;text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>ii.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Google Geo Charts [ref]<o:p></o:p></p>
<p class="ListSubSubtitleCxSpLast" style="margin-left:98.35pt;mso-add-space:auto;
text-indent:-98.35pt;mso-text-indent-alt:-18.0pt;mso-list:l6 level1 lfo5;
tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span>iii.<span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Google Public Data Explorer [ref]<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->All of them seem to be very useful and can
provide rich visualizations. (i) and (iii) seem to be the most powerful of
them.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->c. <!--[endif]-->To do<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->We need to think about good and useful mashups to
use within this entire project and showing them on these visualization APIs.
One type of mashup that is being investigated is retrieving data from DBPedia
to cross PingER data with information related to universities (such as
endowment, number of students, if the university is public or private, etc).<o:p></o:p></p>
<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->8. <!--[endif]-->Documentation<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo10;tab-stops:36.0pt"><!--[if !supportLists]-->a. <!--[endif]-->Goal<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Document the whole project.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->b. <!--[endif]-->Progress<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->This Project Progress is being built. It is
being kept in both MS WORD format and HTML (to be used in Confluence).<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->The Confluence page <i>To do-Doing-Done</i> is kept updated more frequently.<o:p></o:p></p>
<p class="ListSubtitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
12.75pt;mso-list:l1 level2 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->c. <!--[endif]-->To do<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->The Java Project should be documented. Javadocs
are supposed to be generated for each class and method.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->An interactive JavaScript document is to be
generated to graphically represent the ontology in order to support users to
use the RDF data.<o:p></o:p></p>
<p class="ListParagraphLevel1" style="text-indent:-2.0cm;mso-text-indent-alt:
0cm;mso-list:l1 level3 lfo1;tab-stops:36.0pt"><!--[if !supportLists]--><span style="font-size: 7pt; font-family: 'Times New Roman';">
</span><!--[endif]-->Installation guide should be written. This
should include how to configure the environment and everything needed to
compile and run the project. Both the RDF Repository (with Tomcat settings) and
the Sparql Endpoint projects should have an installation guide.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="ListTitle" style="margin-left:1.0cm;mso-add-space:auto;text-indent:
0cm;mso-list:l1 level1 lfo1;tab-stops:36.0pt"><!--[if !supportLists]-->9. <!--[endif]-->References<o:p></o:p></p>
<p class="MsoNormal">[1] Project MOMENT Ontologies. Retrieved from <a href="https://svn.fp7-moment.eu/svn/moment/public/Ontology/">https://svn.fp7-moment.eu/svn/moment/public/Ontology/</a>
on June 5, 2013.<o:p></o:p></p>
<p class="MsoNormal">[2] ETSI. Measurement Ontology for IP traffic (MOI); IP
traffic measurement ontologies architecture. Retrieved from <a href="http://www.bioontologyetsi.org/wiki/images/6/6a/Triple_Storesdeliver/etsi_gs/MOI/001_099/003/01.01.01_60/gs_moi003v010101p.pdf">http://www.bioontologyetsi.org/wiki/images/6/6a/Triple_Storesdeliver/etsi_gs/MOI/001_099/003/01.01.01_60/gs_moi003v010101p.pdf</a>
on June 5, 2013.
<<o:p></o:p></p>
<p>
[9
<p class="MsoNormal">[3] JenaPingER SQLNode DBDetails. Retrieved from <a href="httphttps://jenaconfluence.slac.apachestanford.orgedu/documentationdisplay/sdbIEPM/PingER+NODEDETAILS">http>https://jenaconfluence.slac.apachestanford.orgedu/documentationdisplay/sdbIEPM/<PingER+NODEDETAILS</a>
on June 75, 2013.
<<o:p></o:p></p>
<p>
[10] SPARQL
<p class="MsoNormal">[4] Food And Agriculture Organization Of The United Nations
Ontology. Retrieved from <a href="http://www.w3fao.org/countryprofiles/TR/rdf-sparql-querygeoinfo/geopolitical/">http://www.w3fao.org/countryprofiles/TR/rdf-sparql-querygeoinfo/geopolitical/</a>
on June 5, 2013.
<<o:p></o:p></p>
<p>
[11] SPARQL Endpoint
<p class="MsoNormal">[5] Friend of a Friend Ontology. Retrieved from <a href="http://semanticwebwww.foaf-project.org/wiki/SPARQL_endpoint">http://semanticwebwww.foaf-project.org/wiki/SPARQL_endpoint<</a> on June 5, 2013.
<
2013.<o:p></o:p></p>
<p>
[12] Extract, Transform, Load (ETL)
<p class="MsoNormal">[6] DBPedia. Retrieved from <a href="http://en.wikipediadbpedia.org/wiki/Extract,_transform,_loadAbout">http://en.wikipediadbpedia.org/wiki/Extract,_transform,_load</About</a> on June 5, 2013.
<<o:p></o:p></p>
<p>
[13] Pentaho Kettle Project
<p class="MsoNormal">[7] Comparison of Triple Stores. Retrieved from <a href="http://kettlewww.pentaho.com/bioontology.org/wiki/images/6/6a/Triple_Stores.pdf">http://kettlewww.pentaho.com/</a> bioontology.org/wiki/images/6/6a/Triple_Stores.pdf</a>
on June 5, 2013.
<<o:p></o:p></p>
{html} |
Page History
Overview
Content Tools