Page History

...

rrdtool create sss.rrd \--start 1174347943 \--step 1800  DS:uptime:GAUGE:3600:0:UDS:loss:GAUGE:3600:0:UDS:median:GAUGE:3600:0:
UDS:ping1:GAUGE:3600:0:UDS:ping2:GAUGE:3600:0:
UDS:ping3:GAUGE:3600:0:UDS:ping4:GAUGE:3600:0:UDS:ping5:GAUGE:3600:0:UDS:ping6
:GAUGE:3600:0:UDS:ping7:GAUGE:3600:0:
UDS:ping8:GAUGE:3600:0:UDS:ping9:GAUGE:3600:0:UDS:ping10:GAUGE:3600:0:URRA:AVERAGE:
0.5:1:5760
 

rrdtool update /nfs/slac/g/net/pinger/pingerdata/hep/RRDCache/pinger.slac.stanford.edu/pinger-slac-stanford-edu_134-79-206-144_100_10.sss.rrd 1174477592 :      
NaN:NaN:0:0.0251:0.0248:0.0249:0.025:0.025:0.0251:0.0251:0.0251:0.0252:0.0254:0.026

...

0254:0.026

Architecture

The scripts that produce the Pinger data files and the RRD files are placed at : /afs/slac.stanford.edu/package/pinger/smokeping (Added by AmberZeb 03/07/2012)

The scripts that produce the RRDs are located in /afs/slac/g/scs/net/netmon/pinger_smokeping/bin.
The CGI scripts are located in /afs/slac/g/www/cgi-wrap-bin/net/shahryar/smokeping.

...

monitor.niit.edu.pk
monitor.niit.edu.pk_monalisa.niit.edu.pk_1000_10
monitor.niit.edu.pk_monalisa.niit.edu.pk_100_10
monitor.niit.edu.pk_www.niit.edu.pk_1000_10
****

The four stars separate the information of one monitoring site from the other. The benefit of this srcdest.txt file is that a list of keys against which the converted data is available, is formed so that when a user clicks on one item in the list the graph.cgi can plot the converted data against the key. Conversely if the file was not there, and the user entered his choice of monitoring site, remote site, ping packet site, number of ping packets hoping that the data for this combination is present, he might be  wrong. It's easy for the user to select what exists, then to let him guess by allowing him to make the selection.

...

Detailed description can be found at http://maggie.niit.edu.pk/newwebsite/federation_projectdesc2.html MAGGIE - SEECS - Smokeping.

Performance Analysis

 In order to get  list of unique Source Destination pairs (links) SelectSrcDest1.pm used to traverses through all the raw pinger files for the last 120 days and generated the list. This was extremely expensive in terms  of time consumed so we needed to reduce this. This was reduced to one day which made the code 120 times more faster.

...

CreateRRD took:38 wallclock secs (19.76 usr  3.99 sys \+  2.57 cusr  0.82 csys = 27.14 CPU)
CreateRRD took:38 wallclock secs (20.06 usr  3.39 sys \+  2.46 cusr  0.84 csys = 26.75 CPU)
CreateRRD took:67 wallclock secs (21.25 usr  9.75 sys \+  2.49 cusr  1.03 csys = 34.52 CPU)
CreateRRD took:66 wallclock secs (21.19 usr  9.06 sys \+  2.73 cusr  0.90 csys = 33.88 CPU)
CreateRRD took:68 wallclock secs (20.97 usr  9.47 sys \+  2.78 cusr  0.74 csys = 33.96 CPU)
CreateRRD took:68 wallclock secs (20.80 usr  9.40 sys \+  2.44 cusr  0.96 csys = 33.60 CPU)
CreateRRD took:23 wallclock secs (19.19 usr  0.86 sys \+  2.27 cusr  0.91 csys = 23.23 CPU)
CreateRRD took:23 wallclock secs (19.09 usr  1.13 sys \+  2.55 cusr  0.92 csys = 23.69 CPU)
CreateRRD took:66 wallclock secs (21.13 usr  8.45 sys \+  2.53 cusr  0.93 csys = 33.04 CPU)
CreateRRD took:67 wallclock secs (21.24 usr  9.61 sys \+  2.23 cusr  0.91 csys = 33.99 CPU)

...

    I     I created the same 10 RRD files on local disk, clearly its less expensive to write to local disk.

...

The code created new RRD files based on unique source, destination and number of packets sent. Since the number of packets sent is variable for each day, it created a new RRD file each day which is very expensive. An average of 100 additional RRDs were created every day which costs 10 hours time to run. I have set the value of the number of packets sent to 10, so now  only the previous RRDs will be updated and no new RRD will be created. So this will cause the script to run much faster( will takes on average 10 hours less to run daily).and no new RRD will be created. So this will cause the script to run much faster( will takes on average 10 hours less to run daily).

On a pinger machine with 2 X 2.00 GHZ processors it takes 95 secs to create a new rrd file and few seconds to update. Total files to be created/updated are 630. So, for the first time a rough estimate is 16.25 hrs to complete the job

Cron Jobs

I have run SelectSrcDest1.pm as a cron job which is supposed to run every 24 hours to generate the rrd files for unique source destination pairs. The cron job is set as a TrsCron tab and is supposed to run at 12:00 am midnight on tersk08 machine.

...

 Here is an example of a congested link in China  (www.hz.zj.cn). You can clearly see the day night effect here and huge losses. In first graph shows the data for the last two days in this case. (By default it shows the last 120 days and can be changed by selecting the date). The second one shows the data for the last 10 days and the third one shows the data for the last 30 days.

Interpreting the Graph

Smoke colors