Page History

...

• PingER2:
  
  • /afs/slac/package/pinger/pinger2/share/pinger/pinger2.pl: is  the new/current tool to perform ping measurements and store raw data. 
    • At SLAC pinger2.pl is called by /afs/slac.stanford.edu/package/pinger/pinger2/share/pinger/pinger_startup.pl
      This is to check that all is well (pinger2.pl exists, pinger.xml exists and pinger2.pl is not already/still running).
      It executes:

      /usr/bin/perl  /afs/slac/package/pinger/pinger2/share/pinger/pinger2.pl >/afs/slac/package/pinger/pinger2/share/pinger/pingerCronStat.stdout

      Look at afs/slac/package/pinger/pinger2/share/pinger/pingerCronStat.stdout to see how it is doing.
    • It "use"'s 
      

      useXML::Simple;
      useData::Dumper;
      useFcntl;
      useFile::stat;
      useNet::Domain qw(hostname hostfqdn hostdomain);

    • Since it is run under perl, it shows up in top as:
      

      Code Block
      PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 15038 pinger 20 0 3769872 3.5g 11604 R 5.8 7.3 231:43.85 perl 112054 ytl 20 0 202832 10032 1372 S 0.1 0.0 8:52.46 sshd

  • See http://confluence.slac.stanford.edu/display/IEPM/PingER2+at+SLAC]

  • At SLAC stores raw data on nfs at:

    Code Block
    /nfs/slac/g/net/pinger/pinger2/data/ping-<YYYY>-<MM>.txt e.g. /nfs/slac/g/net/pinger/pinger2/data/ping-2021-08.txt lines aree of the form: pinger.slac.stanford.edu 134.79.104.80 www.alief.dz 194.163.133.136 100 1627776002 10 10 148.694 149.496 152.268 1 2 3 4 5 6 7 8 9 10 148 152 148 150 148 148 151 148 148 148

    This file is opened and written to for each set of pings in pingers.pl via the sub log_it in pinger2.pl.  Failure to access the file will result in the following message example in the CronStat.stderr file:
    

    Code Block
    Wed Sep 8 17:09:03 2021 Can't open PINGDATA /afs/slac/package/pinger/pinger2/share/pinger/ping-2021-09.txt: No such file or directory

  • At SLAC:
    • The configuration file is at /u/sf/pinger/pinger.xml
    • The ping commands are:
      

      <PingV4Cmd>/bin/ping -n -w %deadline -c %count -i %interval -s %packetsize %destination</PingV4Cmd>
      with defaults %deadline=30, %ccount 10, %packetsize 100
      <PingV6Cmd>/bin/ping6 -n -w %deadline -c %count -i %interval -s %packetsize %destination</PingV6Cmd>

    • The log file is at /afs/slac.stanford.edu/package/pinger/pinger2/share/pinger/pingerCronStat.stdout   
    • pinger2.pl gets the hosts to ping and other configuration information from the file /afs/slac.stanford.edu/package/pinger/pinger2/share/pinger/pinger.xml
    • To test
      Copy the pinger2.xml file to pinger3.xml. In pinger3.xml strip most of the hosts from <BeaconList>...</BeaconList>, <HostList>...</HostList>
      Copy pinger2.pl to pinger3.pl, in pinger3.pl edit the line:
      my$xmlfile='pinger.xml';           #$cwd/pinger.xml, configuration file for pinger2.pl
      to yield
      my $xmlfile='pinger3.xml';         #$cwd/pinger.xml, configuration file for pinger2.pl 
      In trscrontab -e comment out /afs/slac.stanford.edu/package/pinger/pinger2/share/pinger/pinger_startup.pl and wait for next half-hour interval (by which time the current pinger2.pl should have stopped running).
      Then run cd /afs/slac.stanford.edu/package/pinger/pinger2/share/pinger ; perl -d /afs/slac.stanford.edu/package/pinger/pinger2/share/pinger/pinger3.pl;
• Ping_Data
  • /afs/slac/g/www/cgi-bin/net/offsite_mon/ping_data/ping_data.pl:
  • Represents the web interface(cgi script) used to fetch data from remote monitoring node.
  • zips the data before transfer.
  • It is used by getdata.pl
• getdata.pl
  • Typically called from /afs/slac/package/pinger/getdata_all.pl
  • /afs/slac/package/pinger/getdata.pl
  • queries ping_data http://slac.stanford.edu/cgi-bin/ping_data.pl to fetch data (e.g. from  /nfs/slac/g/net/pinger/pinger2/data)

  • Since 2004 it stores old files as zipped File at

    Code Block
    /nfs/slac/g/net/pinger/pingerdata/hep/data/<host>/ping-<YYYY>-<MM>-<DD>.txt.gz e.g. /nfs/slac/g/net/pinger/pingerdata/hep/data/pinger.slac.stanford.edu/ping-20112022-0311-2223.txt.gz /nfs/slac/g/net/pinger/pingerdata/hep/data/pcgiga.cern.ch/ping-2006-09-28.txt.gz /nfs/slac/g/net/pinger/pingerdata/hep/data/2001:da8:270:2018:f816:3eff:fef3:bd3/ping-2018-09-09.txt.gz

    At the Measurement Agents (MAs) it is typically stored at: /usr/local/share/pinger/data

  • There is also a wrapper to get data from multiple days. It is at /afs/slac/package/pinger/getdata_all.pl
  • See Restoring PingER data for more details.
  • Data from 1997-2003 can be found in 

    or [pinger@pinger ~]$ ls -l /nfs/slac/g/net/pinger/pingerdata/hep/data/pinger.slac.stanford.edu/ping-2022-11-23.txt.gz -rw-rw-r--. 1 pinger iepm 2720699 Nov 24 09:05 /nfs/slac/g/net/pinger/pingerdata/

    . The files are zipped

    hep/data/pinger.slac.stanford.edu/ping-2022-11-23.txt.gz

    At the Measurement Agents (MAs) it is typically stored at: /usr/local/share/pinger/data

  • There is also a wrapper to get data from multiple days. It is at /afs/slac/package/pinger/getdata_all.pl
  • See Restoring PingER data for more details.

  • Data from 1997-2003 can be found in 

    and compressed and contain the gathered data for 100 and 1000 byte pings for each day for all monitors. Code Block157cottrell@pinger:~$ls -l

    /nfs/slac/g/net/pinger/pingerdata/. The files are zipped and compressed and contain the gathered data for 100 and 1000 byte pings for each day for all monitors.

    Code Block

    157cottrell@pinger:~$ls -l /nfs/slac/g/net/pinger/pingerdata/ total 11 drwxrwsr- total 11 drwxrwsr-x 2 6995 iepm 512 Dec 2 2011 1997/ drwxrwsr-x 2 6995 iepm 1024 Jan 25 2005 1998/ drwxrwsr-x 2 6995 iepm 1024 Jan 25 2005 1999/ drwxrwsr-x 2 6995 iepm 1024 Aug 3 2007 2000/ drwxrwsr-x 2 6995 iepm 512 Jan 25 2005 2001/ drwxrwsr-x 2 6995 iepm 512 Jan 25 2005 2002/ drwxrwsr-x 38 6995 iepm 1536 Feb 8 2007 2003/ drwxrwsr-x 6 iepm iepm 512 Jul 20 15:19 hep/ drwxrwsr-x 3 pinger iepm 512 Mar 8 2012 new/ drwxrwsr-x 4 6995 iepm 512 Jan 25 2005 oldftp/ 158cottrell@pinger:~$ls -l /nfs/slac/g/net/pinger/pingerdata/2002/ total 3844288 -rw-r--r-- 1 6995 iepm 349112320 Jan 25 2005 data-2002-01.tar -rw-r--r-- 1 6995 iepm 316467200 Jan 25 2005 data-2002-02.tar -rw-r--r-- 1 6995 iepm 332656640 Jan 25 2005 data-2002-03.tar -rw-r--r-- 1 6995 iepm 326103040 Jan 25 2005 data-2002-04.tar -rw-r--r-- 1 6995 iepm 347064320 Jan 25 2005 data-2002-05.tar -rw-r--r-- 1 6995 iepm 324648960 Jan 25 2005 data-2002-06.tar -rw-r--r-- 1 6995 iepm 319150080 Jan 25 2005 data-2002-07.tar -rw-r--r-- 1 6995 iepm 320245760 Jan 25 2005 data-2002-08.tar -rw-r--r-- 1 6995 iepm 336117760 Jan 25 2005 data-2002-09.tar -rw-r--r-- 1 6995 iepm 335669760 Jan 25 2005 data-2002-10.tar -rw-r--r-- 1 6995 iepm 303858176 Jan 25 2005 data-2002-11.tar -rw-r--r-- 1 6995 iepm 323323904 Jan 25 2005 data-2002-12.tar 168cottrell@pinger:~$cp /nfs/slac/g/net/pinger/pingerdata/2002/data-2002-01.tar /nfs/slac/g/net/pinger/pingerdata/hep/data.unite/ $cd /nfs/slac/g/net/pinger/pingerdata/hep/data.unite/ $tar -xvf /nfs/slac/g/net/pinger/pingerdata/hep/data.unite/data-2002-01.tar cache01.ansp.br/ping-2002-01-01.txt.gz cache01.ansp.br/ping-2002-01-02.txt.gz ... yumj2.kek.jp/ping-2002-01-30.txt.gz yumj2.kek.jp/ping-2002-01-31.txt.gz 182cottrell@pinger:/nfs/slac/g/net/pinger/pingerdata/hep/data.unite$ls 172.23.52.7/ monitor.seecs.edu.pk/ pinger.cdacmumbai.in/ pingerlhr-pu.pern.edu.pk/ aup.seecs.edu.pk/ moore.ece.rice.edu/ pinger.cemb.edu.pk/ pingerpwr.pern.edu.pk/ ... 177cottrell@pinger:/nfs/slac/g/net/pinger/pingerdata/hep/data.unite$cp yumj2.kek.jp/ping-2002-01-31.txt.gz /tmp/ 178cottrell@pinger:/nfs/slac/g/net/pinger/pingerdata/hep/data.unite$gunzip /tmp/ping-2002-01-31.txt.gz 180cottrell@pinger:/nfs/slac/g/net/pinger/pingerdata/hep/data.unite$tail /tmp/ping-2002-01-31.txt yumj2.kek.jp 130.87.34.37 ultra.edu.uy 164.73.128.70 100 1012520610 10 10 379.815 381.366 384.206 0 1 2 3 4 5 6 7 8 9 380.758 380.938 382.205 380.480 380.878 384.206 379.815 381.087 382.547 380.743 yumj2.kek.jp 130.87.34.37 frcu.eun.eg 193.227.1.1 100 1012520610 10 10 364.870 399.626 446.021 0 1 2 3 4 5 6 7 8 9 398.877 395.008 388.069 396.147 416.314 392.862 398.349 399.744 364.870 446.021

    Note that after the above, the raw input data to  wrap-analyze-hourly.pl (see below) for 1998..2003 comes from /nfs/slac/g/net/pinger/pingerdata/hep/data.unite/ rather than /nfs/slac/g/net/pinger/pingerdata/hep/data/.  

    • There is a script /afs/slac/package/pinger/pre2004.pl that will take the data in /nfs/slac/g/net/pinger/pingerdata/<1998..2003> copy it, unzip, and untar into /nfs/slac/g/net/pinger/pingerdata/hep/data.unite/.   

    • There is a second script  /afs/slac/package/pinger/pre2004-hourly.pl that using /afs/slac/package/pinger/analysis/analyze-all.pl reads the raw data from /afs/slac/g/net/pinger/pingerdata/hep/data.unite/ and analyzes and stores the hourly data for the selected years.

• Unite-monthly

  • There is a script /afs/slac/package/pinger/unite-monthly.pl that unites all the raw data for a month into a single file of the form /nfs/slac/g/net/pinger/pingerdata/hep/data/<host>/ping-<YYYY>-<MM>.txt. It is run from a trscronjob at 4:00am each morning. The output data appears as:morning. The output data appears as:

    Code Block
    cottrell@pinger $ head /nfs/slac/g/net/pinger/pingerdata/hep/data/pinger.slac.stanford.edu/ping-2020-03.txt pinger.slac.stanford.edu 134.79.104.80 www.alief.dz 91.121.45.149 100 1583020804 10 10 149.672 149.979 150.649 1 2 3 4 5 6 7 8 9 10 149 149 150 150 149 149 150 149 149 149 pinger.slac.stanford.edu 134.79.104.80 www.ons.dz 196.20.66.100 100 1583020805 10 10 193.218 193.371 193.576 1 2 3 4 5 6 7 8 9 10 193 193 193 193 193 193 193 193 193 193 [pinger@pinger ~]$ tail
    Code Block
    cottrell@pinger $ head /nfs/slac/g/net/pinger/pingerdata/hep/data/pinger.slac.stanford.edu/ping-20202022-0311.txt pinger.slac.stanford.edu 134.79.104.80 wwwmail.aliefneda.dzaf 91117.12155.45192.1495 1001000 15830208041669247047 1011 10 149310.672965 149311.979160 150311.649281 1 2 3 4 5 6 7 8 9 10 14911 149311 150311 150311 149311 149 150 149 149 149 pinger.slac.stanford.edu 134.79.104.80 www.ons.dz 196.20.66.100 100 1583020805 10 10 193.218 193.371 193.576 1 2 3 4 5 6 7 8 9 10 193 193 193 193 193 193 193 193 193 193311 311 311 310 311 311 ... Where for example you can use https://www.epochconverter.com/ to find that the epoch time 1669247047 was equivalent to GMT: Wednesday, November 23, 2022 11:44:07 PM

• Data formats for timeping.pl, pinger2.pl, ping_data.pl, getdata.pl and getdata_all.pl

• Check the data has been gathered:
  • /afs/slac/package/pinger/checkdata_gif.pl calls /afs/slac/package/pinger/checkdata.pl to see if there is gathered data, and then writes out out a summary web page to http://www-iepm.slac.stanford.edu/monitoring/checkdata/
• Analyze
  • There is a group of analysis scripts, that pick up zipped data; do their analysis, aggregate data and prepare web reports.
    • Note that to be included in the analyzed data the host must be in NODEDETAILS.

  • The first script to be executed is wrap-analyze-hourly.pl (which is executed by calling the wrapper analyze-all.pl --date 1days from the trscronjob) which takes as input data the output of getdata.pl (raw data) and from this aggregates the data to by day and writes the latest  to the /nfs/slac/g/net/pinger/pingerreports/hep/<metric>/ directory with the file name <metric><size><by><yyyy><mm>-<dd>.txt.X., this is referred to as the hourly analyzed data. The wrap-analyze-hourly.pl script is run daily from the trscrontab on pinger and by default analyzes the data gathered for yesterday. 
    
    The file names appear as:

...