...
Volume of analyzed data May , 2015. There is a script to assist with getting the data volumes at ~cottrell/bin/sumdir-regexp.pl.
Saving Raw Data
Gathered /raw data for 1998-2003 is 1998-01 - 2007-07 is saved in
Code Block |
---|
/nfs/slac/g/net/pinger/pingerdata/hep/data |
...
/<YYYY>/<host>/ e.g. /nfs/slac/g/net/pinger/pingerdata/hep/data/2007/brunsvigia.tenet.ac.za/ With files of the form: -rw-rw-r-- 1 pinger iepm 584920 Jan 2 2007 ping-2007-01-01.txt.gz |
Gathered data from 2004 2007-07 onwards is saved in:
Code Block |
---|
/nfs/slac/g/net/pinger/pingerdata/hep/data/<host>/ping-<YYYY>-<MM>-<DD>.txt.gz e.g. /nfs/slac/g/net/pinger/pingerdata/hep/data/pinger.slac.stanford.edu/ping-2011-03-22.txt.gz /nfs/slac/g/net/pinger/pingerdata/hep/data/pcgiga.cern.ch/ping-2006-09-28.txt.gz |
Code Block |
---|
/nfs/slac/g/net/pinger/pingerdata/hep/data.unite/<host>/ping-<YYYY>-<MM>-<DD>.txt.gz
with files of the form:
-rw-rw-r-- 1 cottrell sf 52464462 Jul 20 18:11 ping-2007-04.txt |
Code Block |
---|
226cottrell@pinger:~$bin/pinger-tar.pl -p true -g true | tee pinger-tar CopyingTime so far=7688 secs cmd=rm /afs/slac/public/users/cottrell/vle.iiu.edu.pk/ping-* Fri Nov 13 10:14:50 2015 Took 1 secs, for rm /afs/slac/public/users/cottrell/vle.iiu.edu.pk/ping-* Copy data for host(010/1199/210)=drwxrwsr-x 2 iepm cottrell iepm 1126431744 Jul 1421 1605:4023 111wanmoninst1.68cern.102.40ch debug=-10, production=true gzip:Time so far=7689 secs cmd=cp -p /nfs/slac/g/net/pinger/pingerdata/hep/data/wanmoninst1.cern.ch/ping-* /afs/slac/public/users/cottrell/111.68.102.40/ping-2010-04.txt: unknown suffix -- ignored ... 159559781 bytes inwanmoninst1.cern.ch/ Fri Nov 13 10:15:28 2015 Took 38 secs, for cp -p /nfs/slac/g/net/pinger/pingerdata/hep/data/wanmoninst1.cern.ch/ping-* /afs/slac/public/users/cottrell/wanmoninst1.cern.ch/ Time so far=7727 secs cmd=gunzip -f /afs/slac/public/users/cottrell/wanmoninst1.cern.ch/ping-* Fri Nov 13 10:17:06 2015 Took 98 secs, for gunzip -f /afs/slac/public/users/cottrell/wanmoninst1.cern.ch/ping-* Time so far=7825 secs cmd=tar -Pczf /afs/slac/public/users/cottrell/111wanmoninst1.68cern.102.40ch/111wanmoninst1.68cern.102.40ch.tar(1/1/210) Copying data for host(1/2/210)=drwxrwsr-x 2 pinger iepm 1536 Jul 14 18:19 140.105.28.27 debug=-1, production=true 5041746 /afs/slac/public/users/cottrell/wanmoninst1.cern.ch/ping-*.txt Fri Nov 13 10:18:04 2015 Took 58 secs, for tar -Pczf /afs/slac/public/users/cottrell/wanmoninst1.cern.ch/wanmoninst1.cern.ch.tar /afs/slac/public/users/cottrell/wanmoninst1.cern.ch/ping-*.txt Fri Nov 13 10:18:04 2015: 168840707 bytes in /afs/slac/public/users/cottrell/140wanmoninst1.105cern.28.27ch/140wanmoninst1.105cern.28ch.27.tar(211/2199/210), time so far=7883secs |
This saves about 200 tar files in ftp://ftp.slac.stanford.edu/users/cottrell. They are for data from 2007-07 forwards. They appear as directories with the directory name being the name of the PingER monitoring agent (MA) host. For example
The files are tar files and compressed. Together they occupy about 70GBytes compressed or about 300GBytes uncompressed.
Information on each host MA can be found by going to http://www-iepm.slac.stanford.edu/pinger/pingerworld/all-nodes.cf which can be require'd in a perl script, or for currently working MAs http://www-iepm.slac.stanford.edu/pinger/pingerworld/slaconly-nodes.cf or in the XML file http://www-iepm.slac.stanford.edu/pinger/pingerworld/rss.xml .