Page History

...

• The copying of data to the anonymous FTP server and moving from there to the PingER raw data archive needs to complete beforegetdata.pl starts at 01:32 local time each night at SLAC
• The data is copied from the MA to anonymous FTP incoming spaceat  thenproxy.pl, needs to be scheduled to copy the data from anonymous FTP directory to the directory accessible by the ping_data.pl CGI script.
  • This also has to complete before getdata.pl is scheduled, 
  • proxy.pl takes < 5 seconds to execute. 
  • proxy.pl is therefore currently scheduled to run at01:20amCalifornialocaltime
• Once the move is completed by proxy.pl then getdata.pl can be scheduled to gather and save the selected data from the MAs in the PingER raw data archiveatarchive at:

  /nfs/slac/g/net/pinger/pingerdata/hep/data/<host>/ping-<YYYY>-<MM>-<DD>.txt.gz

  • ThisThis (getdata.pl) takes about 15 minutes.
  • getdata.pl is scheduled to run at01:32amlocaltime.

• The analysis of the hourly data by analyze-all.pl needs to start after getdata.pl has completed. Currentlyanalyze-all.pl starts as a cron job at 55 minutes past 2 am local time at SLAC each morning. Theschedulinggof The scheduling of the jobs at SLAC is shown below:

  20 1 * * * /afs/slac/package/pinger/proxy.pl 32 1 * * * /afs/slac/package/pinger/getdata.pl > /afs/slac/g/www/www-iepm/pinger/slaconly/getdata.err 55 2 * * * /afs/slac/package/pinger/analysis/analyze-all.pl --date 1days #Takes 25:1410/20/2011(55mins 9/21/2011, 70minutes 5/11/2018)

...

The analyzed data from analyze-all.pl is saved in files of the form below, the contents are described in PingER data flow at SLAC:

• See https://confluence.slac.stanford.edu/display/IEPM/PingER+data+flow+at+SLAC for the data flow. The use of the proxy is triggered by the  NODEDETAIL Data Server entry (e.g. proxy=1) to tell ping_data.pl to gather the data from the SLAC anonymous ftp server (via the copy)that is basically acting as a proxy. 

This whole mechanism is also interesting since it could be extended and a step to providing support for the Android PingER project at Amity in Delhi, India which also needs a proxy.

Support for Ping_data.pl for debugging etc.

The CGI script ping_data.pl provides remote access to PingER information at the MA. However, it is not possible to access the Beijing web site (or a PingER Androi) from outside. Thus we need a solution to provide access to the various information that ping_data.pl provides for the MA. This includes (where for example <MA domain might be unimas.my):

1. The pinger.xml file (e.g. http://pinger.<MA domain>/cgi-bin/ping_data.pl?admin_func=pinger_xml)
2. Error log from pinger2.pl (e.g. http://pinger.<MA domain>/cgi-bin/ping_data.pl?admin_func=std_cron_err)
3. Progress log from pinger2.pl (e.g. http://pinger.<MA domain>/cgi-bin/ping_data.pl?admin_func=std_cron_out)
4. Beacon.txt (e.g. http://pinger.<MA domain>/cgi-bin/ping_data.pl?admin_func=beacons)
5. FAQ (e.g. http://www-iepm.slac.stanford.edu/pinger/faq.html)
6. Web server configuration (e.g. http://pinger.<MA domain>/cgi-bin/ping_data.pl?admin_func=httpd)
7. Help for current version of pinger2.pl being run at MA (e.g. http://pinger.<MA domain>/cgi-bin/ping_data.pl?help=1)
8. PingER configuration (e.g. http://pinger.<MA domain>/cgi-bin/ping_data.pl?admin_func=fileLog)
9. Plus access to the actual data for a specified time range.

A possible way to address items 1 thru 8 for Beijing might be to copy the files to anonymous incoming FTP at SLAC on a daily basis at the same time the actual data is copied. Thus the files (updated as of the previous night) would be available for review. Since Bejing has a copy of ping_data.pl this could be used to acces the files so they could be copied.  For the Android some new code would be needed to copy the files to SLAC. 

...

'* If an anonymous ftp server allows anonymous upload, and also world read access, it quickly becomes a haven for stolen software distribution

...