Downloading recent data from the FTP:
There is a file called download.pl in the FTP. This file can be used to download recent data from the FTP.
- Download the file from the FTP;
- Copy download.pl to the location you want to download the files (e.g. /home/Documents/PingER)
- Through your commnad line interface type: perl [path to download.pl]/download.pl -d [number of days]
- -d specifies how many days to go back in time. For example -d 3 will download files related to the previous three days.
- -d specifies how many days to go back in time. For example -d 3 will download files related to the previous three days.
PS: If needed files for the a period larger than 60 days, they must be downloaded manually from the FTP: ftp://ftp.slac.stanford.edu/users/cottrell/
Updating the FTP
The script responsible to update the public FTP is in the pinger machine on the following location:
/afs/slac/package/pinger/ftp-update.pl
To run this script you must use the following command:
$ /opt/TWWfsw/bin/perl /afs/slac/package/pinger/ftp-update.pl
You can SSH pinger machine with the following command:
ssh pinger@pinger
Output:
It represents how many files were missing during the update process.
Cron Job
The script runs every morning at 4AM. To modify the cron job you should edit the trscrontab:
$ trscrontab -e
and edit the following line:
lnxcron;480 00 4 * * * /opt/TWWfsw/bin/perl /afs/slac/package/pinger/ftp-update.pl
Log Files:
Log files with missing files can be found at:
$ /afs/slac/public/users/cottrell/pinger_log_files
Each .txt file contains the names of the files missing in the FTP.
The log files are generated automatically by ftp-update.pl everytime a file is missing in the pingerreports folder.
The files also can be generated by the following script:
$ /opt/TWWfsw/bin/perl /afs/slac/package/pinger/ftp-update-log.pl
Algorithm
Foreach metric Copy correspondent tar file to pinger_temp_files folder Untar the file inside pinger_temp_files folder Check the most recent file in the recently untared folder Copy from pingerreports folder to the pinger_temp_files folder all the correspondent files until the current previous day Tar the updated folder Copy back to the FTP main folder
ftp main folder folder:
$ /afs/slac/public/users/cottrell
pinger_temp_files folder:
$ /afs/slac/public/users/cottrell/pinger_temp_files
pingerreports folder:
$ /nfs/slac/g/net/pinger/pingerreports/hep