Design:
http://confluence.slac.stanford.edu/display/IEPM/FTMA+Design
Source:
Pre-Req:
You might need to install the following perl modules if not already installed. Easiest way to do this is to use cpan e.g. "cpan module_name"
Perl:
- Module::Load
- HTTP::Daemon
- XML::SAX
- Config::General
- aliased
- Readonly
- Term::ReadKey
- DBI::DBD
- DBD::mysql
- DBD::SQLite
- Class::Accessor
- Class::Fields
- Params::Validate
- Statistics::Descriptive
- Data::UUID
- IO::Interface
- DateTime
- Error
- Date::Manip (this may require perl upgrade to 5.10 or above, directions)
- Log::Dispatch::FileRotate
- Log::Log4perl
Additionally more components may be required, which may come up when one tries to run the startup script of the service (section "How to run the service"):
- XML::LibXML
Installation:
Theoretically, the service can work with any kind of File Transfer protocol. For test purposes and limiting the large amount of possibilities we use GridFTP as a primary File Transfer Tool.
...
- There are a number of online resources, that can help an administrator, install GridFTP. Some of them are:
- A bug in GridFTP causes intervals to be calculated incorrectly unless you use a very recent version of GridFTP or the subversion trunk version of NetLogger.
- There is generally no restriction on the version of GridFTP to be used, however, it is advised to use the latest one, with minimum security risks.
NetLogger Installation
http://confluence.slac.stanford.edu/display/IEPM/Installing+NetLogger
PerfSONAR_PS FTMA
...
- Create the log Directory
Code Block mkdir /var/log/perfsonar (Create the log directory as is shown by daemon_logger.conf)
- Create a mysql database for FTMA service to interact with:
Code Block mysql -u $USER -p $PASS -e "create database ft_ma"
- Run the SQL Procedure(mysql) on the netlogger database to produce a secondary database.
...
Settings for bin/daemon.conf file
Change "db_name", "db_username" and "db_password" in the following file:
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
max_worker_lifetime 360 max_worker_processes 30 disable_echo 0 ls_registration_interval 60 ls_instance http://192.168.117.129localhost:9995/perfSONAR_PS/services/hLS root_hints_url http://www.perfsonar.net/gls.root.hints <port 9000> <endpoint /perfSONAR_PS/services/FT/MA> service_type MA module perfSONAR_PS::Services::MA::FT <ftma> service_description FT MA service_accesspoint http://localhost:9000/perfSONAR_PS/services/FT/MA enable_registration 0 service_name perfSONAR_PS FT MA ls_registration_interval 60 service_timeout 360 query_size_limit 100 db_host localhost db_username root db_name ft_ma db_password **** db_type mysql </ftma> </endpoint> </port> |
How to run the Service:
Code Block |
---|
* cd bin
* sh FtpMaExecute.sh \--skip-input \|\| \--help
|
- Attempts to install the missing, perl modules.
- Stops any previous running instance of the service.
- Creates a backup of any existing log file.
- Everytime the script executes, it copies the log file from the main log directory to an hourlyfolder in the same directory. The folder is named as: Ftp_MM-DD-YY:HH
- Starts a new Instance of the service by running daemon.pl
Client Application:
The service includes a client tool to do some preliminary testing. This tool is present inside the bin dir as well.
How to run the Service:
Before running the service, we must have the database. So far we only created the database and did not copy any tables/contents. To do this we must have a file that performs sql functions of adding and filling tables. The file should be a part of the FTMA package retrieved via svn checkout and should be placed inside "FileTransfer/perfSONAR_PS-FileTransfer/contrib" directory. The file can used in the following manner (you might need to do sudo):
If mysql not started then start it:
Code Block |
---|
/etc/init.d/mysqld start
|
Load the mysql dump into a database:
Code Block |
---|
mysql -D ft_ma < mysql_backup
|
Once done do the following to start the service:
Code Block |
---|
* cd contrib
* sh FT_MA_Startup.sh \ |
Code Block |
* sh FtpMaExecute.sh --skip-input || \--help |
...
- Attempts to install the missing, perl modules.
...
- Stops any previous running instance of the service.
...
- Creates a backup of any existing log file.
...
- Everytime the script executes, it copies the log file from the main log directory to an
...
- hourlyfolder in the same directory. The folder is named as: Ftp_MM-DD-YY:HH
...
- Starts a new Instance of the service by running daemon.pl
Client Application:
The service includes a basic client tool as wellto do some preliminary testing. This tool is present inside the bin dir.
Code Block perl FTPFT_client.pl
This simple execution will fetch all the metadata from the service, and provide the user with a final output showing all the metadata keys mapped on source and destination ip addresses.
...