Obtain software
First you will need the iepm-bw source code. Typically this is obtained from SLAC by logging on (if you have an account) or by an email request to jaraedg@slac.stanford.edu. Either you or he will tar up the entire v3src directory, below are the commands to use:
>cd $bandsrc >tar -cvf v3src .tar >gzip -c v3src.tar /tmp/v3src.tar.gz >ls -l /tmp/v3src.tar.gz -rw-r--r-- 1 cottrell sf 125632972 May 17 12:34 /tmp/v3src.tar.gz >rm v3src.tarcp /tmp/v3 >mv /tmp/v3src.tar.gz ./ >ls -l v3src.tar.gz -rw-r--r-- 1 cottrell sf 125632972 May 17 12:34 v3src.tar.gz >cd /afs/slac/public/users/cottrell >cp $bandsrc/v3src.tar.gz ./
This requires a lot of disk space (e.g. v3src.tar is ~ 180MBytes). It creates a file $bandsrc/v3src.tar.gz which is probably too big to email so it is made available by anonymous ftp. To access it use:
>ftp ftp.slac.stanford.edu Connected to ftp.slac.stanford.edu (134.79.18.30). 220-==================================================================== 220-By using this system you expressly consent to the terms and ...220 FTP server ready. Name (ftp.slac.stanford.edu:cottrell): anonymous331 Anonymous login ok, send your complete email address as your password. Password: 230-==================================================================== 230- 230-Stanford Linear Accelerator Center Anonymous FTP Server 230- 230-====================================================================... ftp> binary 200 Type set to I ftp>cd users/cottrell ftp> ls 227 Entering Passive Mode (134,79,18,30,188,42). 150 Opening ASCII mode data connection for file list dr-xr-xr-x 2 ftp ftp 4096 May 17 19:45 . dr-xr-xr-x 2 ftp ftp 4096 May 14 18:21 .. -r--r--r-- 1 ftp ftp 335 Oct 5 2001 identity.pub -r--r--r-- 1 ftp ftp 304 Feb 9 1996 pgp.publickey -r--r--r-- 1 ftp ftp 125632972 May 17 19:45 v3src.tar.gz 226 Transfer complete.ftp> get v3src.tar.gz local: v3src.tar.gz remote: v3src.tar.gz 227 Entering Passive Mode (134,79,18,30,188,44). 150 Opening BINARY mode data connection for v3src.tar.gz (125632972 bytes) 226 Transfer complete. 125632972 bytes received in 28.4 secs (4.3e+03 Kbytes/sec) ftp>quit >
You will need to unzip and detar the file using: tar -xzvf v3src.tar.gz
Install
Currently there is no automated way to install the IEPM-BW monitoring kit on a host. Here are the steps which should be followed, in order, to install the monitoring host software.This assumes that the requirements for hardware and software configuration have been met. Note that if the default account 'iepm' with home directory '/home/iepm' is used, the source directory '/home/iepm/v3src' is used for the source, '/home/iepm/mysql' is used for the data base, then much of the installation can be done by cutting and pasting from this documentation.
Note that before starting this installation procedure, MySQL must have been sucessfully installed and all the required perl modules must have been installed. I used to provide MySQL installation instructions, but I have found that the installation procedure depends on the specific system you are installing it on. You must use the information in the MySQL documentation to install it. In addition, the installation of the required perl modules can vary system to system, and so in some cases you can use CPAN directly, but in others you may have to do the installations manually.
The following instructions must be performed as root
- Decide what accounts are going to be needed. Assume the account 'cal' will have root privileges and be used for operations requiring root. Assume that the monitoring will run under the non-root access account 'iepm'. Both, for purposes of this documentation are assumed to be in group '<account group>, 'slac'. For this example, I will assume that the home directory for the 'cal' account is '/home/cal', and the home directory for the 'iepm' account is '/home/iepm'. For the sake of consistency beteween monitoring systems, the account 'iepm' with '/home/iepm' is strongly recommended.
- The monitoring host has a name. This is referred to as '<fully qualified monitoring host name>'.
- The monitoring host should be given an <aliasname> that starts with 'iepm-bw'. The idea is to hide the real name of the node, but provide it with a name that can recognize its place in the world. This is important. Examples include:
- iepm-bw.twaren.net
- iepm-bw.caltech.ul-org
- iepm-bw.cern.ch
- iepm-bw.bnl.org
- iepm-bw.snv.ul-org
- iepm-bw.fzk.de
- If the account mysql:mysql has not been allocated, allocate it by:
groupadd -g 20000 mysql useradd -u 20000 -g mysql mysql
- Decide on the directory structure for the mysql data base and the monitoring code. It is best to have the mysql data base on a separate physical disk from the system disk, but if this is not possible, use '/home/iepm/mysql' as the base. Make the mysql directory.
mkdir /home/iepm/mysql chown mysql:mysql /home/iepm/mysql
- Create a configuration file for /etc/my.cnf which MySQL will use when you install the database.
[mysqld] datadir=/home/iepm/mysql socket=/home/iepm/mysql/mysql.sock port=3306 [mysqld_safe] datadir=/home/iepm/mysql socket=/home/iepm/mysql/mysql.sock port=3306 [mysql.server] datadir=/home/iepm/mysql socket=/home/iepm/mysql/mysql.sock port=3306 [client] socket=/home/iepm/mysql/mysql.sock port=3306
- If mysql is already running, it is running with the 'test' database. Kill the mysql daemons.
- Restart MySQL with 'mysqld_safe&'. This will restart the MySQL server with the configuration file /etc/my.cnf.
- Create the data base by executing 'mysql_install_db'.
- Change the ownership of '/home/iepm/mysql' by the command chown 'chown -R mysql:mysql /home/iepm/mysql'.
- Set the database root password with 'mysqladmin -u root password "new password" '.
- You are now ready to create the directories with '/home/iepm/mysql' that must be owned by the iepm account. These directories are used by the iepm monitoring to track the state of the monitoring system. Execute the following commands with 'iepm:slac' replaced with 'slac' replaced by the appropirate group for the 'iepm' account.
# the data directory is where the results from the probes are stored. # These files are picked up by the 'load-datad' daemon and put into the database. mkdir /home/iepm/mysql/data chown iepm:slac /home/iepm/mysql/data # the logs directory contain the log files from each process mkdir /home/iepm/mysql/logs chown iepm:slac /home/iepm/mysql/logs # the pids from the daemons are stored here mkdir /home/iepm/mysql/pids chown iepm:slac /home/iepm/mysql/pids # the files in the keepalives directory are used by the monitoring software to ascertain that # the daemons are running mkdir /home/iepm/mysql/keepalives # prime the pump with: touch /home/iepm/mysql/keepalives/traced.alive touch /home/iepm/mysql/keepalives/pingd.alive touch /home/iepm/mysql/keepalives/bw-synched.alive touch /home/iepm/mysql/keepalives/pathchirpd.alive touch /home/iepm/mysql/keepalives/load-datad.alive touch /home/iepm/mysql/keepalives/load-scheduled.alive touch /home/iepm/mysql/keepalives/owpingd.alive # set the permissions so that the iepm account can access them chown -R iepm:slac /home/iepm/mysql/data chown -R iepm:slac /home/iepm/mysql/logs chown -R iepm:slac /home/iepm/mysql/keepalives chown -R iepm:slac /home/iepm/mysql/pids
- The output from the analysis and reports must go into a directory which is writeable by the 'iepm' account. Generally this is '/home/iepm/public_html'. Make this directory with:
Where <account> is the account your unix id is under. For example, mine is 'cal:sf' (i.e. <account> would be replaced by sf) and our iepm group account was 'iepm:sf'. Depending upon how the /home/iepm/public_html directory was created...for example if I created it via sudo from my cal account, then owner must be set to the 'iepm:<account>' where in SLAC's case <account> is 'sf', but in other cases it mught not be.
mkdir /home/iepm/public_html mkdir /home/iepm/public_html/<aliasname> chown -R iepm:<account> /home/iepm/public_html
- Create a link to this area from /var/www/html
ln -s /home/iepm/public_html/<aliasname> /var/www/html/<aliasname>
- IEPM-BW requires a simple configuration file, '/etc/iepm.cnf'. The contents of /etc/iepm.cnf should contain:
# required iepm configuration parameters # /etc/iepm.conf dataBase = "iepm" dataBaseHost = "<fully qualified monitoring host name>" dataBaseDir = "/home/iepm/mysql" iepmSrcDir = "/home/iepm/v3src" updateAcct = "iepm" acctGroup = "<account group>" readAcct = "readonly" # This is provided as the monitoring host and database host could be different machines # It is not advisable for them to be different. In monitoring a network, one should avoid using the # network for the monitoriong control and data logging. monHost = "<fully qualified monitoring host name>"; aliasName = "<aliasname>"; monHostip = "<monitoring host ip address>"; mysqlport = 3306; #Connie Logg 5/12/04 cal@slac.stanford.edu <- configurator's name and date please
The following installation instructions do not need root privileges can be performed under the iepm account
Installing the iepmtools directory
- 'gunzip's the source distribution
- Uses tar to extract the source into a directory
- Runs ./configure in the directory - if there is no ./configure, a message will be printed and the step is ignored
- Runs make to make the code
- Locates the needed binaries (as specified in the array %clients_servers)
- Copies them to the IEPM-BW bin directory
- Makes any required links.
- After the binaries are copied to the bin directory, and the links are defined, the installation program pauses to allow the user to look at the output from step 1-7 for each tool that is made.
This is a simple script, and may not be extendable to all packages which someone might want to install with it. It should never be run from a cron job, only from the command line. It requires that the IEPM-BW installation be in the standard form. The base directory is '/home/iepm/v3src', the binaries are to be loaded into '/home/iepm/v3src/bin', and the iepmtools directory is '/home/iepm/v3src/iepmtools'.
IEPM-BW Database Setup
A MySQL system has its own set of users which are distinct from the unix users, although the names can be the same. The recommended users are:
- iepm - this is the account the monitoring system runs with
- readonly - this is a readonly account for general access to the data
- apache (or nobody) - the account used by httpd to access the database
- cal (or other user) - the administrator account
- root - used for MySQL system administration
I remind that these accounts are Mysql user accounts and should not be confused with system user account.
The accounts are defined by issuing grants to the account for accessing the data base.
Create a file named pws in /home/iepm/mysql i.e.
touch /home/iepm/mysql/pws
edit the file and put the password of your iepm mysql account in it.
Create the IEPM data base. Log into MySQL with the root account and password and issue the following command:
create database iepm;
The following grant commands can be set up in a text file for 'batch' entry into MySQL, or they can be entered one by one. The root account and password must be known to set up the grants. Make sure, you put the passwords in and don't use the string 'password'. Note the host name component of the grant. All these forms may not be necessary, however I have found that they are for various means of access.
#For mysql administrator ('cyrus' in this case) grant all on *.* to cyrus@localhost identified by 'password' with grant option;
grant all on *.* to cyrus@202.83.167.108 identified by 'password' with grant option;
grant all on *.* to cyrus@iepm-ncp identified by 'password' with grant option;
grant all on *.* to cyrus@'iepm-ncp.ncp.edu.pk identified by 'password' with grant option;
grant file on *.* to cyrus;
# For the iepm account used by iepm to load data into the data base: grant all on iepm.* to iepm@localhost identified by 'password'; grant all on iepm.* to iepm@202.83.167.108 identified by 'password'; grant all on iepm.* to iepm@iepm-ncp identified by 'password'; grant all on iepm.* to iepm@'iepm-ncp.ncp.edu.pk' identified by 'password'; grant file on *.* to iepm;
# For the readonly account (report generation) - no password grant select on iepm.* to readonly@localhost; grant select on iepm.* to readonly@'202.83.167.108'; grant select on iepm.* to readonly@iepm-ncp; grant select on iepm.* to readonly@'iepm-ncp.ncp.edu.pk'; grant file on *.* to readonly;
# For the web server account (may be nobody or apache or other) - no password grant select on iepm.* to apache@localhost; grant select on iepm.* to apache@'202.83.167.108'; grant select on iepm.* to apache@iepm-ncp; grant select on iepm.* to apache@'iepm-ncp.ncp.edu.pk'; grant file on *.* to apache;
Creating Tables
Create the tables in iepm database. The scripts of these tables can be found here. However you can use the SQL queries in the files in create tables directory. Create tables by giving these files as input to mysql. Please check if you might need to change nodes.txt in case you have different paths for ping, grep, traceroute etc. Use the command
mysql -u root -p > file_name Database name
For example, for creating monhost table in iepm database
mysql -u root -p > monhost.txt iepm
It will prompt you for password; enter your mysql root password.
Loading the IEPM Database
Next comes the loading of data in to the database tables. Look at the following links of tables at SLAC IEPM node to have some idea of the data to be inserted in the database tables.
http://www.slac.stanford.edu/comp/net/iepm-bw.slac.stanford.edu/monitoring-hosts.html
http://www.slac.stanford.edu/comp/net/iepm-bw.slac.stanford.edu/NODESPECS.html
http://www.slac.stanford.edu/comp/net/iepm-bw.slac.stanford.edu/targets-iepm-bw.slac.stanford.edu.html
http://www.slac.stanford.edu/comp/net/iepm-bw.slac.stanford.edu/toolspecs.html
http://www.slac.stanford.edu/comp/net/iepm-bw.slac.stanford.edu/plotspecs/pltspecs-entries.html
The scripts for populating the database tables are located in v3src/mysql. You can load tables with data using CSV files. Other way you can also create your own queries, put them in a file and give as input to mysql using the command
mysql -u root -p > file_name databasename
At the moment, I recommend creating table by using queries.
For reference, see the following links
Populating the NODES Table
Populating the GROUP Tables
Populating the MONHOST Table
Populating the TOOLSPECS Table
For more information on the iepm-bw mysql table setup see IEPM Database Table Summary.
Loading the IEPM Database
Currently loading of the monitoring host is done from iepm-bw.slac.stanford.edu.
- Enter the node into the 'NODES' data base.
- Enter the node into the 'MONHOST' data base.
- On iepm-bw.slac.stanford.edu, make a tar file including all the code.
cd /afs/slac/package/netmon/bandwidth-tests/v3src tar -cvf dist.tar -T dist.scripts.list scp dist.tar iepm@<fully qualified monitoring host name>:/home/iepm/dist.tar
- Log onto the new monitoring host node under the 'iepm' account.
- Execute the following to install the source code
mkdir v3src cd v3src tar -xvf ../dist.tar
- Now configure the 'config' directory. The following files may need configuration:
- The 'config' directory contains a sample .cshrc. Modify this to match the node being installed and copy it to '/home/iepm/.cshrc'. This will defined where java is located, and two environment variables "$bandsrc" which points to the v3src directory and "$banddata" which points to the directory where the data and reports are stored. Note that there is a link from '/var/www/html' to this directory.
- 'cleanup-list' - should be configured correctly. If there are processes which are known to hang, they should go in this file. 'bw-cleanup' uses this file to clean processes known to hang (now or in the past). If the process with the name in this list exists longer than the number of minutes indicated on its line, it will be killed.
An example is: # cleanup process list # Connie Logg 5/18/06 cal@slac.stanford.edu post-test,120 gnuplot,5 triganal,120 runperiod,120 pathload_rcv,3 pathchirp_rcv,3 /bin/ping,3 /bin/owping,3 /bin/iperf,3 /bin/tlaytcp,3 /bin/bw-iperf-client,3
- 'owampd.conf.template', 'owampd.limits', and 'owampd.keys' are configuration files for owamp. 'owampd.limits' and 'owampd.keys' should be ok. Look at owampd.conf.template. 'datadir' and 'vardir' are defined to be '/home/iepm/owamp/data' and '/home/iepm/owamp/run' respectively. If this is ok (recommended default), save 'owampd.conf.template' as 'owampd.conf'. Then create the directories:
mkdir /home/iepm/owamp mkdir /home/iepm/owamp/data mkdir /home/iepm/owamp/run
- 'servers.alive' may need to be configured. This is the file that has the information for starting the servers. It is used by 'keep-servers-alive' to start, and if necessary, restart the servers. For a standard monitoring host, this file should be:
The first field is the name of the script. The second field is the directory in $iepmSrcDir where the script resides. The third is a list of parameters to start it with.
bw-iperf-server,bin,-s -p 5000 -w 20M thrulayd,bin, pathload_snd,bin, -q -i map-updated,, pathchirp_snd,bin, owampd,bin, -c /home/iepm/v3src/config -Z >& /tmp/owampd.log &
- Make copies of /etc/my.cnf and /etc/iepm.cnf in the config directory. They are used to provide configuration information to the CGI scripts.
cp /etc/my.cnf /home/iepm/v3src/config cp /etc/iepm.cnf /home/iepm/v3src.config/iepm.cnf.<aliasname>
- Install JAVA, if required. If it must be installed, the recommended installation directory is '/home/iepm'. To install it, copy the 'JDK...bin' in 'home/iepm/iepmtools' to '/home/iepm' and execute it.
- Configure Geoplot.
- Load the NODES and NODESPECS tables, MONHOST table, TOOLSPECS table, PLOTSPECS table, and ther GROUPS and NODEGROUPS tables if you want to use their functionality.
- Once the tables are loaded, run the various reports for them. 'report-nodes' should be run first. If MONHOST is not set up correctly, none of the reports will run. You must fix MONHOST first.
- Other reports to run are:
- report-monhost
- report-nodegroups
- report-nodespecs
- report-plotspecs
- report-table-structure
- report-toolspecs
- Once all the reports run, you are ready to try the probes. 'schedule-load' will force the probes to be scheduled and run. Look in $mysqldb/data and check the data files that are output from the probes running. If they look ok, you can execute the various data loading scripts called by 'load-datad'. These include:
- load-ping-data
- load-trace-data
- load-iperf-data
- load-tlayctp-data
- load-trace-data
- load-pathchirp-data
- load-pathload-data
If all of the above tests work, and the data base is loaded with all probes, nodes, plotspecs, etc. then you can load the crontab. This should start the system up. If there are problems, be sure to check the /home/iepm/mysql/logs directory for the output from the cronjobs.