Mobile Rack Port Forwarding

Concept

The LICOS mobile rack is isolated from SLAC Central Unix systems by a pair of firewalls, one at B33, and one in the rack itself. However, ssh port forwarding can provide the appearance that software clients running on the mobile rack are able to communicate directly with their corresponding server applications on Central Unix hosts. To achieve this connectivity, an intermediate host located between (in a network sense) the two firewalls can be used to chain together a pair of ssh forwarding tunnels for each application / protocol. In the following configuration description, the Central Unix host initiating the SLAC end of the tunnels is 'glast02', the machine behind the ODS-DMZ firewall is 'ods-foo', and the Mobile Rack bastion host is 'lat-dmz01'.

Additional utilities

An open-source utility called 'autossh' is used as a wrapper to invoke the ssh client processes that establish the forwarding tunnels. This utility will restart the client process if it terminates for some reason; in addition, it "piggybacks" a monitoring connection through a parallel tunnel to detect network partition conditions.

Implementation

The forwarding tunnels are initiated using the "remote forward" invocation of ssh (-R option), so that the connections flow across the firewalls in the same sense or direction as an interactive ssh session initiated by a user at SLAC. On glast02, a public/private ssh key-pair has been created for the ISOC group account 'glastops', and the glastops public key added to the authorized_keys2 file of a corresponding 'glastops' account on ods-foo. Similarly, glastops@ods-foo has a public-private key pair, with the public key installed in glastops@lat-dmz01:~/.ssh/authorized_keys2.

On both glast02 and ods-foo, a SysV-style boot-time script is installed in /etc/init.d to invoke the individual autossh-wrapped ssh client instances for each forwarded service. The glast02 version is presented here as an example:

#
# lattunnel     This starts and stops SSH tunnels for the LICOS mobile rack
#
# chkconfig: 345 90 10
# description: lattunnel is a boot-time script that automatically brings up \
#              an SSH tunnel to the LICOS mobile rack through which FASTCopy or \
#              MySQL transfers can occur.  It uses the "autossh" \
#              utility to maintain the tunnelled connection.
#

# set the PATH
PATH=/sbin:/bin:/usr/bin:/usr/sbin:/u/gl/glastops/Prod/bin
export PATH

# start up the tunnels
start(){
    su -s /bin/bash -c "autossh -f -M 40001 -2 -q -n -N -x -R 40000:glast02:40000 ods-foo" glastops
    su -s /bin/bash -c "autossh -f -M 3308  -2 -q -n -N -x -R 3306:glast03:3306 ods-foo" glastops
    su -s /bin/bash -c "autossh -f -M 20023 -2 -q -n -N -x -R 20022:centaurusa:22 ods-foo" glastops
}

# stop the tunnels
stop(){
    killall autossh
}

# restart the tunnels
restart(){
    stop
    start
}

# show some sort of status
status(){
    ps -efwww | grep -v grep | grep autossh
}

# see how we were called
case "$1" in
    start)
        start
        ;;

    stop)
        stop
        ;;

    restart|reload)
        restart
        ;;

    status)
        status
        ;;

    *)
        echo $"Usage: %0 {start|stop|status|restart|reload}"
        exit 1
esac

The following table describes the forwarding configuration for the various applications. A graphical depiction is also available in either SVG or Visio format.

Application-specific notes

• FASTCopy: Since lat-dmz01:40000 is effectively redirected to glast02:40000, the fcopyd daemon that would normally listen for incoming FASTCopy transfers must be disabled on lat-dmz01. Thus it is not possible to perform a FASTCopy transfer to lat-dmz01, only from lat-dmz01 to glast02.

• MySQL: The LICOS Electronic Log (ELog) application uses a MySQL database running local to the mobile rack on lat-dmz01. This database's mysqld listens on the "standard" MySQL port 3306. Therefore, port 3307 on lat-dmz01 is redirected to glast03:3306 for access to the MOOD and FMX databases.

• CVS: Access to the LAT Flight Software CVS repository in support of the CMX build tool requires some per-user configuration in order to make use of the tunnel to centaurusa:
  • Each user must create ssh public-private key pairs for their mobile-rack account, and install their mobile-rack public keys in user@centaurusa:~/.ssh/authorized_keys and user@centaurusa:~/.ssh/authorized_keys2.
  • Each user must export CVSROOT=:ext:lat-dmz01:/nfs/slac/g/glast/flight/archive in their mobile-rack account.
  • Each user must add the following lines to user@lat-dmz01:~/.ssh/config:

    lat-dmz01*
          Port 20022
          ForwardX11 no