Page History

...

For example, a request to: glast-ground.slac.stanford.edu/DataCatalog will forward to a tomcat server running on glast-tomcat08.slac.stanford.edu on port 8080, which is behind the fiewall. A request to glast-ground.slac.stanford.edu/Pipeline-II, instead forwards to glast-tomcat09.slac.stanford.edu, also on port 8080. This is primarily to allow us to manage load on a per-application basis.

Installation

nginx is installed on rhel6 by adding the software collections library, then installing the nginx-nginx16 package. When RHEL6 actually installs it, it locates the install in it's own isolated root folder under /opt/rh/nginx16/root. So, for example, if you read documentation which says "edit /etc/nginx.conf", you need to actually edit /opt/rh/nginx16/root/etc/nginx.conf.

Configuration

nginx has strong support for including file(s) at most any level of configuration. We can prevent long configurations, and thus, mistakes, by leveraging this feature. In addition to this, we can separate the configuration out for each domain into it's own file. For this, nginx is currently configured to import all *.conf files from the /etc/conf.d directory (/opt/rh/nginx16/root/etc/conf.d). Organize each domain as it's own .conf file, e.g. glast-ground.slac.stanford.edu.conf. 

Inside each .conf file, we can tell nginx where to route requests based on a URL pattern.

A snippet of the glast-ground.slac.stanford.edu.conf file looks like this, for example:

server {
    server_name glast-ground.slac.stanford.edu;
    listen 80;
    listen 443 ssl;
    include /opt/rh/nginx16/root/etc/nginx/conf.d/ssl/glast-ground.inc;

    # host http://glast-tomcat01.slac.stanford.edu:8080
    location / {
        proxy_pass http://glast-tomcat01.slac.stanford.edu:8080;
        include /opt/rh/nginx16/root/etc/nginx/conf.d/default_reverse_proxy.inc;
    }

    location /Commons {
        proxy_pass http://glast-tomcat01.slac.stanford.edu:8080;
        include /opt/rh/nginx16/root/etc/nginx/conf.d/default_reverse_proxy.inc;
    }


    location /GroupManager {
        proxy_pass http://glast-tomcat01.slac.stanford.edu:8080;
        include /opt/rh/nginx16/root/etc/nginx/conf.d/default_reverse_proxy.inc;
    }
    ...
    ...
    # host http://glast-tomcat08.slac.stanford.edu:8080
    location /DataCatalog {
        proxy_pass http://glast-tomcat08.slac.stanford.edu:8080;
        include /opt/rh/nginx16/root/etc/nginx/conf.d/default_reverse_proxy.inc;
    }
    ...
    ...
 
}

Each location entry has a proxy_pass directive which tells nginx which server the request will be routed too, and, for simplicity, it also includes a snippet from default_reverse_proxy.inc. This snippet just sets some headers which can be used by backend applications to determine information about the request, such as the protocol of the request that came into nginx. If, for example, the protocol was https, a backend tomcat server could see that the original request was secure and modify the servlet container request accordingly to reflect that.

            proxy_set_header        X-Real-IP $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header        X-Forwarded-Proto $scheme;
            proxy_set_header        Host $http_host;

 It's best to organize applications conceptually by backend server (glast-tomcat01, glast-tomcat02, ...), then by URL pattern (/, /Commons, /GroupManager).

There's an additional include at the top of the site's file for SSL settings. When we get an SSL certificate for our other sites, the configuration for each site should be a .inc file under the /etc/conf.d/ssl which we can include in sites to be secured by SSL.

SSL/TLS

nginx will operate as an SSL termination endpoint, and we will attempt to use TLS 1.2+ for all connections across all domains, when possible, to increase security in our scientific web applications.

Tomcat

In the nginx section, a snippet for the reverse proxy configuration of nginx was demonstrated. In order for SSL termination to work, we use those nginx-set headers and tell tomcat to interpret them through a valve, so we need to add a valve to server.xml. The valve which does that is included in Tomcat, and it is the RemoteIpValve.

The configuration for that valve looks like this:

         <Valve className="org.apache.catalina.valves.RemoteIpValve"
              internalProxies="134.79.129.91|134.79.129.92"
              remoteIpHeader="x-forwarded-for"
              remoteIpProxiesHeader="x-forwarded-by"
              protocolHeader="x-forwarded-proto" />

The internalProxies attribute is important; By default, the RemoteIpValve only looks at localhost/internal network IP addresses when it is interpreting those headers. Tomcat will not bother to read the headers from our sca-nginx01/02 servers and will not bother upgrading the requests. The IP addresses in this configuration correspond to sca-nginx01 and sca-nginx02 respectively. Notice that the headers in this valve configuration are the same headers as the nginx snippet.

Taylor

On sca-nginx* machines, /etc/taylor.opts is configured as follows:

...