Installing HPS Java

Quick Build Instructions

Assuming you have internet access and the right command line tools installed already, this should do it.

cd /my/work/dir
git clone https://github.com/JeffersonLab/hps-java.git
cd hps-java
mvn install -DskipTests
java -jar distribution/target/hps-distribution-[VERSION]-bin.jar

Replace [VERSION] with the actual release that was built.

The last command will check that the distribution was installed correctly.

That's it!  If you need help installing the build tools, then continue reading the next section.

Preliminary Tools

Before beginning the installation process, you will want to have available the following tools on your machine first.

Unix (optional)

Being Java software, the project is cross-platform and should run on Linux, OSX and Windows. 

The setup instructions assume a Unix-like environment using the bash shell. 

Windows users should be able to execute the equivalent commands on their system to perform the installation successfully.

Java

The software is currently compatible with Java versions 1.8 and greater.  It will not currently build with Java 1.7 or other earlier versions.  You need to download the "Java Platform (JDK)" and not just the JRE.

The bundle for your system should be downloaded from the Java SE downloads site.  Then you need to follow the setup instructions listed there.

You may want to add lines like the following to your shell initialization script, especially if Java is not installed in a standard system location like the /usr directory.

export JAVA_HOME=/path/to/java/jdk
export PATH=$JAVA_HOME/bin:$PATH

#for example on rhel6-64 machine @ slac:
export JAVA_HOME=/afs/slac/package/java/amd64_linux/jdk1.8.0_101/
export PATH=$JAVA_HOME/bin:$PATH

This will make sure that the shell can find the Java commands such as java and javac that will be needed for the installation.

There have been problems when using the OpenJDK distribution of java to compile HPS Java.  Therefore, make sure that the distribution of java is Oracle's version.

Once the Java installation is completed, you should make sure that the compiler is available from the command line.

[$] javac -version
javac 1.8.0_101

If this command prints "command not found", the current PATH variable probably does not contain the Java bin directory or the location was set incorrectly.

Maven

Maven is a Java based build tool used to create a runnable version of HPS Java on your machine.

You will need to get Maven from the Maven Download Site and save (for instance) the tar.gz file to your machine.

Then you can set it up as follows.

tar -zxvf apache-maven-3.2.3-bin.tar.gz
export MAVEN_HOME=$PWD/apache-maven-3.2.3
export PATH=$MAVEN_HOME/bin:$PATH

Now you should check that the mvn command works.

[$] mvn -version
Apache Maven 3.2.3 [etc.]

If this command does not work, then you should check that the PATH variable is set correctly.

Git

Git is the source control system system used to obtain the source code from a remote server.  You will need the Git client tool in order to checkout the HPS Java code.

This is an example of using your package manage to install git, assuming it is not already present.

sudo yum install git

Windows users may also want to install a graphical client such as TortoiseGit.

Build Instructions

Obtaining the Source Code

First, you are going to want to create some kind of work directory.

mkdir /work/hps
cd /work/hps

You will now use the git command to checkout the "master", or main development branch, of the HPS Java Project.

git clone https://github.com/JeffersonLab/hps-java.git

This may take awhile as the source code is downloaded from the server.

Now you can cd into hps-java-trunk to execute build commands.

Building the Maven Project

From the hps-java-trunk directory, you can execute the following command to build the project.

mvn -DskipTests

Especially the first time this is done, it may take quite a long time to bootstrap Maven and download all jar dependencies.

The directory containing the Maven jar cache (or repository) is by default in ~/.m2/repository but can be changed using a command line option or user setting.

mvn -Dmaven.repo.local=/path/to/my/repo

At JLab, you might want to use a directory on the work disk for the repo so your home directory does not get filled up.

Building a Maven Module

The list of available modules is available from the Project Modules Webpage on the HPS Java site generated by Maven.

To build a single module, assuming you already have a local copy of the SVN trunk, you just need to go to its sub-directory and type the appropriate Maven command.

For instance, this command will build the tracking reconstruction module.

cd hps-java/tracking
mvn -DskipTests

This will install that module's jar into your local Maven repository.  In order to include these changes in the distribution jar, you also need to build the distribution module, also.

cd hps-java/distribution
mvn

Checkstyle Errors

When introducing local changes, you may get error messages like "You have 2 Checkstyle violations." and the build will fail.

There are currently two checkstyle rules which run with the build, so you will need to determine which is relevant by reading the full error messages.

Firstly, tabs are disallowed completely in all Java code.  

An error message like this will show if you have tabs in one of your files.

[ERROR] src/main/java/org/hps/job/JobManager.java:[29,1] (whitespace) FileTabCharacter: File contains tab characters (this is the first instance).
...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (default) on project hps-job: You have 1 Checkstyle violation. -> [Help 1]

You can fix up your source code by using the Source > Format command in Eclipse, or you may prefer to use a command line Unix tool like expand to fix these issues.

Secondly, you are disallowed from committing Java files that have unused imports.

Error messages such as the following will print to the console if unused imports exist in one of your files.

[ERROR] src/main/java/org/hps/job/JobManager.java:[14,8] (imports) UnusedImports: Unused import - org.apache.bcel.Constants.
...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (default) on project hps-job: You have 1 Checkstyle violation. -> [Help 1]

Unused imports will be underlined with a yellow line in Eclipse and can be removed by using Source > Organize Imports which will strip them out.

Tests

There are two types of tests in the HPS Java project: unit tests and integration tests.  

Unit tests will run by default if not excluded in the project's pom.xml file and if not disabled with the -DskipTests flag to Maven.  These tests typically cover a single class in the project and run quickly.

A specific test can be run from its submodule using a command like the following.

cd hps-java/conditions; mvn test -Dtest=DatabaseConditionsManagerTest

Integration tests cover multiple classes or packages and may take several minutes or more to run.

All the integration tests are in the integration-tests module and can be run as follows.

cd hps-java/integration-tests; mvn verify

The integration tests can also be activated individually.

cd hps-java/integration-tests; mvn -Dit.test=ReconSteeringTest verify

When developing, you will typically want to check periodically that relevant integration tests pass.

Integrated Test Output

The integration tests run the full reconstruction, plus simple analysis modules. These produce histograms, which are then compared to a set of reference histograms. Any differences cause the tests to fail, requiring human intervention to decide whether the changes are acceptable. In that case the reference histograms will be updated to reflect the new state of the reconstruction/analysis.

Running the Distribution Jar

The primary tool produced by the installation is a standalone, runnable jar file that can be used along with XML steering files to run reconstruction or analysis jobs.

In order to test the installation, you may execute a command such as the following, where VERSION is replaced by your actual version from the last successful build.

java -jar ./distribution/target/hps-distribution-[VERSION]-bin.jar

The Maven tool will also install this file into your local repository. 

Here is an example of running the bin jar from its location in the local repository directory.

java -jar ~/.m2/repository/org/hps/hps-distribution/3.0.3-SNAPSHOT/hps-distribution-3.0.3-SNAPSHOT-bin.jar

Running HPS Java describes in much greater detail how to run various command line tools in HPS Java using the distribution jar.

Using Distribution Jars from Nexus

As an alternative to the above instructions for building the software locally, you may also download and run a distribution jar from the Nexus repository.

This search URL will show the list of releases of the distribution jar that can be downloaded.  Select the version that you are interested in and simply click on the "bin jar" link for the distribution version you want to download, save it to your machine, and run it using the previous instructions.

You may also directly download a jar using (rather ugly) URLs like this which should work in tools like wget.

When using wget you need to enclose the URL in quotes.

wget "http://srs.slac.stanford.edu/nexus/repository/lcsim-maven2-snapshot/org/hps/hps-distribution/4.4-SNAPSHOT/hps-distribution-4.4-20190612.032534-24-bin.jar" -O hps-distribution-4.4-bin.jar

The above example command will save the latest 4.4 distribution jar as the file hps-distribution-4.4-bin.jar in the current directory.