Running HPS Java

Before reading these instructions, you will want to follow Installing HPS Java in order to build or obtain an HPS Java distribution jar.

Creating the Distribution Jar

Any references to hps-distribution-bin.jar in command syntax should actually be replaced by the path to the HPS Java distribution jar.  

This jar file should be found in your copy of HPS Java once you have built it locally.

cd hps-java-trunk; ls distribution/target/hps-distribution-3.6-SNAPSHOT-bin.jar

In the commands below, the version, here "3.6-SNAPSHOT", is generally left out for brevity.  When executing commands you need to point to an actual distribution jar that you have built or downloaded.

The jar file contains all of the project's dependencies in a distribution that can be run standalone using the java command.

Running the Jar File

Java jar files can be run in two basic ways.  You can run a default main method using the -jar switch or you can activate any class's main method and supply a list of jar files in the classpath using the -cp switch.

Running the Default Main

Using the -jar switch from the command line will run the main from the class JobManager, which is listed in the manifest inside the jar.

java -jar ./distribution/target/hps-distribution-bin.jar [args]

Without any arguments it will print the command line options and then exit.

The default main method can be used when you are processing LCIO files using XML steering configurations.

Running a Specified Class's Main

You can also run the main method from any class in the jar.

java -cp ./distribution/target/hps-distribution-bin.jar org.hps.evio.EvioToLcio [args]

Some XML steering files have variables that need to be resolved with command line arguments.

If you get "class not found" errors, then either the path to the distribution jar is not correct.  This error may also occur if the class you have specified doesn't exist or does not have a main method.

A class can only be accessed from the command line if it has a valid main method.

package org.example;
 
public class MyClass {
    public static void main(String[] args) {
        System.out.println("hello main");
    }
}

To be directly accessible from the command line, a class must have the following features.

• It must be marked as public or it will not be accessible.
• It must have a public static method called main (as shown above).
• The main method has an array of strings as input (with the command line arguments)

Assuming the above class was bundled inside the distribution (it isn't because it is just a dummy example!), it could be run from the command line as follows.

java -cp ./distribution/target/hps-distribution-bin.jar org.example.MyClass arg1 arg2 [...]

Most classes that implement a command line interface will print out help when run with no options e.g.

java -cp ./distribution/target/hps-distribution-bin.jar org.hps.evio.Evio2Lcio

If implemented by the author, then -h is usually used equivalently to print out a help menu.

java -cp ./distribution/target/hps-distribution-bin.jar org.hps.evio.Evio2Lcio -h

Specific command line syntax is not standardized and will depend on what was implemented in that class by the author.

Steering Files

Steering File Location

The standard location for steering files is steering-files/src/main/resources/org/hps/steering/ in HPS Java.

This folder is organized into the following sub-directories which contain sets of related steering files.

These steering files can be accessed using their path on disk, or they can be referenced as classpath resources.  The exact syntax depends on the command line tool.

For example, a steering file resource might be accessed like this using the job manager.

java -jar ./hps-java-trunk/distribution/target/hps-distribution-bin.jar -x /org/hps/steering/recon/EngineeringRun2015FullRecon.lcsim [...]

The same steering file could also be accessed as a file from the local copy of HPS Java.

java -jar ./hps-java-trunk/distribution/target/hps-distribution-bin.jar hps-java-trunk/steering-files/src/main/resources/org/hps/steering/recon/EngineeringRun2015FullRecon.lcsim [...]

You can use a file rather than a resource if you running a steering file which is not checked into HPS Java or you are using a modified steering file that has not been packaged into the distribution jar.

Steering File Variables

For instance, suppose the XML file has this variable definition.

<driver name="MyDriver" type="org.example.MyDriver">
    <someNumber>${numVar}</someNumber>
</driver>

The var would need to be resolved with a command like this.

 java -cp ./distribution/target/hps-distribution-bin.jar org.hps.evio.EvioToLcio -DnumVar=1234 [...]

All variables defined in the XML files must be resolved from the command line or an error will occur.

System Properties

The command line tools are affected by different environment settings which are set through Java system properties.

This is a table of the system properties that can affect HPS Java.

These values are set as system properties in Java itself.

java -DdisableSvtAlignmentConstants=true [...]

When set in this way, these values are accessible as Java system properties at runtime.

Java Arguments

The JVM accepts a number of command line arguments that alter its behavior.

In particular, you will likely want to increase the default heap space, as the default is too low for running HPS Java.

java -Xmx2g [...]

That will change the heap space to 2 gigabytes.

Also, you may want to run the JVM in server mode.

java -server [...]

The server VM has been optimized for peak operating speed rather than responsiveness.

Command Line Tools

Job Manager

The JobManager runs a series of Drivers defined in an lcsim xml file on input events from one or more LCIO files.  The job manager's is listed in the distribution jar's manifest file, making it the default program which will run when using the -jar switch.

The job manager has the following command line arguments.

[1026 $] java -jar ./hps-java-trunk/distribution/target/hps-distribution-3.6-SNAPSHOT-bin.jar
Feb 18, 2016 4:02:27 PM org.lcsim.job.JobControlManager printHelp
INFO: java org.lcsim.job.JobControlManager [options] steeringFile.xml
usage:
 -b,--batch               Run in batch mode in which plots will not be
                          shown.
 -D,--define <arg>        Define a variable with form [name]=[value]
 -d,--detector <arg>      user supplied detector name (careful!)
 -e,--event-print <arg>   Event print interval
 -h,--help                Print help and exit
 -i,--input-file <arg>    Add an LCIO input file to process
 -n,--nevents <arg>       Set the max number of events to process
 -p,--properties <arg>    Load a properties file containing variable
                          definitions
 -r,--resource            Use a steering resource rather than a file
 -R,--run <arg>           user supplied run number (careful!)
 -s,--skip <arg>          Set the number of events to skip
 -w,--rewrite <arg>       Rewrite the XML file with variables resolved
 -x,--dry-run             Perform a dry run which does not process events

This table explains all of the available options.

The steering file is supplied as an extra argument rather than with a command switch.

Here is an example using many of these command line arguments.

java -jar ./hps-java-trunk/distribution/target/hps-distribution-bin.jar -b -DoutputFile=output -d HPS-EngRun2015-Nominal-v3 -e 100 -i input.slcio \
-n 1000 -p myvars.prop -r -R 5772 -s 10 -w myjob.xml -x /org/hps/steering/dummy.lcsim

This will not actually work (just provided to show all command line options at once).

If a detector name and run number are both supplied as arguments from the command line, the conditions system will be initialized and frozen, meaning that subsequent event numbers from data will be ignored.

EvioToLcio

The EvioToLcio tool converts EVIO files to LCIO and optionally can run a steering file job on the converted events.  This allows the conversion and reconstruction to run in the same job/process for efficiency.

This tool has the following options.

[1037 $] java -cp ./distribution/target/hps-distribution-3.6-SNAPSHOT-bin.jar org.hps.evio.EvioToLcio
EvioToLcio [options] [evioFiles]
usage:
 -b         enable headless mode in which plots will not show
 -d <arg>   detector name (required)
 -D <arg>   define a steering file variable with format -Dname=value
 -f <arg>   text file containing a list of EVIO files
 -h         print help and exit
 -L <arg>   log level (INFO, FINE, etc.)
 -l <arg>   path of output LCIO file
 -m <arg>   set the max event buffer size
 -M         use memory mapping instead of sequential reading
 -n <arg>   maximum number of events to process in the job
 -r         interpret steering from -x argument as a resource instead of a
            file
 -R <arg>   fixed run number which will override run numbers of input
            files
 -t <arg>   specify a conditions tag to use
 -v         print EVIO XML for each event
 -x <arg>   LCSim steeering file for processing the LCIO events

The options are described here.

Here is an example showing how to use most of these command line options.

java -jar ./hps-java-trunk/distribution/target/hps-distribution-bin.jar -b -DoutputFile=output -f evio_files.txt -l lcio_file_output -m 50 \
-v -M -n 1000 -d HPS-EngRun2015-Nominal-v3 -R 5772 -t pass1 -r -x /org/hps/steering/dummy.lcsim

Some of these arguments are similar to the job manager, but the steering file is supplied in a different way.  Evio2Lcio uses a command switch to specifiy the steering whereas the job manager expects this as an extra argument without a command switch.

Run Scripts

Run scripts that wrap a number of HPS Java command line utilities are generated when building the distribution.

After the build completes, they should be found in this HPS Java directory.

distribution/target/appassembler/bin/

For instance, the EvioToLcio utility can be run using this script.

distribution/target/appassembler/bin/evio2lcio.sh [...]

These scripts have several advantages over running the java commands yourself.

• A full classpath is created in the script so it is not necessary to use the distribution jar (meaning a recompilation of distribution is not necessary to pickup changes).
• Reasonable JVM options are included such as setting the min heap size.
• Logging is configured automatically by loading in a default logging properties file.
• Less typing of Java boilerplate commands ("java -jar" etc.)
• You do not need to know the corresponding class's full name to run its command line utility.

Using symlinks to these scripts works fine e.g.

ln -s distribution/target/appassembler/bin/evio2lcio.sh
./evio2lcio.sh

When using these scripts, you cannot directly supply Java system properties, so the JAVA_OPTS variable should be used instead.

export JAVA_OPTS="-DdisableSvtAlignmentConstants=true"

The full list of Java system properties to be used should be included in this variable.

You should not set -Xmx or -Djava.util.logging.config.class as they are already set by the run scripts.

Logging and Debugging

Logging Config

HPS Java uses the built-in logging facilities of Java described in the logging package documentation.

Every package in HPS Java has a default level which is set in the following config file.

hps-java-trunk/logging/src/main/resources/org/hps/logging/config/logging.properties

This config, or any other custom logging config file, can be activated from the command line by setting the config file property to its path.

cd hps-java-trunk; java -Djava.util.logging.config.file=logging/src/main/resources/org/hps/logging/config/logging.properties [...]

You can also activate a Java class which will load this configuration.

java -Djava.util.logging.config.class=org.hps.logging.config.DefaultLoggingConfig [...]

Log Levels

These are the available log levels, in descending order.

Each logger inherits by default from the global log setting.

This is defined in the config file as follows.

# default global level
.level = WARNING

So if a package is not explicitly configured, it will inherit the WARNING log level from the global logger.

Defining Loggers

In the config file, loggers are defined and configured by package rather than class.

# evio
org.hps.evio.level = CONFIG

In the above config, any logger in the org.hps.evio package will have a log level of CONFIG.

A class's logger is typically be defined using the following template.

package org.example;
 
import java.util.logging.Logger;
 
class MyClass {
 
    static private final Logger LOGGER = Logger.getLogger(MyClass.class.getPackage().getName());
 
    void someMethod() {
        LOGGER.info("some method was called");
    }
}

The class uses a package rather than class logger.

The handler and the level should not be assigned in the code, as this will instead be configured in the logging config file.