Page History

Table of Contents

Before You Start

Before reading these instructions, you will want to read Installing HPS Java, which explains how to build or get an HPS Java distribution jar file.

Distribution

...

Jars

The distribution jar file contains all of the project's dependencies in a distribution that can be run standalone using the java command.  The distribution will have "-bin" in the name.

This The jar file should will be found in your copy of HPS Java once after you have built it locally using Maventhe project locally.  You can use a simple ls command to check that it was built correctly.

cd hps-java; ls distributionls distribution/target/hps-distribution-[VERSION]*-bin.jar

The version in subsequent commands will be left out for brevity, and because it will change depending on the version of the distribution you are using.  Of course, when executing commands you need to use the path to an actual jar!

Any references to hps-distribution-bin.jar in command line syntax used on this page should actually be replaced by the path to the HPS Java distribution jarAny references to hps-distribution-bin.jar in command line syntax within these instructions should be replaced by the complete path to the distribution jar which was built.

Running the Jar File

The distribution jar can be run in two basic ways.  You can run a   You may run the default main method using the -jar switch, or you can use the -cp to supply a list of jars in the classpath and activate any  switch can be used along with the path to a class's main method.

Running the

...

Job Manager

Using the -jar switch from the command line will run the main from the class JobManager, which is specified inside a special manifest file in the distribution jar (in case you were wondering how that works) JobManager which processes LCIO files using a steering file configuration.

java -jar ./distribution/target/hps-distribution-bin.jar [args]

Without any arguments, it will print the command line options and then exit.The default main method can be used when you are processing LCIO files using XML steering configurationsand then exit.

Running a

...

Specific Class's Main Method

You can also run the main method from any class in the jar., for example:

java -cp ./distribution/target/hps-distribution-bin.jar org.hps.evio.EvioToLcio [args]

If you get there is a "class not found" errorserror, then either check the following: the correctness of the path to the distribution jar is not correct.  This error may also occur if the class you have specified doesn't exist or does not have a main methodjar, the existence and proper definition of a main method within the Java class, and the read permissions on the jar file.

A class can only be accessed from the command line if it has defines a valid main method.

package org.example;
 
public class MyClass {
    public static void main(String[] args) {
        System.out.println("hello main");
    }
}

To be directly accessible from the command line, a class must also have the following features.

...

Assuming that MyClass was bundled inside the distribution (which it isn't because it is just a dummy example!), then it could be run from the command line via its main method as follows.:

java -cp ./distribution/target/hps-distribution-bin.jar org.example.MyClass arg1 arg2 [...]

Most classes that implement a command line interface will print out a help message when run with no additional options e.g.

java -cp ./distribution/target/hps-distribution-bin.jar org.hps.evio.Evio2LcioEvioToLcio

By convention, the If implemented by the author, then -h switch is also usually used to print out a help menu.

java -cp ./distribution/target/hps-distribution-bin.jar org.hps.evio.Evio2LcioEvioToLcio -h

Specific The specific command line syntax is unfortunately not standardized across all tools in the project and will depend on what was implemented in that class by the particular author.

Running Tests

mvn test -Dtest=ClassNameOfTest[TestClassName]

mvn verify -Dit.test=ClassNameOfTest[TestClassName]

...

The JobManager runs a series of Drivers defined in an lcsim xml file on input events from one or more LCIO files.  The XML steering file may contain lists of input file tags, or they the input files may be (more typically) be supplied with by command line arguments.  The job manager's class is listed in the distribution jar's manifest file, making it the default program which will run when using the java -jar command on the distribution. 

Job Manager Command Line Usage

[1026 $] java -jar ./hps-java-trunk/distribution/target/hps-distribution-3.6-SNAPSHOT-bin.jar
Feb 18, 2016 4:02:27 PM org.lcsim.job.JobControlManager printHelp
INFO: java org.lcsim.job.JobControlManager [options] steeringFile.xml
usage:
 -b,--batch               Run in batch mode in which plots will not be
                          shown.
 -D,--define <arg>        Define a variable with form [name]=[value]
 -d,--detector <arg>      user supplied detector name (careful!)
 -e,--event-print <arg>   Event print interval
 -h,--help                Print help and exit
 -i,--input-file <arg>    Add an LCIO input file to process
 -n,--nevents <arg>       Set the max number of events to process
 -p,--properties <arg>    Load a properties file containing variable
                          definitions
 -r,--resource            Use a steering resource rather than a file
 -R,--run <arg>           user supplied run number (careful!)
 -s,--skip <arg>          Set the number of events to skip
 -w,--rewrite <arg>       Rewrite the XML file with variables resolved
 -x,--dry-run             Perform a dry run which does not process events

Job Manager Command Line Options

...

Switch	Description	Notes
-b	Activates batch mode plotting so plots will not be shown when running job.	This switch can be used to suppress plot display even if an AIDA plotter's show method is called from a driverthe display of interactive plots for batch usage.
-D	Add a variable definition that applies to the input steering file.	All variables in the XML input file must be resolved using this switch or an error a fatal exception will occur.
-d	Set the name of the detector model.	This should be a valid detector model defined in detector-data/detectors of HPS Java. Usually this does not need to be set by the user.
-e	Print out an informational message every N events.	Replaces EventMarkerDriver. When this is left out no information is printed about event numbers event-by-event print outs will display during the job.
-h	Print help and exit.	Using this with other arguments will still cause immediate the help message to print and job will exit afterwards.
-i	Add an LCIO input file.	This switch can be used multiple times to specify more than one input file.
-n	Maximum number of events to run in job.	Leave this unset to To run through all events in the input files; internally -1 is used to indicate unlimited eventsfile(s), do not set this switch.
-p	Load a properties file containing steering file variable definitions.	The input properties file should define name: value pairs for each variable that needs to be defined in the steering file.
-r	Treat the supplied steering as a classpath resource rather than a file.	These are typically resources from org/hps/steering in the steering-files module of HPS Java.
-R	Set the run number to be used when initializing the conditions system.	This has the side effect of "freezing" the conditions system, as the run numbers from the input events will be ignored.
-s	Skip N events at the beginning of the job.	This will not skip N events in each file but the first N events read by the job.
-w	Rewrite the XML steering file with the variables resolved.	This can be used as a simple template engine or for saving the values that were used for a jobto generate steering files without variables in them.
-x	Execute in dry run mode which means actual job will not execute.	Can be This switch is typically used to check for initialization errors in the job.

The steering file is a mandtaory argument, and it is supplied as an extra argument rather than with as a command switch.

If a detector name and run number are both supplied as arguments from the command line, then the conditions system will be initialized and frozen, meaning that subsequent event numbers from data will be ignored.

Here is an example command using many most of these command line arguments switches together.  (This specific command will not work!)

Code Block
language bash
java -jar ./hps-java-trunk/distribution/target/hps-distribution-bin.jar -b -DoutputFile=output -d HPS-EngRun2015-Nominal-v3 -e 100 -i input.slcio \ -n 1000 -p myvars.prop -r -R 5772 -s 10 -w myjob.xml -x /org/hps/steering/dummy.lcsim/dummy.lcsim

The above command is actually nonsense and will not work!  If a detector name and run number are both supplied as arguments from the command line, the conditions system will be initialized and frozen, meaning that subsequent event numbers from data will be ignored.

EvioToLcio

The EvioToLcio tool converts EVIO files to LCIO events format and optionally can may run a steering file job on the in-memory events.  This scheme allows the conversion and reconstruction to run in within the same job /process for efficiency.

EvioToLcio Command Line Usage

...

...

java -jar ./hps-java-trunk/distribution/target/hps-distribution-bin.jar org.hps.evio.EvioToLcio -b -DoutputFile=output -f evio_files.txt -l lcio_file_output -m 50 \
-v -M -n 1000 -d HPS-EngRun2015-Nominal-v3 -R 5772 -t pass1 -r -x /org/hps/steering/dummy.lcsim file1.evio file2.evio file3.evio [etc.]

Some of these arguments are similar to the job manager, but the steering file is supplied in a different way.  Evio2Lcio also uses a command switch to specifiy the steering file, whereas the job manager expects this as an extra argument without a command switch.

...

Run scripts that wrap a number of HPS Java command line utilities are generated when building the distributionare generated when building the distribution.  These can be used to easily run tools in the project without typing the full java command.

After the build completes, they should be found in this HPS Java directory.

...

• A full classpath is created in the script so it is not necessary to use the distribution jar.
• Reasonable JVM options are set such as the min heap size.
• Logging is configured automatically by a default logging properties file.
• Less typing of You do not need to type all the Java boilerplate commands (like "java -jar" etc.).
• You do not need to know the corresponding class's full name in order to run its command line utility.

Using symlinks to these scripts works should work fine e.g.

ln -s distribution/target/appassembler/bin/evio2lcio.sh
./evio2lcio.sh

...

export JAVA_OPTS="-DdisableSvtAlignmentConstants=trueDorg.hps.conditions.enableSvtAlignmentConstants"

The full list of Java system properties to be used should be included in this variable.  You should not set the options -Xmx or -Djava.util.logging.config.class, as these are set by each script.

...

These steering files can be accessed using their path on disk, or they can may be referenced as classpath resources.  The exact syntax depends will depend on the command line tool.

For example, a steering file resource might can be accessed like this using the job manager.:

java -jar ./hps-java/distribution/target/hps-distribution-bin.jar -x /org/hps/steering/recon/EngineeringRun2015FullRecon.lcsim [...]

...

You can use a file rather than a resource if you running a steering file which is not checked into HPS Java or you are using a modified steering file that has not been packaged into the distribution jar.

Steering File Variables

For instance, suppose the XML file has this variable definition.Suppose a steering file contains the following variable definition:

<driver name="MyDriver" type="org.example.MyDriver">
    <someNumber>${numVar}</someNumber>
</driver>

The var variable would then need to be resolved with a command like this.such as:

 java -cp ./distribution/target/hps-distribution-bin.jar org.hps.evio.EvioToLcio -DnumVar=1234 [...]

All variables defined in the XML steering files must be resolved from the command line or an error will occur and the job will terminate.

System Properties

The command line tools may be affected by some Java system properties.

These values are set as system properties in Java itselfin Java itself rather than the command line tool.

java -DdisableSvtAlignmentConstants=trueDorg.hps.conditions.enableSvtAlignmentConstants [...]

When set in this way, these values are accessible to the framework as Java system properties at runtime.

Java Arguments

The JVM accepts a number of command line arguments that alter its behavior.

In particular, when running java you will likely want to increase the default heap space, as the default is generally too low for running HPS Java's full reconstruction jobs.

This command will change the heap space to 2 gigabytes:

java -Xmx2g [...]

...

Also, you may want to run the JVM in server mode.

...

The server JVM has been optimized for performance speed rather than responsiveness, and it should run faster than the default client JVM.

...

HPS Java uses the built-in logging facilities of Java which are described in the logging package documentation.

Every Each package in HPS Java has a default level which is set in the following config file.:

hps-java/logging/src/main/resources/org/hps/logging/config/logging.properties

This config, or any other custom logging config file, can be activated from the command line by setting the config file property to its pathpath as a Java argument.

cd hps-java; java -Djava.util.logging.config.file=logging/src/main/resources/org/hps/logging/config/logging.properties [...]

You can Alternately, you may also activate a Java class which will load this configuration.

...

# default global level
.level = WARNING

So if If a package's log level is not explicitly configured, then it will inherit the default WARNING log level from the global logger.

...

In the config file, loggers are defined and configured by their package rather than class.

...

In the above config, any logger in within the org.hps.evio package will have a log level of CONFIG.

A class's logger is typically defined within the codebase using the following template.

package org.example;
 
import java.util.logging.Logger;
 
class MyClass {
 
    static private final Logger LOGGER = Logger.getLogger(MyClass.class.getPackage().getName());
 
    void someMethod() {
        LOGGER.info("some method was called");
    }
}

...

The handler and the level should not be assigned in within the code, as this will instead be configured in the logging config file. , because changing the level would require recompilation.  Instead, the level should be defined using the config file as described above.