Page History

Overview

This tutorial will show you how to process SLIC output files with the Pandora PFA New package, currently the best performing PFA algorithm, in order to produce Particle Flow Objects (PFOs) (also called ReconstructedParticles) for analysis. It covers building and installing a slic-specific frontend called slicPandora and all its dependencies. Then it will go through the steps for processing slic output using LCSim in order to prepare it for Pandora. Finally, it shows how to run slicPandora using these events. There is also a note on how to generate Pandora detector XML files from compact descriptions.

Quick Setup Instructions

These commands can be executed to produce a working slicPandora without monitoring enabled, using a shell such as bash.

cd /my/work/dir
svn co https://svnsrv.desy.de/public/ilctools/CMakeModules/trunk/ CMakeModules
export CMAKE_MODULE_PATH=`pwd`/CMakeModules
cvs -d :pserver:anonymous@cvs.freehep.org:/cvs/lcd co lcio
cd lcio
cmake .
make
export LCIO_HOME=`pwd`
cd ..
svn co https://svnsrv.desy.de/public/PandoraPFANew/PandoraPFANew/trunk PandoraPFANew
cd PandoraPFANew
cmake -DCMAKE_MODULE_PATH=$CMAKE_MODULE_PATH .
make
export PandoraPFANew_HOME=`pwd
cd ..
cvs -d :pserver:anonymous@cvs.freehep.org:/cvs/lcd co slicPandora
cd slicPandora
cmake -DROOT_HOME=$ROOTSYS -DCMAKE_MODULE_PATH=$CMAKE_MODULE_PATH -DPandoraPFANew_HOME=$PandoraPFANew_HOME .
make

You need to have a preexisting installation of ROOT, which is not covered here.

More detailed instructions are listed below.

Detailed Setup Instructions

Preliminaries

A Linux or Unix platform is assumed and the bash shell is used for all command-line instructions. The actual setup procedures have been tested on Redhat Enterprise Linux 5.5 (Tikanga).

The cmake tool must be installed for building several of the packages.

You will need the standard GNU tools installed, e.g. Make, gcc/g++, etc.

The cvs and svn command-line tools are also required for obtaining project source codes.

Dependencies

The slicPandora package currently has the following dependencies.

Installing Packages

LCIO

Checkout the LCIO project from cvs.

cd /workdir
cvs -d :pserver:anonymous@cvs.freehep.org:/cvs/lcd co lcio

Building this project requires the CMake tool, the installation of which is not covered here. It additionally requires command-line GNU Make.

cd lcio
cmake .
make

The cvs head or last release tag should both work.

Set the LCIO environment variable.

export LCIO=`pwd`

This variable will be used by subsequent packages to locate LCIO and make a setup script.

CMakeModules

Checkout the head of CMakeModules, required by PandoraPFANew.

svn co https://svnsrv.desy.de/public/ilctools/CMakeModules/trunk/ CMakeModules

Make an environment variable that will be referenced later.

cd CMakeModules
export CMAKE_MODULE_PATH=`pwd`

This package does not need to be compiled. It just provides some CMake macros.

PandoraPFANew

Overview

This tutorial will show you how to process SLIC output files with the Pandora Particle Flow Algorithm to produce Particle Flow Objects (PFOs) for analysis. PFOs are represented by the ReconstructedParticle class in the LCIO event data format.

These instructions cover building and installing a slic-specific frontend called slicPandora and all its dependencies. Then the steps will be shown for processing the output using LCSim in order to prepare it for Pandora. Finally, there are instructions for running these events in slicPandora. There is also a brief section on generating a Pandora detector XML file from a compact detector description, the XML format used by lcsim to represent detector and geometry information.

Building SlicPandora

Preliminary Setup

A Linux or Unix platform is assumed and the bash shell is used for all command-line instructions. The actual setup procedures have been tested on Redhat Enterprise Linux 5.5 (Tikanga).

You will need the standard GNU tools installed, e.g. Make, gcc/g++, etc.

The cvs and svn command-line tools are also required for obtaining project source codes.

The cmake tool must be installed, as it is used to build most of the dependencies.

Dependencies

The slicPandora package currently has the following dependencies.

There are build instructions below that show all the steps necessary to build these packages.

Build Instructions

These commands can be executed to produce a working slicPandora.

Start by making a directory where Pandora and its dependencies will be installed.

This is just an example. Any directory with enough space will do.

mkdir /work/pandora_build
cd /work/pandora_build

The following commands should then be executed in order from this working directory to make the slicPandora binary. You should execute each new package build from your working dir.

ilcutil

svn co http://svnsrv.desy.de/public/ilctools/ilcutil/trunk ilcutil
cd ilcutil
mkdir build
cd build
cmake ..
make install
cd ..
export ILCUTIL_DIR=`pwd`

Pandora

svn co http://svnsrv.desy.de/public/PandoraPFANew/PandoraPFANew/trunk PandoraPFANew
cd PandoraPFANew
mkdir build
cd build
cmake -DILCUTIL_DIR=$ILCUTIL_DIR -DCMAKE_SKIP_RPATH=1 ..
make install
cd ..
export PandoraPFANew_DIR=`pwd`

LCIO

svn co svn://svn.freehep.org/lcio/tags/v02-01-01 lcio
cd lcio
mkdir build
cd build
cmake -DINSTALL_DOC=OFF -DBUILD_32BIT_COMPATIBLE=OFF -DCMAKE_SKIP_RPATH=1 ..
make install
cd ..
export LCIO_DIR=`pwd`

slicPandora

cvs -d :pserver:anonymous@cvs.freehep.org:/cvs/lcd co slicPandora
cd slicPandora
export SLICPANDORA_DIR=`pwd`
mkdir build
cd build
cmake -DILCUTIL_DIR=$ILCUTIL_DIR -DLCIO_DIR=$LCIO_DIR -DPandoraPFANew_DIR=$PandoraPFANew_DIR -DCMAKE_SKIP_RPATH=1 ..
make install

If this step completes successfully without compilation or link errors, you should see a binary at slicPandora/bin/PandoraFrontend. You are basically done building slicPandora.

Pandora Settings (optional)

You should also check out the project that contains example config files for PandoraCheck out the version of Pandora that will be used. The example below shows how to checkout the head version.

svn co httpshttp://svnsrv.desy.de/public/PandoraPFANew/PandoraPFANewSettings/trunk PandoraPFANewPandoraSettings

Now we need to build another package before configuring Pandora.

PandoraMonitoring

Checkout the head version of PandoraMonitoring.

svn co https://svnsrv.desy.de/public/PandoraPFANew/PandoraMonitoring/trunk PandoraMonitoring

Now configure the build using cmake.

cmake -DROOT_HOME=/path/to/root -DCMAKE_MODULE_PATH=$CMAKE_MODULE_PATH -DPandoraPFANew_HOME=/path/to/pandora .

LCDetectors (optional)

There are some slicPandora geometry xml files here. (This is a rather big cvs module so might take some time to checkout.)

cvs -d :pserver:anonymous@cvs.freehep.org:/cvs/lcd co LCDetectors

For example, see LCDetectors/detectors/sidloi3/sidloi3_pandora.xml and its compact detector description.

Setting the LD_LIBRARY_PATH

The shared library locations need to be specified in the LD_LIBRARY_PATH for slicPandora to run. The variable needs to be defined as follows, based on environment settings from the buildSet the environment variable.

export PANDORAMONITORING=`pwd`

Technically, the monitoring package is optional, though it is quite useful for event and detector visualization.

PandoraPFANew Continued

Since Pandora has minimal external dependencies, it is straightforward to build. Only the CMakeModules must be located for the build and PandoraMonitoring, and the latter is optional.

cd PandoraPFANew
cmake -DCMAKE_MODULE_PATH=$CMAKE_MODULE_PATH -DPandoraMonitoring_Home=/path/to/PandoraMonitoring .
make

If monitoring is not being used, this CMake command can be used instead.

cmake -DCMAKE_MODULE_PATH=$CMAKE_MODULE_PATH -DCMAKE_INSTALL_PREFIX=`pwd` BUILD_SHARED_LIBS=on .

Also, the Framework library needs to be built.

cd PandoraPFANew/Framework
cmake .
make

Set the environment variable.

cd PandoraPFANew
export PandoraPFANew_HOME=`pwd`

This will be referenced in the build of the next package.

If the release compiles successfully, you're now ready to setup the project that interfaces SLIC to Pandora.

slicPandora

Checkout the slicPandora project from cvs.

cd /workdir
cvs -d :pserver:anonymous@cvs.freehep.org:/cvs/lcd co slicPandora
cd slicPandora

The following should generate a setup script called mysetup.sh for setting up your runtime and compilation environment later.

./make_setup_script.sh

This setup script looks for the following environment variables to determine your Pandora setup.

If ROOTSYS and PANDORAMONITORING are not defined in the environment, then visualization will be disabled.

Source the new setup script that was generated.

. mysetup.sh

Alternately, a Makefile can be generated with CMake, using the following command.

cmake -DPandoraPFANew_HOME=$PandoraPFANew_HOME -DLCIO_HOME=$LCIO_HOME CMAKE_MODULE_PATH=$CMAKE_MODULE_PATH -DROOT_HOME=$ROOTSYS

Now build the slicPandora project.

make

The slicPandora project should now be built successfully, if its dependencies are present and setup correctly. If compilation or linking errors occur, then check that LCIO and Pandora were both installed successfully and that the setup script is pointing to the correct locations of these packages. If monitoring has been enabled, check that ROOT and PandoraMonitoring are configured and built successfully and that their environment variables are set.

Running Events

Steps

The essential steps to generating slicPandora events are as follows.

1) Use SLIC to generate LCIO simulation data.
2) Add tracks to the event by running a tracking package such as LCSim's SeedTracker.
3) Add the TrackState collections so slicPandora knows the Track momenta. (can be combined with #2 into one LCSim job)
4) Run slicPandora with the LCSim output, a Pandora XML geometry generated by GeomConverter, and a Pandora settings XML file.

These steps are covered in more detail below.

Generating Tracks Using LCSim

Before the simulated LCIO events can be run through Pandora, the LCSim tracking needs to run in order to add a collection of tracks.

If you don't know how LCSim batch mode works, then review the LCSim XML instructions.

Tracking Steering File

Below is an example XML steering file for LCSim to generate the Tracks and TrackStates for the sidloi3 detector.

LD_LIBRARY_PATH=$SLICPANDORA_DIR/lib:$LCIO_DIR/lib:$PandoraPFANew_DIR/lib

If these variables are not set in the environment, then using the full, explicit path to the library directories will work fine.

Checking the Build

Now you should be able to run this command to load slicPandora.

./slicPandora/bin/PandoraFrontend

The help message should show for slicPandora. If there are errors about missing libraries, set the LD_LIBRARY_PATH variable accordingly.

PandoraFrontend

The PandoraFrontend binary provides a simple frontend to slicPandora. It uses GNU style command line switches for user input.

The following table explains the function of each of these switches.

The geometry file is in an XML format. It can be created from a compact detector description using GeomConverter with the option "-o pandora". You are required to provide some additional parameters related to sampling fractions (see LCDetectors/detectors/sidloi3 for example).

The Pandora settings XML file contains settings to configure the Pandora PFA algorithm. Example configuration files can be found in the Pandora Settings SVN project (see building instructions above) (see PandoraSettings/PandoraSettingsBasic.xml).

The LCIO input events should contain collections of LCIO CalorimeterHits named according to their subdetectors, plus a collection of LCIO Track objects called "Tracks." There also needs to be a collection called "TrackStates" containing momentum measurements at various points along the Track (see below for details of generating these files).

The LCIO output file will contain the contents of the input file plus the PFO collections created by Pandora, as well as some intermediate collections of hit objects. This file is generated by slicPandora.

The number of events to run should be a positive integer. If slicPandora runs out of events, it will print and warning but should not crash.

The number of events to skip should be a positive integer. These events are read in and discarded before event processing begins.

Here is a contrived example using all input parameters.

./bin/PandoraFrontend -g myGeometry.xml -c PandoraSettings.xml -i inputEvents.slcio -l myCollections.xml -o myOutput.slcio -r 1000 -s 10

These are example arguments only. Actual usage will require files in the correct input data formats, namely the geometry XML, the Pandora settings file, and the XML specifying which LCIO input collections to use, if different from the defaults.

Using GeomConverter to Output the Pandora Geometry Format

The GeomConverter package can convert from compact detector descriptions to various output formats, including one for input to slicPandora.

For example, to generate the Pandora settings for the sidloi3 detector.

cvs -d :pserver:anonymous@co LCDetectors
cd LCDetectors/detectors/sidloi3
java -jar ~/.m2/repository/org/lcsim/GeomConverter/1.13-SNAPSHOT/GeomConverter-1.13-SNAPSHOT-bin.jar -o pandora compact.xml sidloi3_pandora.xml

This will create the file sidloi3_pandora.xml that can be used as input to slicPandora.

FIXME: Need to document the conditions used to set sampling fractions.

Running Events in SlicPandora

Steps

The essential steps to generating events with SlicPandora are as follows.

1) Use SLIC to generate an LCIO file.
2) Add tracks to the event by running a tracking package such as LCSim's SeedTracker.
3) Add the TrackState collections so slicPandora knows the Track momenta. (can be combined with #2 into one LCSim job)
4) Run SlicPandora with the LCSim output, a Pandora XML geometry generated by GeomConverter, and a Pandora settings XML file.

These steps are covered in more detail below.

Generating Tracks Using LCSim

Before the simulated LCIO events can be run through Pandora, the LCSim tracking needs to run in order to add a collection of tracks.

If you don't know how LCSim batch mode works, then review the LCSim XML instructions.

Tracking Steering File

Below is an example XML steering file for LCSim to generate the Tracks and TrackStates for the sidloi3 detector.

<lcsim xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
        xs:noNamespaceSchemaLocation="http://www.lcsim.org/schemas/lcsim/1.0/lcsim.xsd">
    <inputFiles>
        <file>./pi_Theta90_10GeV-0-1000_SLIC-v2r8p3_geant4-v9r3p1_QGSP_BERT_sidloi3.slcio</file>
    </inputFiles>
    <control>
        <numberOfEvents>1000</numberOfEvents>
        <verbose>true</verbose>
        <printSystemProperties>false</printSystemProperties>
    </control>
    <execute>

<lcsim xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
        xs:noNamespaceSchemaLocation="http://www.lcsim.org/schemas/lcsim/1.0/lcsim.xsd">
    <inputFiles>
        <file>./pi_Theta90_10GeV-0-1000_SLIC-v2r8p3_geant4-v9r3p1_QGSP_BERT_sidloi3.slcio</file>
    </inputFiles>
    <control>
        <numberOfEvents>1000</numberOfEvents>
        <verbose>true</verbose>
        <printSystemProperties>false</printSystemProperties>
    </control>
    <execute>
        <driver name="EventMarkerDriver"/>
        <driver name="CalInfoDriver"/>
        <driver name="MainTrackingDriver"/>
        <driver name="TrackStateDriver"/>
        <driver name="Writer"/>
    </execute>
    <drivers>
        <driver name="CalInfoDriver"   EventMarkerDriver"/>
                type="org.lcsim.recon.util.<driver name="CalInfoDriver"/>
        <driver name="MainTrackingDriver"
      />
        <driver  typename="org.lcsim.recon.tracking.seedtracker.trackingdrivers.sidloi2.MainTrackingDriverTrackStateDriver"/>
        <driver name="EventMarkerDriverWriter"/>
 
   </execute>
    <drivers>
        <driver typename="org.lcsim.job.EventMarkerDriver">CalInfoDriver"
            <eventInterval>10</eventInterval>
        </driver>type="org.lcsim.recon.util.CalInfoDriver"/>
        <driver name="TrackStateDriverMainTrackingDriver"
                type="org.lcsim.recon.tracking.seedtracker.trackingdrivers.sidloi2.SeedTrackerTrackStateDriverMainTrackingDriver"/>
        <driver name="WriterEventMarkerDriver"
                type="org.lcsim.utiljob.loop.LCIODriverEventMarkerDriver">
            <outputFilePath>./pi_Theta90_10GeV-0-1000_SLIC-v2r8p3_geant4-v9r3p1_QGSP_BERT_sidloi3_lcsimTracking.slcio</outputFilePath><eventInterval>10</eventInterval>
        </driver>
         </drivers>
</lcsim>

The input files section needs to be changed to point to your local simulated SLIC events, and the outputFilePath would also be changed to have a name based on the input file. You can easily make a script to automate this.

More LCSim Tracking Details

The LCSim job must accomplish three tasks before the events can be read into Pandora, in this order.

1. Generation of a Tracks collection using an appropriate Seed Tracker Driver.
2. Adding TrackState collections using the SeedTrackerTrackState Driver.
3. Writing out the necessary LCIO collections to a data file to be read into Pandora.

To generate the Tracks, a top-level Driver should be run that covers subdetector setup, digitization, and track finding and fitting. This top-level Driver will likely be specific to a certain detector design.

For instance, this simple Driver definition is sufficient to generate tracks in the sidloi3 detector.

 <driver name="MainTrackingDriver"
         type="org.lcsim.recon.tracking.seedtracker.trackingdrivers.sidloi3.MainTrackingDriver"/>

Three TrackState collections need to be added to the LCIO output to provide Pandora with track information at the track origins, the ECal, and the end point.

The following Driver in the Seed Tracker package will add these necessary TrackState collections. This Driver also requires that another Driver be run beforehand to cache Calorimeter subdetector data.

<driver name="CalInfoDriver"   
        type="org.lcsim.recon.util.CalInfoDriver"/>
<driver name="TrackStateDriver"
        type="org.lcsim.recon.tracking.seedtracker.SeedTrackerTrackStateDriver"/>

Finally, an LCIODriver should be added to the end of the event processing to output the appropriate collections.

<driver name="TrackStateDriver"
                type="org.lcsim.recon.tracking.seedtracker.SeedTrackerTrackStateDriver"/>
        <driver name="Writer"
                type="org.lcsim.util.loop.LCIODriver">
            <outputFilePath>./pi_Theta90_10GeV-0-1000_SLIC-v2r8p3_geant4-v9r3p1_QGSP_BERT_sidloi3_lcsimTracking.slcio</outputFilePath>
        </driver>
    </drivers>
</lcsim>

The input files section needs to be changed to point to your local simulated SLIC events, and the outputFilePath would also be changed to have a name based on the input file. You can easily make a script to automate this.

More LCSim Tracking Details

The LCSim job must accomplish three tasks before the events can be read into Pandora, in this order.

1. Generation of a Tracks collection using an appropriate Seed Tracker Driver.
2. Adding TrackState collections using the SeedTrackerTrackState Driver.
3. Writing out the necessary LCIO collections to a data file to be read into Pandora.

To generate the Tracks, a top-level Driver should be run that covers subdetector setup, digitization, and track finding and fitting. This top-level Driver will likely be specific to a certain detector design.

For instance, this simple Driver definition is sufficient to generate tracks in the sidloi3 detector.

 <driver name="MainTrackingDriver"
 

<driver name="Writer"
        type="org.lcsim.util.loop.LCIODriver">
            <outputFilePath>OUTPUT_FILE</outputFilePath>
</driver>

The OUTPUT_FILE argument needs to be replaced with the actual name of the LCIO output file to be fed to Pandora.

Once these Drivers are defined in the <driver> section of your LCSim XML file, the execution order should look like the following.

<execute>
    <driver name="CalInfoDriver"/>
    <driver name="MainTrackingDriver"/>
    <driver name="TrackStateDriver"/>
    <driver name="Writer"/>
</execute>

Now that tracks and track states have been added to the events, we are ready to use Pandora itself.

PandoraFrontend

The PandoraFrontend binary provides a simple frontend to slicPandora.

The syntax of this command is the following.

./bin/PandoraFrontend geometry.xml pandoraSettings.xml inputEvents.slcio reconOutput.slcio [nevents]

The nevents command is optional, but all other arguments are required and must be supplied in order.

For example, to repeat the results of the JobManagerTest using PandoraFrontend, execute this command.

./bin/PandoraFrontend ./examples/sidloi3_pandora.xml ./examples/PandoraSettingsRelease.xml ./input.slcio ./pandoraRecon.slcio 2

The above command requires that the input events were generated in the sidloi3 detector and that a symlink has been setup pointing from "input.slcio" to your input event file.

Using GeomConverter to Output the Pandora Geometry Format

The GeomConverter package can convert from compact detector descriptions to various output formats, including one for input to slicPandora. (Installation of GeomConverter is not covered here.)

Use this command from your GeomConverter root directory to generate a Pandora geometry file from a compact detector description.

java -jar ./target/GeomConverter-1.10-SNAPSHOT-bin.jar -o pandora ./myCompactDetector.xml ./myPandoraGeom.xml

...

.recon.tracking.seedtracker.trackingdrivers.sidloi3.MainTrackingDriver"/>

Three TrackState collections need to be added to the LCIO output to provide Pandora with track information at the track origins, the ECal, and the end point.

The following Driver in the Seed Tracker package will add these necessary TrackState collections. This Driver also requires that another Driver be run beforehand to cache Calorimeter subdetector data.

<driver name="CalInfoDriver"
        type="org.lcsim.recon.util.CalInfoDriver"/>
<driver name="TrackStateDriver"
        type="org.lcsim.recon.tracking.seedtracker.SeedTrackerTrackStateDriver"/>

Finally, an LCIODriver should be added to the end of the event processing to output the appropriate collections.

<driver name="Writer"
        type="org.lcsim.util.loop.LCIODriver">
            <outputFilePath>OUTPUT_FILE</outputFilePath>
</driver>

The OUTPUT_FILE argument needs to be replaced with the actual name of the LCIO output file to be fed to Pandora.

Once these Drivers are defined in the <driver> section of your LCSim XML file, the execution order should look like the following.

<execute>
    <driver name="CalInfoDriver"/>
    <driver name="MainTrackingDriver"/>
    <driver name="TrackStateDriver"/>
    <driver name="Writer"/>
</execute>

Now that tracks and track states have been added to the events, we are ready to use Pandora itself.

Links

• PandoraPFANew FAQ
• Pandora SVN Repository