Page History

Overview

HPS uses a conditions database which is accessible through the DatabaseConditionsManager.

Database Connection

Command Line Options

The URL, username and password for connecting to the database can be specified from the command line as Java properties.

The default connection information corresponds to the following command line options:

  The conditions framework provides classes that read information from the database and return them as lists of typed Java objects with methods for accessing each data field.  Users can access these objects to configure their jobs, typically in the detectorChanged() method of their Driver classes.  The conditions system is setup by specifying the unique name of a detector description and the run number.  Typically, this is done by reading this information automatically from the event headers in an LCIO file.

Database Connection

Command Line Options

The parameters for connecting to the database can be specified from the command line as global Java properties.

The default connection information for reading from the database corresponds to the following command line options:

java -

java -Dorg.hps.conditions.url=jdbc:mysql://hpsdb.jlab.org:3306/hps_conditions \  
    -Dorg.hps.conditions.user=hpsuser \
    -Dorg.hps.conditions.password=darkphoton [...]

These options should always be provided immediately after the java command as they are global Java properties and not rather than command line options provided to a specific application or program within hps-java.

The default database connection uses a read-only replica of the primary MySQL conditions database at Jefferson Lab.  Therefore, when connecting from a computer which is outside of the jlab.org domain, you will  not be able to make any changes to this database.  If you need to insert records into the database, i.e. for new calibrations, then it must be done behind the JLab firewall.  You , and you must provide proper valid credentials that will allow writing to the database (they are not given provided here!).

Using

...

the SLAC Database

The following connection properties can be used to connect to the SLAC conditions database

...

Support for SQLite is included in the database conditions manager for running jobs locally without an internet connection.

A db file may be obtained by using the following commands:

java -Dorg.hps.conditions.url=jdbc:mysqlhttps://github.com/JeffersonLab/hps-conditions-backup/raw/master/hps_conditions.db.tar.gz
tar -zxvf hps_conditions.db.tar.gz

The local db file can be used by using this option when running Java:

java -Dorg.hps.conditions.url=jdbc:sqlite:hps_conditions.db [...]

No username or password is required when connecting locally in this way.

Creating a Local SQLite Database

In order to create a local SQLite database, you will need to create a snapshot of the MySQL database and convert it to a SQLite db file.   

This converter script can be used to produce the db file.

It can be downloaded using these commands:

  wget https://raw.githubusercontent.com/dumblob/mysql2sqlite/master/mysql2sqlite
  chmod +x mysql2sqlite

You can create a dump of the current conditions database using this command: 

mysqldump --skip-extended-insert --compact -u hpsuser --password=darkphoton -h hpsdb.jlab.org  --lock-tables=false hps_conditions > hps_conditions.mysql

Now, you can load it into SQLite as follows:

  mysql2sqlite hps_conditions.mysql | sqlite3 hps_conditions.db

You should have an up to date copy of the master conditions database locally, now.

Administration

Backing Up the Database

The conditions database can be backed up using a command similar to the following:

mysqldump -h hpsdb.jlab.org -ujeremym -pXXXXXX hps_conditions &> hps_conditions.sql

... where 'jeremym' is replaced by your account name and 'XXXXXX' with your password.

To load the database from a backup, the following command would be used.

mysql -h hpsdb.jlab.org -u jeremym -pXXXXXX hps_conditions < hps_conditions.sql

The above command is listed for reference only and you should not try to execute it. 

mysql-node03.slac.stanford.edu:3306/rd_hps_cond \  
    -Dorg.hps.conditions.user=rd_hps_cond_ro \
    -Dorg.hps.conditions.password=2jumpinphotons. [...] 

This database is updated only periodically, not automatically, so you may need to check if it is up-to-date before using it for your jobs.

Using a Local Conditions Database

Local jobs can be run without an internet connection using the built-in support for SQLite.

A db file compatible with sqlite3 may be obtained by using the following commands:

https://github.com/JeffersonLab/hps-conditions-backup/raw/master/hps_conditions.db.tar.gz
tar -zxvf hps_conditions.db.tar.gz

This file may not be up to date with the current master in the JLab conditions database!

The local db file can be used by using this option when running Java:

java -Dorg.hps.conditions.url=jdbc:sqlite:hps_conditions.db [...]

No username or password is required when connecting locally in this way.

Creating a Local SQLite Database

In order to create a local SQLite database, you will need to create a snapshot of the MySQL database and then convert it to a SQLite db file.   

This converter script can be used to produce the db file.

It can be downloaded using these commands:

  wget https://raw.githubusercontent.com/dumblob/mysql2sqlite/master/mysql2sqlite
  chmod +x mysql2sqlite

You can create a dump of the current conditions database using this command: 

mysqldump --skip-extended-insert --compact -u hpsuser --password=darkphoton -h hpsdb.jlab.org --extended-insert=false --lock-tables=false hps_conditions > hps_conditions.mysql

Now, you can load the database dump into sqlite3 as follows:

  mysql2sqlite hps_conditions.mysql | sqlite3 hps_conditions.db

You should now have an up to date copy of the master conditions database locally that can be specified on the command line.

Administration

Creating a Conditions Database Backup

The conditions database can be backed up using a command similar to the following:

mysqldump -h hpsdb.jlab.org -u$USER -p$PASSWORD hps_conditions &> hps_conditions.sql

... where $USER is replaced by your account name with the proper permissions and $PASSWORD with your password.

Loading a Conditions Database Backup

To load the database from a backup, the following command would be used.

mysql -h hpsdb.jlab.org -u $USER -p$PASSWORD hps_conditions < hps_conditions.sql

The above command is for informational purposes only.  Fully restoring the database from a backup would need to go through a JLAB CCPR, as the accounts we have access to do not have all the proper permissions for doing this.

Replicating the Database to SLAC

This is a general outline of dumping the database and replicating it to the SLAC MySQL database, which is used for releases, as connecting from SLAC to the JLab database is not reliable enough.

First, from a JLab machine such as ifarm, a SQL dump should be created that includes all required statements for dropping tables (the default configuration of mysqldump should be fine).

mysqldump -h hpsdb.jlab.org -u$USER -p$PASSWORD --disable-lock-tables hps_conditions &> hps_conditions_for_slac.sql

This file should then be copied over to SLAC.

scp hps_conditions_for_slac.sql $USER@rhel6-64.slac.stanford.edu:/nfs/slac/g/hps/someDir

Finally, the SQL dump should be loaded into the SLAC database using a command similar to the following.

mysql -D rd_hps_cond -h mysql-node03.slac.stanford.edu -P 3306 -u $USER -p $PASSWORD < hps_conditions_for_slac.sql

The command may take awhile to execute.  If it is successful, the SLAC database should contain an exact replica of the primary conditions database from JLabFully restoring the database from a backup would need to go through a JLAB CCPR, as the accounts we have access to do not have all the proper permissions for doing this.

Configuring Detector Conditions for a Job

...

The conditions configuration is typically done via arguments to command line programsperformed using arguments to command line programs, or the system is setup automatically from information in the LCIO or EVIO events.

The detector name and run number to be used can be provided explicitly to the job manager to override these settings from input files.

java -jar hps-distribution-bin.jar -d detector_name -R 5772 [args]

Configuration of the EvioToLcio utility is similardone similarly.

java -cp hps-distribution-bin.jar org.hps.evio.EvioToLcio -d detector_name -R 5772 [args]

Providing conditions in this way will cause the manager to automatically "freeze" itself after it initializes initialization so that subsequent run numbers and detector header information from the input file files will be ignored in the job.

Additionally, tags can be specified to filter out the available conditions records in the job, which is described in the Detector Conditions Tags documentation.

Java Initialization

The HPS conditions manager is automatically installed by creating a new instance of the DatabaseConditionsManager class.global instance of the conditions manager can be accessed using the following command:

newfinal DatabaseConditionsManager mgr = DatabaseConditionsManager.getInstance();

If an instance has not already been instantiated, one will be created.

The conditions system is initialized using the ConditionsManager's setDetector method which takes the name of a detector and a run numberThis will automatically install the manager as the global conditions manager, which can then be accessed using the following method.

DatabaseConditionsManager mgr = DatabaseConditionsManager.getInstance().setDetector("detector_name", 5772);

Users should not normally need to install their own conditions manager as this is done in the setup of the various job tools, but it may be necessary when writing standalone scripts and command line tools which do not use these classes.

...

In some special cases, the conditions system may need to be completely reset by creating and installing a new instance of the manager.

This can be done by calling a special static method on the manager.

DatabaseConditionsManager.reset();

You should absolutely  not do this under normal circumstances such as within your Driver code.  The method is public only so that this can be done if necessary. 

Additional HPS Features

...

HPS adds several features to the lcsim conditions system.

You can add one or more tags for filtering the conditions records.  Only those records belonging to the tag will be accessible.

Code Block
DatabaseConditionsManager.getInstance().addTag("pass0");

The conditions system can be "frozen" after it is initialized, meaning that subsequent calls to set a new detector and run number will be completely ignoredthe conditions records.  Only those records belonging to the tag will be accessible.

Code Block
DatabaseConditionsManager.getInstance().freeze();

...

().addTag("pass0");

The conditions system will be initialized for you automatically or configured using switches to the various HPS Java command line tools (exact syntax depends on the tool).

...

title	Initializing the Conditions System

can be "frozen" after it is initialized, meaning that subsequent calls to set a new detector and run number will be completely ignored.

Code Block
DatabaseConditionsManager.getInstance().freeze();

This is useful to force the system to load a specific configuration by run number if the actual event data does not have the same run number (or for run 0 events from simulation).

...

Accessing Conditions from a Driver

...