...

from psalg.configdb.typed_json import *
import psalg.configdb.configdb as configdb

Typed JSON

JSON does not provide any type information for the values it contains.  For a configuration database, this can be problematic, as some numeric values might be limited to integers or even a small set of integers.  Therefore, additional information is added to the JSON configuration object to provide type information.  Any field name ending in ":RO" is considered to be read-only and will not be displayed by the graphical editor.  An additional key, ":types:" will be added to the top level dictionary.  This key maps to a dictionary with roughly the same structure as the JSON object, except that instead of containing values, the dictionary contains type information.  This information is either:

...

• "detType:RO", a mandatory CHARSTR.
• "detName:RO", a mandatory CHARSTR.
• "detId:RO", a mandatory CHARSTR.
• "doc:RO", an optional CHARSTR.
• "alg:RO", an optional dictionary containing:
  • "alg:RO", a mandatory CHARSTR.
  • "doc:RO", an optional CHARSTR.
  • "version:RO", a mandatory ["INT32", 3].

The cdict Class

In order to simplify the creation of typed JSON objects, the typed_json module defines the cdict class.  The constructor for this class takes an optional argument which is either an instance of another cdict or a dictionary representing a typed JSON object that is used to initialize the new cdict.  The main methods for this class are:

...

• getType(dict, name) returns the type of the value referred to by the flattened name, throwing an error if dict does not have any such value.
• getValue(dict, name) returns the value referred to by the flattened name, throwing an error if dict does not have any such value.
• updateValue(dict, name, value) stores a new value into the typed JSON dict referred to by name.  value is always a string.  Numeric values are converted, and array values are space-separated.  0 is returned on success, and non-zero values indicate an error.

The configdb Class

Configuration management with MongoDB is handled by the configdb class.   In general, we add additional documents to the database, but do not change existing ones, so we can keep a complete history of configuration changes.  The basic organization is:

...

• set_hutch(h, create=False)
  Set the default hutch to h.  If create is True, create any necessary DB entries.
• add_alias(alias)
  Add a new alias to the default hutch.
• add_device_config(cfg)
  Add a new device type collection named cfg.
• get_hutches()
  Return a list of all defined hutches.
• get_aliases(hutch=None)
  Return a list of all aliases for the specified hutch (or the default hutch if the parameter is None).
• get_device_configs()
  Return a list of all device types.
• get_key(alias, hutch=None)
  Return the highest version number for the alias in the specified hutch (or default hutch if None).
• get_devices(key_or_alias, hutch=None)
  Return a list of devices in the specified hutch (or default hutch if None).  key_or_alias specifies the particular configuration to examine:  if it is a string, use the current configuration for this alias and if it is an integer, use the configuration with the specified version number.
• modify_device(alias, value, hutch=None)
  Modify the current configuration for the specified alias in the specified hutch.  value is a typed JSON dictionary where the detName:RO field is the name of the device and detType:RO is the device type.  This raises an exception if there is an error and returns the newly written version number otherwise.
• get_configuration(key_or_alias, device, hutch=None)
  Get the configuration for the specified device in the specified hutch (or the default if this is None).  key_or_alias specifies the particular configuration to examine:  if it is a string, use the current configuration for this alias and if it is an integer, use the configuration with the specified version number ("key").  Internally "key" is used for all lookups by the server-side code below.
• transfer_config(oldhutch, oldalias, olddevice, newalias, newdevice)
  Copy the current configuration for device olddevice with alias oldalias in hutch oldhutch to the current hutch configuration for with alias newalias device newdevice.
• get_history(alias, device, plist, hutch=None)
  Retrieve a history for the list of flattened names in plist for the specified device, alias, and hutch.  The return value is a list of dictionaries, each one with date, key, and all of the plist elements as keys.

Design

Mike Browne designed this based roughly on Matt's original LCLS1 configdb document: https://docs.google.com/document/d/12BlCMCWGy3X9Z9QEZn9AtkjVipYA0_x3ZiyfDYrRo-c/edit.  Chris Ford worked with Murali Shankar on the backend http database development.

Code

The server-side code (which receives http requests on the database server) is here:  https://github.com/slac-lcls/psdm_configdb.  The client-side code (which forms the http requests) is here:  https://github.com/slac-lcls/lcls2/tree/master/psdaq/psdaq/configdb.

Configuration history management

Two scripts are available for configuration history management:

• configdb_key_history: this script can be used to explore configuration history. It takes hutch, alias and device as arguments (plus authentication information):
  

  python configdb_key_history.py --user tmoopr --password pcds --hutch tmo --alias NOBEAM --device hsd_0

  And returns a list of configurations for the specific hutch/alias/device with date and key. The key can be used to retrieve the specific configuration using the configdb_rollback.py script.
  
  

• configdb_rollback.py: this script can be used to retrieve a specific device configuration, and, optionally, to make it the current configuration for the device. It takes hutch, alias and device (and authentication information) as arguments, plus a key defining a device configuration (see the configdb_key_history.py script above):
  

  python configdb_rollback.py --user tmoopr --password pcds --hutch tmo --alias NOBEAM --device hsd_0 --key 266

  
  By default, the script just shows the configuration. However, when the --write-to-database option is used, the scripts write the retrieved configuration as the current configuration of the same hutch/alias/device.

Configuration History and Keys

DISCLAIMER: The following are just deductions. I (Valerio) have no direct insight on how configuration keys work.

...