...

Two Suggested New Tools

Merging Schema Changes

There is a working example of schema update in psdaq/psdaq/configdb/hsd_config_update.py

issue: we don't handle merging of schema changes to the config objects

ric's figurative proposal:
- script to read db and write json file
- modify ts_config_store.py to put the new schema in dbase
- third script applies settings from json file to the dbase ignoring the ones
  that are dissimilar
(get_config.py has some of the tools to do this)

...

ts_config_V2:
config[group][0][rate]=10Hz
             [1][rate]=100Hz
             [2][rate]=100Hz
             [4][rate]=100HzHiding

Rename/

...

Remove Old Configdb Objects

ability remove and hide (in the control gui) detector configurations that aren't needed anymore
(hide: prepend an "_"?, rename timing_1 to _timing_1?)
rename = delete+create

Ric added: Maybe this should be thought of as archiving unneeded ConfigDb objects and we should therefore provide a way to unarchive them for when archiving is done erroneously or we change our mind.  Need to consider how to handle the case when, for example, timing_1 is archived to _timing_1 and then a new timing_1 is created.

Mike Browne implemented the following rename/delete functionality.  Murali is implementing a web interface on top of this:

c2 = cdb.configdb(mdb.server, "SXR", create=True, root=dbname)
c2.add_alias("FOO")                                       # 0
c2.transfer_config("AMO", "BEAM", "evr0", "FOO", "evr3")  # 1
with pytest.raises(Exception):
    c2.transfer_config("AMO", "BEAM", "evr0", "FOO", "evr3")
print("Configs:")
c2.print_configs()
cfg1 =  c.get_configuration("BEAM", "evr0")
cfg2 = c2.get_configuration("FOO", "evr3")
# These should be the same except for the detector names!
del cfg1['detName:RO']
del cfg2['detName:RO']
assert cfg1 == cfg2
c2.rename_device("evr3", "evr7", "FOO")
cfg3 = c2.get_configuration("FOO", "evr7")
# These should be the same except for the detector names!
del cfg3['detName:RO']
assert cfg2 == cfg3
with pytest.raises(ValueError):
    c2.remove_device("evr6", "FOO")
c2.remove_device("evr7", "FOO")
with pytest.raises(ValueError):
    cfg4 = c2.get_configuration("FOO", "evr7")

Rename/Remove Discussion

with Chris Ford, Murali, Matt, cpo on April 17, 2024

Murali highlighted some important questions with the new rename/remove features.  Here they are, with answers from the group.  I believe the conclusions were unanimous:

• do we want to update all old keys or only the latest key?
  • the consensus was that updating only the latest key did a better job of preserving the history (e.g. for comparison with xtc values).  there was a weak counter-argument that the "configdb rollback" command would not work as naturally (would need to know both the old name and the new name).   
• the detector name (e.g. trigger_0) shows up twice: once as a database-lookup key ("device") and once in the document with typed-json ("detName:RO", which is useful for converting to xtc).  Do we want to edit both?
  • Yes, it is more natural to edit both.  Matt pointed out that the daq code may overwrite detName:RO, but we weren't certain, and in any event it feels more elegant to have it be correct in the database
• do we want to create a new key with the changes?
  • Yes, because this does a better job of preserving the history
• do  we want to create a new document with the new detName:RO?
  • Yes, because this does a better job of preserving the history
• what do we do if the user tries to rename to name that already exists in the current key?
  • We should raise an error, which would force the user to explicitly remove the existing conflicting name before doing the rename.

Removal is more straightforward: a new key will be created with the appropriate device removed, but history will be preserved in the old keys.

These answers will require changes to Mike Browne's code which Murali has kindly agreed to do.

CLI Overview

$ configdb -h
usage: configdb [-h] [--url URL] [--root ROOT] {cat,rm,cp,mv,history,rollback,ls} ...

configuration database CLI

positional arguments:
  {cat,rm,cp,mv,history,rollback,ls}
    cat                 print a configuration
    rm                  remove a configuration
    cp                  copy a configuration (EXAMPLE: configdb cp --create --write tmo/BEAM/timing_0 newhutch/BEAM/timing_0)
    mv                  rename a configuration (EXAMPLE: configdb mv --write tst/BEAM/timing_45 timing_46)
    history             get history of a configuration
    rollback            rollback configuration to a specific key
    ls                  list directory contents

optional arguments:
  -h, --help            show this help message and exit
  --url URL             configuration database connection
  --root ROOT           configuration database root (default: configDB)

Configdb Server Logs

The configdb server runs on a subset of psdm[01-04], where multiple nodes are used in a "load balancer" manner.  You have login to each of these nodes and look for logfiles in /u1/psdm/logs/configdb* to find the one that contains the transaction you are interested in.

Sample curl Commands

From Murali (for the dev configdb).

Get configuration for a device
curl -s -u "tstopr:passwordremoved" "https://pswww.slac.stanford.edu/ws-auth/devconfigdb/ws/configDB/get_configuration/tst/BEAM/trigger_0/"

Rename device
curl -s -u "tstopr:passwordremoved" "https://pswww.slac.stanford.edu/ws-auth/devconfigdb/ws/configDB/rename_device/tst/BEAM/trigger_0/?newname=trigger_1"

Remove device
curl -s -u "tstopr:passwordremoved" "https://pswww.slac.stanford.edu/ws-auth/devconfigdb/ws/configDB/remove_device/tst/BEAM/trigger_0/"