Ford, Christopher 
Mon 11/12/2018, 4:30 PM             

Hi Mikhail,
Currently there is no documentation for the JSON messages, only prototype code.
Thanks,
-caf

See create_msg() and its context in collection.py:
  https://github.com/slac-lcls/lcls2/blob/master/psdaq/psdaq/control/collection.py

The create_msg() function is also imported and used by a command line client, cmd.py:
  https://github.com/slac-lcls/lcls2/blob/master/psdaq/psdaq/control/cmd.py

Control-level gui thoughts   
O'Grady, Paul Christopher <cpo@slac.stanford.edu> 
Thu 1/24, 10:40 AMDubrovin, Mikhail;Ford, Christopher             

code is in psdaq/psdaq/control
interface is in collection.py
can run one (or more) dtiproxy in a standalone mode for gui development (caf will send an example)
requirements:
- asynchronous processes to watch for the completion of the transitions
- error display
- copy the lcls1 daq gui as closely as we can
- receive messages directly from many drp nodes using a zmq many-to-one pattern
- longer term, if we follow lcls1 model, this gui would also be used to managing detector configurations

preview: 'daqstate' CLI for DAQ   
   
Ford, Christopher <caf@slac.stanford.edu> 

Thu 1/24, 12:06 PMO'Grady, Paul Christopher;Weninger, Clemens;Dubrovin, Mikhail             
The brief demo below shows 
three LCLS-II DAQ transitions (plat/alloc/connect) being automated by 
the new 'daqstate' CLI. Two instances of dti_proxy were running at the 
time, as shown by 'showPlatform.'
I intend to commit 'daqstate' to git this afternoon. Then, I will add an updated demo to Confluence.
Thanks,
  -caf

(ps-0.1.2) -bash-4.2$ daqstate -p1
reset

(ps-0.1.2) -bash-4.2$ showPlatform -p1
Platform | Partition      |    Node
         | id/name        | level/pid/host
---------+----------------+------------------------------------
    1      1/(None)     

(ps-0.1.2) -bash-4.2$ daqstate -p1 --state connected
connected

(ps-0.1.2) -bash-4.2$ showPlatform -p1
Platform | Partition      |    Node
         | id/name        | level/pid/host
---------+----------------+------------------------------------
    1      1/(None)         drp-no-teb/19944/psbuild-rhel7-01
                            drp-no-teb/19916/psbuild-rhel7-01
(ps-0.1.2) -bash-4.2$ daqstate -p1
connected
(ps-0.1.2) -bash-4.2$

Weaver, Matt <weaver@slac.stanford.edu> 
Thu 1/24, 12:22 PMO'Grady, Paul Christopher;Dubrovin, Mikhail;Ford, Christopher             

You can execute the DAQ on drp-tst-dev011 by running

cd /reg/g/pcds/dist/pds/tst/scripts
../../tools/procmgr/procmgr start testgui.cnf
../../tools/procmgr/procmgr stop testgui.cnf  (when you're done)

The IP address of 
drp-tst-dev011 appears in testgui.cnf.  You can replace it with some 
other node.  Don't try to run fast, because the event data is getting 
broadcasted on the cds-tst network (since we don't have
a FEZ or multicast filtering).
-Matt

How to run your own state machine   
Ford, Christopher <caf@slac.stanford.edu> 
Fri 2/1, 4:54 PMDubrovin, Mikhail             
             
Hi Mikhail,
Here is a test procedure to get you going:

1. Open two terminals on the same host (e.g. drp-tst-acc06)
2. In both terminals, "source setup_env.sh" in your fullly built git repository.
3. In one terminal, start a state machine by running "collection -p6 -v".
4. In the other terminal, set a state by running "daqstate -p6 --state connected" (for example).

I reserved platform 6 for your use.  Please include the "-p6" flag when running DAQ programs that accept a platform flag.
I've put a daqstate demo including screen shots on Confluence: https://confluence.slac.stanford.edu/display/~caf/daqstate+Demo
As we discussed this morning, your GUI should be able to set and get the
state using the DaqControl interface like daqstate does.
Thanks, -caf

Ford, Christopher <caf@slac.stanford.edu> 
2019-02-06, 3:37 PMO'Grady, Paul Christopher;Weninger, Clemens;Dubrovin, Mikhail             
   
Earlier today I pushed incremental improvements to collection.py and daqcontrol.py.
Now daqcontrol supports monitoring of events  via a simple blocking interface.
I think the implementation of the daqstate command, below, is really quite succinct.
Thanks,-caf

---------------------------------------------------------

    # instantiate DaqControl object
    control = DaqControl(host=args.C, platform=args.p, timeout=args.t)

    if args.state:
        # change the state
        rv = control.setState(args.state)
        if rv is not None:
            print('Error: %s' % rv)

    elif args.transition:
        # transition request
        rv = control.setTransition(args.transition)
        if rv is not None:
            print('Error: %s' % rv)

    elif args.monitor:
        # monitor the status
        while True:
            transition, state = control.monitorStatus()
            if transition is None:
                break
            print('transition: %-10s  state: %s' % (transition, state))

    if not args.monitor:
        # print current state
        print(control.getState())

===================

Ford, Christopher <caf@slac.stanford.edu> 
2019-02-06, 3:19 PMDubrovin, Mikhail             

Open three terminals on drp-tst-acc06, then proceed with instructions below.

Terminal #1: Start as before
  $ collection -p6

Terminal #2: Run daqstate with new "--monitor" flag
  $ daqstate -p6 --monitor

Terminal #3:  Run these commands
  $ daqstate -p6 --state connected
  connected
  $ daqstate -p6 --state paused
  paused
  $ daqstate -p6 --state running
  running

You should see the following output on terminal #2:
  transition: plat        state: unallocated
  transition: alloc       state: allocated
  transition: connect     state: connected
  transition: configure   state: paused
  transition: enable      state: running

Thanks, -caf

I just recently created a new branch, collection_front, for this development activity.
I think my branch missed your example code. Would you commit it to the new branch please?

> To merge it with real gui I need to get somehow the port name for zmq socket connection
> and "topic" of messages for filtering.
> Is there any method which returns this info for monitoring process?

We don't use zmq filtering.  In our current python code call: setsockopt(zmq.SUBSCRIBE, b'')
We do have a method for finding the zmq port number.  It is a function of the platform number.

In the collection_front branch it looks like this:
    from psdaq.control.collection import front_pub_port
    port = front_pub_port(platform) # getting the zmq port name   
   

Ford, Christopher <caf@slac.stanford.edu> 
2019-02-07, 6:04 PMDubrovin, Mikhail             
  
Hi Mikhail,
> To merge it with real gui I need to get somehow the port name for zmq socket connection
> and "topic" of messages for filtering.

For reference, please see how the socket is initialized in the DacControl constructor, here:

  https://github.com/slac-lcls/lcls2/blob/collection_front/psdaq/psdaq/control/collection.py

    def __init__(self, *, host, platform, timeout):
        self.host = host
        self.platform = platform
        self.timeout = timeout
        # initialize zmq socket
        self.context = zmq.Context(1)
        self.front_sub = self.context.socket(zmq.SUB)
        self.front_sub.connect('tcp://%s:%d' % (host, front_pub_port(platform)))
        self.front_sub.setsockopt(zmq.SUBSCRIBE, b'')

Remember to use the collection_front branch!
Thanks, -caf

Ford, Christopher <caf@slac.stanford.edu> 
Thu 2019-02-28, 4:59 PM             
                 
msg['body']['error']

See liine 21 for example decoding: 
https://github.com/slac-lcls/lcls2/blob/collection_front/psdaq/psdaq/control/daqstate.py

Ford, Christopher 
2019-03-07, 1:59 Dubrovin, Mikhail             

Hi Mikhail,
Yesterday I added a new 'active' field to the JSON object returned by getPlatform().  It can be 0 or 1.
In control_gui, the table displayed by the Partition / Display 
button should only include entries for which body[level][id]['active'] 
== 1.
In other words, the entries with active==0 should be hidden for that button.
See below for an example getPlatform() reply.

getPlatform() reply:

{'drp': {'10924343274495976064': {'active': 1,
                                  'proc_info': {'host': 'drp-tst-dev008',
                                                'pid': 280697}}},
'drp-no-teb': {'-5055484696195183223': {'active': 1,
                                         'proc_info': {'host': 'daq-tst-dev02',
                                                       'pid': 18384}}},
'teb': {'8929931608234960614': {'active': 1,
                                 'proc_info': {'host': 'drp-tst-dev001',
                                               'pid': 279130}}}}

======= Commonly used procmgr commands =========
Start:
  procmgr start psdaq/psdaq/cnf/lab3-dubrovin-test.cnf

Restart (stop then start):
   procmgr restart psdaq/psdaq/cnf/lab3-dubrovin-test.cnf

Status:
  procmgr status psdaq/psdaq/cnf/lab3-dubrovin-test.cnf

2019-03-11 1:39 PM

Hi Mikhail,
I just checked in (to the lcls2 master branch) a new selectPlatform()
interface for updating the 'active' field.
See below for a demonstration of the interface using the CLI.
Thanks,
-caf
(ps-2.0.0) -bash-4.2$ selectPlatform -p6 -h
usage: selectPlatform [-h] [-p {0,1,2,3,4,5,6,7}] [-C COLLECT_HOST]
                      [-t TIMEOUT] [-s LEVEL/PID/HOST] [-u LEVEL/PID/HOST]
optional arguments:
  -h, --help            show this help message and exit
  -p {0,1,2,3,4,5,6,7}  platform (default 0)
  -C COLLECT_HOST       collection host
  -t TIMEOUT            timeout msec (default 2000)
  -s LEVEL/PID/HOST     select (may be repeated)
  -u LEVEL/PID/HOST     unselect (may be repeated)
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ showPlatform -p6
Platform | Partition      |    Node
         | id/name        | level/pid/host (* = active)
---------+----------------+------------------------------------
    6      6/(None)         drp-no-teb/27023/daq-tst-dev02 *
                            teb/379729/drp-tst-dev001 *
                            drp/385531/drp-tst-dev008 *
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ selectPlatform -p6 -u teb/379729/drp-tst-dev001
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ showPlatform -p6
Platform | Partition      |    Node
         | id/name        | level/pid/host (* = active)
---------+----------------+------------------------------------
    6      6/(None)         drp-no-teb/27023/daq-tst-dev02 *
                            teb/379729/drp-tst-dev001 
                            drp/385531/drp-tst-dev008 *
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ selectPlatform -p6 -u drp-no-teb/27023/daq-tst-dev02
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ showPlatform -p6
Platform | Partition      |    Node
         | id/name        | level/pid/host (* = active)
---------+----------------+------------------------------------
    6      6/(None)         drp-no-teb/27023/daq-tst-dev02  
                            teb/379729/drp-tst-dev001 
                            drp/385531/drp-tst-dev008 *
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ selectPlatform -p6 -s
drp-no-teb/27023/daq-tst-dev02 -s teb/379729/drp-tst-dev001
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ showPlatform -p6
Platform | Partition      |    Node
         | id/name        | level/pid/host (* = active)
---------+----------------+------------------------------------
    6      6/(None)         drp-no-teb/27023/daq-tst-dev02 *
                            teb/379729/drp-tst-dev001 *
                            drp/385531/drp-tst-dev008 *
(ps-2.0.0) -bash-4.2$

======================================================
2019-03-11 4:30 PM
Hi Mikhail,
I recently checked in a refinement to selectPlatform() that limits it to
the UNALLOCATED state.
CLI testing is shown below.
At first, selectPlatform returns an error because the system is in
CONNECTED state.
After changing to UNALLOCATED state, selectPlatform succeeds.
What this means for control_gui...
In the "Partition Selection" GUI, the checkboxes and "Apply" button
should only be active while in the UNALLOCATED state.
Thanks,
 -caf

(ps-2.0.0) -bash-4.2$ showPlatform -p1
Platform | Partition      |    Node
         | id/name        | level/pid/host (* = active)
---------+----------------+------------------------------------
    1      1/(None)         drp/388851/drp-tst-dev008 *
                            teb/382746/drp-tst-dev001 *
                            drp-no-teb/4524/daq-tst-dev02 *
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ selectPlatform -p1 -u teb/382746/drp-tst-dev001
Error: selectPlatform only permitted in unallocated state
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ daqstate -p1
last transition: connect  state: connected
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ daqstate -p1 --state unallocated
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ daqstate -p1
last transition: dealloc  state: unallocated
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ selectPlatform -p1 -u teb/382746/drp-tst-dev001
(ps-2.0.0) -bash-4.2$
(ps-2.0.0) -bash-4.2$ showPlatform -p1
Platform | Partition      |    Node
         | id/name        | level/pid/host (* = active)
---------+----------------+------------------------------------
    1      1/(None)         drp/388851/drp-tst-dev008 *
                            teb/382746/drp-tst-dev001 
                            drp-no-teb/4524/daq-tst-dev02 *
(ps-2.0.0) -bash-4.2$

=================================================================
2019-03-12 9:49 AM
Hi Mikhail,
The CLI equivalent of updating checkboxes to select nodes is
selectPlatform.py.

Three steps to read/modify/write:
  1) body = control.getPlatform()
  2) Modify 'active' values (0 or 1) in body dictionary.
  3) control.selectPlatform(body)
https://github.com/slac-lcls/lcls2/blob/master/psdaq/psdaq/control/selectPlatform.py