You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Life of a Partition

  • collection manager (CM) alive (long lived)
  • restart .cnf
  • python-proxies comes up
  • proxies reports existence/identity (Detector/DRP, EB, monshmserverAMI, monshmserverPsana, DTI, XPM, Control) to CM
  • control level queries CM for list of all processes in the platform
  • control level notifies CM of processes it wants for partition
  • CM tells all processes about all the nodes in the partition
  • processes figure out how many "ports" (an abstract idea, could be q-pair for IB) they need given the nodes in the partition and report back their ports to CM
  • CM treats ports as opaque information
  • CM also manages connection information for detector-to-dti-link-to-drp-node mapping table (connection information) which is used by the DTI-proxy to compute a dti-link-mask.
  • after gathering all port information, CM broadcasts all ports, as well as CM-assigned "id" to proxies
  • proxies are specialized for particular levels (DRP, EB, DTI) and select ports they are interested in
  • proxies make appropriate connections based on their CM-assigned ID number
  • proxies report that they are connected, or failure (e.g. if DTI-link-mask is already allocated, or IB connection fails)
  • after all proxies report in, manager notifies control level that transitions can be sent
  • control level starts to send transitions
  • graceful teardown: unconfigure/unmap transitions complete.  CM sends disconnect message to proxies
  • ungraceful teardown: feels like we need to restart, since things can be broken in various ways

Contents of .cnf file

  • collection related
    • EB use IB or IP (goes to Ric)
    • ip of CM (determines network)
    • (maybe) map of detector-to-dti-link-to-drp-node-pgp-lane
    • monshmserver group IDs (e.g. psana/AMI)
  • non-collection
    • static detector discovery?  (dynamic feels difficult)
    • outfile paths
    • configdb info
    • shmem names
    • drp algs (e.g. ROI)

Design Decisions

  • one CM per .cnf (long-lived process)
  • does collection control the BOS?  We believe not.  Doesn't change frequently
  • does monshm participate in collection?  yes
  • how flexible are DRP exe's:  ideally multiple dlopen's, so long-lived, but feels unlikely given constraints
  • more seamless restarts?  on-the-fly restarts feel very difficult.  would complicate the code significantly.  better to keep code simple.  AMI can be restarted independently.
  • use zmq select to timeout
  • use python-proxies for C++.  Communicate python-created datagrams to C++ via zmq.

Issues

  • Will procmgr scale?  Can we reduce restart time?
  • Does map of detector-to-dti-link-to-drp-node-pgp-lanes live in cnf or database?
  • How do we manage the BOS?
  • No labels