Page History

...

• Reduce is necessary for MTIP (cannot be a looser send/receive pattern)
• Use of Reduce requires user-control of loops to ensure that users can call it the same number of times on all cores (even at the ragged end-of-run).  This can be done, for example, with a Pull-model (not a callback or Push).  Users may have to explicitly specify how many times they want to Reduce (not elegant, but workable, although not in a live-streaming case where you don't know how long the run is going to be).  A better solution could be to have SMD0 inject step transitions at a regular interval, which get broadcast to all BD cores which could trigger the reduce.
• Option (1) above feels like it is covered by (2) (work done on BD cores) and (4) (work done on SRV cores)
• Option (4) would require psana SRV cores to be made available for expert-user-level programming (i.e. not doing usual h5 production pattern)
• Option (4) could use an "mpiChannel" package that Johannes has worked on with mpi3 one-sided communication.  Note: a few years ago one-sided communication performed poorly in tests cpo did, but Johannes believes it is much better now.

...