This script lives in /reg/g/psdm/tutorials/examplePython/mpiGather.py and demonstrates how to save small pieces of information on every event (in parallel) and then gather those pieces together onto one core.
#run this with: mpirun -n 2 python mpiGather.py
from psana import *
import numpy as np
ds = DataSource('exp=xpptut15:run=54:smd')
det = Detector('cspad',ds.env())
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
intensity = []
for nevent,evt in enumerate(ds.events()):
if nevent%size==rank: # different ranks look at different events
img = det.image(evt)
intensity.append(img.sum())
if nevent==3: break
allIntensities = comm.gather(intensity) # get intensities from all ranks
if rank==0:
allIntensities = np.concatenate((allIntensities[:])) # put in one long list
print allIntensities
MPI.Finalize()
Note that this example uses the lower-case "gather" method (not "Gather"). This means it can transmit python objects (via "pickle") which is more flexible and easier, but it is less performant than the capitalized "Gatherv" approach, which works more efficiently (but is more difficult to use). An example with Gatherv is here.
Overview
Content Tools