To run the parallel data reading application:

  • install anaconda (pretty easy to do)
  • "conda env create -n lcls2 -f env_create.yaml" using env_create.yaml file shown below
  • activate the conda environment with "conda activate lcls2"
  • git clone https://github.com/slac-lcls/lcls2.git
  • "cd lcls2"
  • make sure you have gcc>=6
  • build software with "./build_python2_psana.sh"
  • export PYTHONPATH=`pwd`/install/lib/python2.7/site-packages/
  • run software with "nosetests psana/psana/tests/test_manager.py:Test.test_parallel"
  • after doing the previous line (which generates some small "fake data") you can run the software by hand with "mpirun -n 3 python psana/psana/tests/user.py"

Our "big data" is in the directory .tmp/*.xtc and our index information (fseek offsets and other small physics data to be used for things like filter/veto are in .tmp/smalldata/*.smd.xtc).  One pair of files is generated (.xtc and .smd.xtc) per LCLS detector, roughly speaking.  We expect to have about 50-100 pairs of files for a typical LCLS2 run (20GB/s).

env_create.yaml:

channels:
  - defaults
dependencies:
  - python=2.7
  - cmake
  - numpy
  - cython
  - mpi4py
  - nose
  - matplotlib
  • No labels