• Created by Unknown User (jscott), last modified on Apr 05, 2017

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

It may be advantageous or necessary to do data analysis without a connection to SLAC. Below are the steps to install psana along with the data to be used in the analysis onto a local machine for later, offline use on a RHEL 5, 6 or 7 machine. In the simplest case, a single conda environment is maintained but multiple conda environments can be created and used if necessary. The steps for both cases are shown below.

Installation of a Single Conda Environment

  1. Go to https://conda.io/miniconda/
  2. Install the Python 2.7 64 bit bash installer (unless your OS is 32 bit but it most likely will not be)
  3. Find the installed file Miniconda3-latest-Linux-x86_64.sh
    1. It will probably be in Downloads, so do cd Downloads
    2. Run it with bash Miniconda3-latest-Linux-x86_64.sh
  4. Add miniconda2 to PATH
    1. The installation will automatically add the path to the .bashrc file
    2. Close and reopen terminal and check by typing conda list which will print out the installed packages
    3. If conda is not found, follow step 5, else skip to step 6
  5. If not already created, create a .bash_profile file in your home directory

    1. Add the following script to it. This checks for the .bashrc and will run it on start up

      #!/bin/bash
 
if [ -f $HOME/.bashrc ]; then
	source $HOME/.bashrc
fi

  6. Run conda update -y conda to update miniconda

  7. Install Psana Conda with conda install -y --channel lcls-rhel<num> psana-conda where <num> is the the RHEL version

  8. Copy the experiment database from /reg/g/psdm/data/ExpNameDb/experiment-db.dat 

    1. Make a directory in the home directory with mkdir -p psdm/data/ExpNameDb 

    2. Copy the database

      1. With rsync: rsync -t psexport:/reg/gpsdm/data/ExpNameDb/experiment-db.dat ~psdm/data/ExpNameDb/

      2. With SCP: scp -p psexport:/reg/g/psdm/data/ExpNameDb/experiment-db.dat ~psdm/dataExpNameDb/ 

  9. Copy the experiment data that will be used for analysis. This step requires patience if many runs will be copied

    1. For example, downloading run 54 from the experiment xpptut15, the steps were taken:
      1. First create the required directories with mkdir -p psdm/xpp/xpptut15/
      2. Then these files were copied

        1. From /reg/d/psdm/xpp/xpptut15/xtc to ~/psdm/xpp/xpptut15/xtc: 

          -bash-4.2$ ls /reg/d/psdm/xpp/xpptut15/xtc/ | grep 54
e665-r0054-s00-c00.xtc
e665-r0054-s01-c00.xtc
e665-r0054-s02-c00.xtc
e665-r0054-s03-c00.xtc
e665-r0054-s04-c00.xtc
e665-r0054-s05-c00.xtc

        2. From /reg/d/psdm/xpp/xpptut15/xtc/index to ~/psdm/xpp/xpptut15/xtc/index:

          -bash-4.2$ ls /reg/d/psdm/xpp/xpptut15/xtc/index | grep 54
e665-r0054-s00-c00.xtc.idx
e665-r0054-s01-c00.xtc.idx
e665-r0054-s02-c00.xtc.idx
e665-r0054-s03-c00.xtc.idx
e665-r0054-s04-c00.xtc.idx
e665-r0054-s05-c00.xtc.idx

        3. From /reg/d/psdm/xpp/xpptut15/xtc/smalldata to ~/psdm/xpp/xpptut15/xtc/smalldata:

          -bash-4.2$ ls /reg/d/psdm/xpp/xpptut15/xtc/smalldata | grep 54
e665-r0054-s00-c00.smd.xtc
e665-r0054-s01-c00.smd.xtc
e665-r0054-s02-c00.smd.xtc
e665-r0054-s03-c00.smd.xtc
e665-r0054-s04-c00.smd.xtc
e665-r0054-s05-c00.smd.xtc
        4. And the entire directory /reg/d/psdm/xpp/xpptut15/calib to ~/psdm/xpp/xpptut15/calib
    2. Copy using either rsync or SCP. SCP may be simpler in this case because it copies the data of a symbolic link which is what is desired.
  10. Two environment variables must be set, SIT_DATA and SIT_PSDM_DATA by adding the following commands to the .bash_profile file
    1. export SIT_DATA=$HOME/psdm/data
    2. export SIT_PSDM_DATA=$HOME/psdm
    3. If step 5 was not done, create a .bash_profile file instead and begin it with #!/bin/bash 

Psana Conda can now be used. If new packages need to be added, removed, etc., the simple conda comands can be used. A link the specifics of these commands is given in the section below.

Installation of Multiple Conda Environments

More information on managing Conda environments can be found here. Begin by following the steps above for the single environment but do not set the environment variables (i.e. step 10). It is recommended to have these variables set when the environment is activated and unset when deactivated. First create the environment with conda --create <environment name> -c lcls-rhel<num> psana-conda where <environment name> is the name of the environment to be created and <num> is the RHEL version. Then follow the steps below, for a given environment examplenv, which were pulled from the link given before.

  1. Change directories to the environment directories like cd ~/miniconda2/env/examplenv (the actual path may be slightly different but it will be under env in the conda package
  2. Create the following directories and files:
    1. mkdir -p etc/conda/activate.d
    2. mkdir -p etc/conda/deactivate.d
    3. touch etc/conda/activate.d/env_vars.sh
    4. touch etc/conda/deactivate.d/env_vars.sh
  3. Add the following to this files

    1. To etc/conda/activate.d/env_vars.sh: 

      #!/bin/sh

export SIT_DATA=$HOME/psdm/data
export SIT_PSDM_DATA=$HOME/psdm

    2. To /etc/conda/deactivate.d/env_vars.sh:

      #!/bin/sh

unset SIT_DATA
unset SIT_PSDM_DATA

Now examplenv will have the correct environment variables when activated with the command source activate examplenv and deactivate with source deactivate. And packages may be added, removed, upgraded, downgraded, etc. from this conda environment like usual. For information on this, follow link given at the beginning of this section.

Example

It's important to test whether or not the above steps actually worked. To do this, you can use any code that imports experiment data with psana. Below is the sample code from the Event Iteration page from the Building Blocks section, here. If following the multiple environments steps, make sure to activate the environment with the command source activate examplenv.

from psana import *

ds = DataSource('exp=xpptut15:run=54:smd')
nevent = 0
 
for evt in ds.events():
    nevent += 1
    if nevent == 3:
        break


print 'Processed', nevent, 'events.'

If successful, you should see the output:

Processed 3 events.

 

 

 

cd /home/jsmith/anaconda3/envs/analytics
  • No labels