GPAW and ASE Information

GPAW Convergence Behavior

A talk given by Ansgar Schaefer studying convergence behaviour for rutiles is here (pdf).

General suggestions for helping GPAW convergence are here.

A discussion and suggestions for converging some simple systems can be found here.

Other convergence experience:

GPAW Memory Estimation

The get a guess for the right number of nodes to run on for GPAW, run the
following line interactively:

gpaw-python <yourjob>.py --dry-run=<numberofnodes>
(e.g. gpaw-python graphene.py --dry-run=16)

Number of nodes should be a multiple of 8. This will run quickly
(because it doesn't do the calculation). Then check that the
following number is <3GiB:

Memory estimate
---------------
Calculator  574.32 MiB

Building a Private Version of GPAW

These directions are adapted from the GPAW website, and you should check that to make sure this still makes sense. These particular instructions are in order to have it integrate well with the suncat farm systems.

The user should already have an svn version of ASE somewhere in their home directory – the bleeding edge version of GPAW will often only work with the most up-to-date version of ASE.

Create a space where custom checked-out versions of software live. Probably:

mkdir /nfs/slac/g/suncatfs/${username}/usr/

(where ${username} is your username). From now on this will be referred to as ${usr}. You can optionally create an environment variable in your .cshrc to make the rest of the steps cut-and-paste. Add to your .cshrc:

setenv usr /nfs/slac/g/suncatfs/\{username}/usr/

with your username in place of ${username}. (Then log out and back in.) Create a directory to put a checked-out version of gpaw and set this as the variable ${GPAW_HOME}

mkdir -p \${usr}/lib/svn/gpaw
setenv GPAW_HOME \${usr}/lib/svn/gpaw

Check out the latest version of GPAW.

svn checkout https://svn.fysik.dtu.dk/projects/gpaw/trunk ${GPAW_HOME}

Build the C extensions.

cd ${GPAW_HOME}
python setup.py build_ext |& tee build_ext.log

(Consult https://wiki.fysik.dtu.dk/gpaw/devel/developer_installation.html#developer-installation to make sure that the proper files have been built.)

Install the latest GPAW setups. Create and enter a new directory.

mkdir -p ${usr}/lib/gpaw-setups
cd ${usr}/lib/gpaw-setups

Visit the GPAW setup webpage and get the version number of the latest version of the setups. As of this writing, it is 0.8.7929. Modify the line below with the correct version number, then download and unpack them with

wget http://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-<version>.tar.gz
tar zxf gpaw-setups-<version>.tar.gz
rm gpaw-setups-<version>.tar.gz

Create a path for the environment settings (if it doesn't exist).

mkdir ${usr}/env

If you don't already have one, create an environment file to your SVN version of ASE. Create a file called ${usr}/env/ase-svn which contains the two lines:

#!/bin/csh
setenv PYTHONPATH /path/to/ase/svn:${PYTHONPATH}

where /path/to/ase/svn is the path to your SVN version of ASE (e.g., ${usr}/lib/svn/ase). Next create an environment file to your SVN version of GPAW, called ${usr}/env/gpaw-svn, which contains the following lines:

#!/bin/csh

setenv GPAW_HOME ${usr}/lib/svn/gpaw
setenv GPAW_PLATFORM `python -c "from distutils import util, sysconfig; print util.get_platform()+'-'+sysconfig.get_python_version()"`
setenv PYTHONPATH ${GPAW_HOME}:${PYTHONPATH}
setenv PYTHONPATH ${GPAW_HOME}/build/lib.${GPAW_PLATFORM}:${PYTHONPATH}
setenv PATH ${GPAW_HOME}/build/bin.${GPAW_PLATFORM}:${GPAW_HOME}/tools:${PATH}
setenv GPAW_SETUP_PATH ${usr}/gpaw-setups/gpaw-setups-XXXX:${GPAW_SETUP_PATH}
source ${usr}/env/ase-svn

Note in the above that you will need to substitute in (for XXXX) the correct directory for the version of GPAW setups you have installed.

Lastly you will need to make an executable submission script. If you don't have one already, create a directory that contains your executables:

mkdir ${usr}/bin

To make sure you can execute files from this directory, you will need to put the path to this in your PATH environment. Add the following line to your .cshrc file:

setenv PATH ${usr}/bin:${PATH}

Now create your submission script that will be used to submit gpaw jobs from your svn version. Create a file called ${usr}/bin/gpaw-svn-bsub and place the following three lines in it.

#!/bin/tcsh
source ${usr}/env/gpaw-svn
/afs/slac/g/suncat/bin/dobsub -a openmpi -R "span[ptile=8]" gpaw-python $argv

Then change the mode of this file to be executable.

chmod u+x ${usr}/bin/gpaw-svn-bsub

Run the tests as described on the GPAW website:

https://wiki.fysik.dtu.dk/gpaw/install/installationguide.html#running-tests

If everything is working right, you should be able to submit scripts using your svn version with

gpaw-svn-bsub <scriptname>.py