Most Recent GPAW Versions
Farm | New Setups (0.9.9672) | Old Setups (0.8.7929) |
suncat/suncat2 | gpawv40 | gpawv41 |
suncat3 | gpawv40a | gpawv41a |
Submitting Batch Jobs
Code Block |
---|
gpaw-bsub myscript.py ("old/stable" version) gpaw-ver-bsub <version> myscript.py (user-selected version) |
GPAW Convergence Behavior
5/1/2013: Jun Yan reports better convergence behavior with a new version of gpaw (v40/v41) that has an improved eigensolver. Use code like this:
...
System | Who | Action |
---|---|---|
Graphene with vacancy | Felix Studt | Increase Fermi Temp from 0.1 to 0.2, use cg |
Graphene with vacancy | Chris O'Grady | change nbands from -10 to -20, MixerDif(beta=0.03, nmaxold=5, weight=50.0) |
Nitrogenase FeVCo for CO2 reduction | Lars Grabow | use Davidson solver (faster as well?), although later jvarley said MixerSum |
Several surfaces | Andy Peterson | Broyden mixer with Beta=0.5 |
TiO2 | Monica Garcia-Mota | MixerSum(0.05,6, 50.) |
MnxOy | Monica Garcia-Mota | Broyden MixerSum |
Co3O4 | Monica Garcia-Mota | Davidson eigensolver, MixerSum(beta=0.05, nmaxold=5, weight=50) or MixerSum(beta=0.05, nmaxold=6, weight=100) |
MnO2 with DFT+U U=+2eV | Monica Garcia-Mota | Marcin suggests we disable the DipoleCorrectionPoissonSolver (not yet tested) |
MnO2 with DFT+U U=+2eV | Monica Garcia-Mota | Henrik Kristofferson suggests: convergence is easier with high U (U=4eV) and then |
MnO2 with DFT+U U=+2eV | Monica Garcia-Mota | (from Heine) increase U in steps of say 0.1 (or smaller) and reuse the density and/or wave functions from the previous calculation? This tends to reduce the problem of being trapped in meta-stable electronic states, and it also makes convergence easier. Monica later reported that this helped. |
Cu | Ask Hjorth Larsen | first mixer parameter should probably be 0.1 for faster convergence, because it has a low DOS at the Fermi level. (Other transition metals may require lower values.) |
N on Co/Ni (with BEEF) | Tuhin | rmm-diis and MixerSum(beta=0.1, nmaxold=5, weight=50) |
Other Tricks:
- To speed up the poisson solver, use something like gpts=h2gpts(h=0.18, atoms.get_cell(), idiv=8) to get a nicely divisible-by-large-power-of-2 grid. This helps the "multi grid" poisson solver.
- experiment with Fermi smearing to help improve convergence
- experiment with number of empty bands to help improve convergence
be sure to specify nbands, otherwise GPAW will add "plenty" of bands which is very expensive in FD calculations. nbands=6*\[number of atoms\] should be more than enough.Wiki Markup
GPAW Planewave Mode
Jens Jurgen has a post here that discusses how to select plane wave mode in your script.
...
- To get parallelization over bands, we can at the moment only use the rmm-diis eigensolver (cg and davidson don't work).
- The number of bands must be divisible by the number of CPUs, according to Jens Jurgen.
- At the moment there is no dipole correction in planewave mode.
- Density mixing is still done in real-space
GPAW Geometry Optimizations
Thoughts from Lin and Thomas: With GPAW one can do geometry optimizations a factor of 10 faster in LCAO mode (with smaller memory requirements). Then it's necessary to "tweak" the optimization with a little bit of running in FD mode.
...
AJ adds: I would also warn against using LCAO as an initial guess for NEB calculations. I have tried this for 2 different systems and it turned out to be a tremendous waste of time. The NEB did not converge much faster with LCAO, and when I used the LCAO images as an initial guess for finite difference mode it still took several restarts to converge. I have had better luck using decreased k-point/grid spacing and relaxed convergence criteria as an initial optimization for NEBs.
GPAW Memory Estimation
The get a guess for the right number of cores to run on for GPAW, run the following line interactively:
...
Code Block |
---|
Memory estimate --------------- Calculator 574.32 MiB |
Tips for Running with BEEF
If you use the BEEF functional:
- use xc='BEEF-vdW'
- the code parallelizes over 20 cores (because of the way the VDW contribution is calculated)
- you typically need a lot of memory, even for small calculations. For larger calculations you sometimes have to use more than 20 cores, just to get enough memory (you don't get additional parallelization).
- the GPAW memory estimates are incorrect
- it is typically best to run on the suncat2 (more memory per core) or, even better, the suncat3 farm (more memory per core, plus a much faster infiniband node-node interconnect)
- use the suncat3 farm, which has a fast (4GB/s) infiniband interconnect.
Building a Private Version of GPAW
- Use svn to check out the version of GPAW that you want to use (described here).
- copy /afs/slac/g/suncat/share/scripts/privgpaw.csh into whatever directory you like
- edit the two variables GPAW_BASE ("base" GPAW release that you want to re-use for numpy, mpi etc.) and GPAW_HOME (directory where you checked out GPAW)
- Use the commands:
Code Block ./privgpaw.csh build (build) ./privgpaw.csh build-rh6 (build for the suncat3 RedHat6 farm. can use same source directory as build for old farms) ./privgpaw.csh test (run gpaw self-tests) ./privgpaw.csh gpaw-bsub <arguments> (submit batch job) ./privgpaw.csh gpaw-bsub-rh6 <arguments> (submit batch job to suncat3 RedHat6 farm) ./privgpaw.csh <cmd> (run <cmd> interactively, using privgpaw.csh environment. e.g. "gpaw-python junk.py")
...
- The syntax for the "bsub" option is identical to the gpaw-bsub command described here.
- The above assumes that the base release is compatible with your checked out GPAW version. Talk to cpo if you have questions about this.
- The above doesn't include support for a private version of ASE. When that becomes important we will add it.
Versions
Version | Date | Comment |
gpawv1 | 9/14/10 | gpaw 0.7.6383, first attempt at SUNCAT gpaw installation |
gpawv2 | 9/21/10 | move from standard lapack libraries to acml,blacs,scalapack to improve performance |
gpawv3 | 9/30/10 | move from ase 3.4.0 to 3.4.1 |
gpawv4 | 10/15/10 | check out a gpaw svn branch for hannesj/aleksandra |
gpawv5 | 10/15/10 | copy of v3, add scipy |
gpawv6 | 10/22/10 | copy of v5, move GPAW to head for venkat (gpaw 0.8.0) |
gpawv7 | 11/9/10 | gpaw 0.7.2.6974, switch from ifort v9 to gcc 4.1.2, simplify setupenv |
gpawv8 | 11/16/10 | use goto2 blas and open64 to improve performance |
gpawv9 | 12/9/10 | copy of gpawv8, but HEAD of gpaw/ase to fix non-orthorhombic problem for strabo/pgmoses |
gpawv10 | 12/9/10 | copy of gpawv9, but put in numpy site.cfg to try to use acml for numpy |
gpawv11 | 1/10/11 | copy of gpawv9, a special version for hildur to use |
gpawv12 | 6/27/11 | copy of gpawv9, a special version for jewe to use (preliminary BEEF version) |
gpawv13 | 7/8/11 | softlink to gpawv9, change setupenv to use torque/maui aware openmpi |
gpawv14 | 8/10/11 | copy of gpawv9, update to HEAD of gpaw svn, and HEAD of ase svn (for wulff construction) requested by Lin |
gpawv15 | 10/5/11 | test version built with icc/mkl |
gpawv16 | 10/12/11 | version like gpawv14, but with python2.7 so we can include hdf5 support for Lin. Only serial version functional with /opt/TWWfsw/python27/bin/python |
gpawv17 | 11/16/11 | copy gpaw/ase from v9, but use icc/ifort/mkl to improve performance |
gpawv18 | 1/3/12 | copy gpaw/ase from v17, but mkl 10.3 to fix LD_PRELOAD hack |
gpawv19 | 1/20/12 | like gpawv17, but GPAW from head of SVN to get first BEEF release, also most recent setups, and more recent ASE |
gpawv20 | 1/21/12 | mkl 10.3 with scalapack, gpaw from head of SVN, newer ASE, new setups. serial version broken because removed static linking? |
gpawv21 | 2/1/12 | uses v17 executables, but adds head of ASE and version 0.3.2 of CMR to PYTHONPATH (requested by AJ) |
gpawv22 | 2/6/12 | like v19 but uses GPAW/ASE from head of SVN to pickup BEEF error estimation. also enables scalapack, CMR |
gpawv23 | 3/26/12 | like v22 but uses GPAW v0.9, required ASE 3.6.0, required CMR 0.3.3.580 |
gpawv24 | 4/27/12 | like v23 but uses head of GPAW and new ASE to pick up fix for vdW parallel NEB |
| 6/22/12 | NOTE THAT GPAW VERSIONS OLDER THAN 25 WILL NO LONGER WORK BECAUSE OF PYTHON-LIBS SECURITY PATCH PROBLEM |
gpawv25 | 6/20/12 | copy of v22 but rebuilt to fix python-libs patch problem |
gpawv26 | 6/20/12 | copy of v17 but rebuilt to fix python-libs patch problem |
gpawv26a | 11/16/12 | (suncat3 farm only) copy of v26 ("old/stable") gpaw, but some tests fail still |
gpawv27 | 6/20/12 | copy of v24 but rebuilt to fix python-libs patch problem |
gpawv27a | 11/16/12 | v27 rebuilt for suncat3 farm (softlink to v35) |
gpawv28 | 6/22/12 | copy of v24 but try python2.7 with updated numpy/scipy/matplotlib. gtk graphics broken |
gpawv29 | 7/10/12 | copy of v16 but rebuilt gpaw because of the python-libs problem. only serial version functional with /opt/TWWfsw/python27/bin/python |
gpawv30 | 7/23/12 | head of GPAW svn to get FixedOccupations class in occupations.py for Joel. But turned out v27 already had that. Also newer ASE. DISABLED this because wasn't optimally built. |
gpawv31 | 10/12/12 | same as V30, but with latest official setups. DISABLED this because need to update to latest GPAW for latest setups. |
gpawv32 | 11/7/12 | head of GPAW svn but with latest official setups |
gpawv32a | 11/14/12 | v32 rebuilt for suncat3 farm (softlink to v33) |
gpawv33 | 11/14/12 | (suncat3 farm only) like v32, but rebuilt for Sandy Bridge and RHEL6 |
gpawv34 | 11/14/12 | (suncat3 farm only) like v33, but uses older setups DISABLED because saw strange results |
gpawv35 | 11/16/12 | (suncat3 farm only) copy of v27 gpaw, so we can use older setups |
gpawv36b | 11/16/12 | (gpu farm only) checkout of Jun's branch rpa-gpu-expt built for the new gpu farm. old setups. |
gpawv37b | 1/28/13 | (gpu farm only) checkout of Samuli's cuda branch built for the new gpu farm. old setups. |
gpawv38 | 2/1/13 | build of libxc 2.0.0 branch for doylead/alevoj so they can use HSE. |
gpawv39 | 2/25/13 | copy of v27 with newer ASE 3.6.1.2981 |
gpawv40 | 3/24/13 | latest gpaw from trunk to fix get_all_electron_density problem. new setups. two tests fail running with -j 8, but work running tests without -j 8 |
gpawv41 | 3/24/13 | like v40, but old setups. |
gpawv42 | 4/18/13 | copy of v29 environment to pick up hdf5 for Lin. parallel version broken. latest gpaw from trunk to fix get_all_electron_density problem. old setups (so self-tests don't work). still need to use /opt/TWWfsw/python27/bin/python, I believe |
gpawv43 | 6/28/13 | first version built with new libxc |
gpawv44 | 10/7/13 | like v43, but with newer ASE for chank to pick up new minimahopping.py |