GPAW to QMCPACK converter

[tiihonej]

I'm not really requesting as much as offering to write a converter from GPAW orbitals to QMCPACK. I'm anticipating some usability benefits (e.g. native and versatile python bindings) and potential new users from this capability. However, it would really help me out to get some answers from the experts instead of digging them up from the source codes.

Let me start with the following topics:

1. GPAW capabilities

GPAW currently runs with three alternative representations of the wavefunction: real-space grid, plane-waves and LCAO. As far as I know, each of these can be converted for QMCPACK, albeit quite differently. The priority will be the plane-wave representation, and the rest can be left for later.

GPAW also implements at least the following Hamiltonians: all-electron, projector-augmented waves (PAW), and a limited support for norm-conserving KB pseudopotentials in the UPF format. The priority is AE, because it can be readily implemented and tested. The PAW method won't do. The UPF route is important, but a more general support for correlation-consistent ECPs will need consideration by the GPAW developers.

The GPAW release notes indicate support for simulation of (non-)collinear spin polarizations and spin-orbit coupling. I'll save these for later and start with spin-unpolarized calculations.

Does this sound correct and feasible?

2. The converter

GPAW is written in Python, so it seems natural to write the converter also in Python (in contrast to the present Fortran or C++ converters). The converter could take a GPAW restart file (.gpw) as an input, and if everything checks out, produce an output ES-HDF using h5py and a template XML input. All the modes outlined above could be considered by the same script.

A feature like that could ship with the GPAW package, if they are willing, and be invoked with a single command. Alternatively, it could be a part of Nexus or a standalone executable Python script.

3. About ES-HDF

I've been reverse engineering to understand the soul of an appropriate ES-HDF file for the plane-wave representation. Feel free to address the following propositions and questions:

• There is no supported representation for a real-space mesh, but the appropriate way (e.g. from RMG) is conversion to PWs, according to ES-HDF, which QMCPACK will convert to bsplines in the runtime.
• The plane-wave bases are expressed as integer Miller indices, because they take the least storage space.
• The are several complex arrays associated with individual PW bases, which represent the electronic densities and a few lowest wave functions for each spin species. Are there implicit or explicit expectations of normalization within each individual quantity, or consistency between them? Namely, is the electronic density basically the ground state wavefunction squared, or is it more complicated than that? It somewhat puzzles me that (at least from QE) the densities and wavefunctions have differently sized PW bases.
• The most recent ES-HDF version is 2.1.0 (?), which should be preferred. Is there a specification for this file type?

I appreciate any comments and guidance with this project!

[prckent]

Definitely write everything in python. I think if we were starting again we would try to do that for all the converters.

Ask questions about ESHDF as you go. We might want to update it. It has evolved accidentally over the years and now that more codes are being interfaced might need some polish. Definitely starting with planewaves. All electron won't be very useful in that case.

RMG now implements semilocal evaluation so that the effect/errors of KB projectors can be checked. The UPF spec actually allows for these, though few/no codes make use of this flexibility.

[tiihonej]

Thank you, Paul. I need to do some learning about the RMG capabilities that you mention.

Meanwhile, I took some time to find out if there's already a scenario suitable for testing the conversion from GPAW to QMCPACK. It appears we are close but not quite there yet. The crux of the matter seems not to be the wavefunction conversion itself but the implementation of pseudopotentials in the UPF format.

QMCPACK/ppconvert:

• Only supports so-called UPF v1 input specification
• Ideal solution: implement QMCPACK/ppconverter support for UPF v2 or xml
• Short-term solution: use ECPs that are supported by ppconvert (and thus, outdated by the UPF spec)

QE:

• Supports both UPF v1 and v2 and gives consistent results
• Allows conversion from v1 to v2 or xml using upfconv.x tool, but not back to v1

GPAW:

• Supports UPF v2 input spec (save for minor bugs and other impracticalities)
• Tries to cope with other UPF formats, but fails parsing ccECPs (v1-ish) at least
• WIP, but some tests indicate serious trouble with implementing the pseudo wavefunction (see below). Don't know what to make of it, but it may need attention from the devs.

OPIUM:

• Only outputs UPF v1 (probably explains the present state of QMCPACK)
• Question: where does one get support for qmcpack output format in OPIUM? A private repository?

Feel free to correct me if I'm wrong.

What lead to me some of these conclusions was trying (among other things) to run H2 (ccECP) in GPAW. The ccECP for H looks quite exotic, and while QE works just fine, this blows up GPAW somehow. This tells me that the pseudo wavefunction or something else is not correctly implemented. What GPAW actually claims to support is the SG15 package of pseudopotentials, which doesn't seem to ship the pseudo wavefunction in a numerical form. Instead, it's constructed on runtime in some LCAO basis and I don't know if it is by design or not. It's an open investigation.

I understand that the focus of my reports is quite scattered at the moment, but I'm just trying to bracket a reasonable test case for even a minimalistic converter tool. Maybe I should try Lithium instead.

[tiihonej]

Update: the converter works fine for spin-unpolarized PW calculations with ccECP pseudopotentials with non-trivial k-point arrangement!

Here's some VMC data based on bulk silicon with ccECP pseudos, 64-atom supercell, tilematrix -4232-2332-2, kgrid=(2,2,2) for twist averaging.

Data based on GPAW SCF, then NSCF with custom set of 512 k-points:

                            LocalEnergy               Variance           ratio 
qmc.g000  series 0  -501.469573 +/- 0.048709   5.330615 +/- 0.125213   0.0106 
qmc.g001  series 0  -501.481654 +/- 0.035584   5.197326 +/- 0.115067   0.0104 
qmc.g002  series 0  -501.480678 +/- 0.039378   5.397799 +/- 0.120144   0.0108 
qmc.g003  series 0  -501.560259 +/- 0.036211   5.395127 +/- 0.113085   0.0108 
qmc.g004  series 0  -501.460322 +/- 0.047430   5.147071 +/- 0.112895   0.0103 
qmc.g005  series 0  -501.523456 +/- 0.036273   5.379292 +/- 0.107853   0.0107 
qmc.g006  series 0  -501.473742 +/- 0.038072   5.391445 +/- 0.109762   0.0108 
qmc.g007  series 0  -501.539309 +/- 0.039533   5.401939 +/- 0.115491   0.0108 
avg  series 0  -501.497350 +/- 0.014537   5.558456 +/- 0.035883   0.0111 

qmc.g000  series 0  MPC                   =  -129.301713 +/- 0.089811 
qmc.g001  series 0  MPC                   =  -129.296786 +/- 0.101683 
qmc.g002  series 0  MPC                   =  -129.171797 +/- 0.082912 
qmc.g003  series 0  MPC                   =  -129.222577 +/- 0.087976 
qmc.g004  series 0  MPC                   =  -129.085968 +/- 0.124102 
qmc.g005  series 0  MPC                   =  -129.114474 +/- 0.099612 
qmc.g006  series 0  MPC                   =  -129.123375 +/- 0.083154 
qmc.g007  series 0  MPC                   =  -129.262844 +/- 0.094863 
avg  series 0  MPC                   =  -129.205605 +/- 0.033828

Corresponding VMC data based on QE orbitals:

                            LocalEnergy               Variance           ratio 
qmc.g000  series 0  -501.467709 +/- 0.042793   5.276210 +/- 0.138328   0.0105 
qmc.g001  series 0  -501.440808 +/- 0.041467   5.351175 +/- 0.124553   0.0107 
qmc.g002  series 0  -501.519156 +/- 0.045116   5.417494 +/- 0.174620   0.0108 
qmc.g003  series 0  -501.459627 +/- 0.035851   5.451935 +/- 0.122811   0.0109 
qmc.g004  series 0  -501.526598 +/- 0.037161   5.263695 +/- 0.120848   0.0105 
qmc.g005  series 0  -501.616965 +/- 0.041030   5.397737 +/- 0.127271   0.0108 
qmc.g006  series 0  -501.560535 +/- 0.041422   5.420456 +/- 0.130393   0.0108 
qmc.g007  series 0  -501.466011 +/- 0.040119   5.291218 +/- 0.109746   0.0106 
avg  series 0  -501.509945 +/- 0.014386   5.606010 +/- 0.047275   0.0112

qmc.g000  series 0  MPC                   =  -129.190287 +/- 0.168933 
qmc.g001  series 0  MPC                   =  -129.368029 +/- 0.102724 
qmc.g002  series 0  MPC                   =  -129.248594 +/- 0.097726 
qmc.g003  series 0  MPC                   =  -129.161480 +/- 0.081334 
qmc.g004  series 0  MPC                   =  -129.213795 +/- 0.092018 
qmc.g005  series 0  MPC                   =  -129.327133 +/- 0.090667 
qmc.g006  series 0  MPC                   =  -129.196086 +/- 0.102036 
qmc.g007  series 0  MPC                   =  -129.135441 +/- 0.097629 
avg  series 0  MPC                   =  -129.236165 +/- 0.032349

Differences in the averages are statistically insignificant, although some of the individual twists are not. Not sure if it's okay or not. There is some more work to do to make the bandinfos match between GPAW and QE, but I'll spare the details for the moment.

Right now the tool is a standalone python script gpaw4qmcpack to be executed on the command line, much like all the other converters. It can be found here under the nexus executables (although that's probably not the best place for it in the end):
https://github.com/tiihonej/qmcpack/tree/gpaw_converter

There's a bunch of action items to do including tests, examples and documentation. Foremost, it's important to figure out the best way to use the converter in a workflow, and implement the necessary framework. When we've settled that, and whatever else will be needed, I'd like to make an initial PR to get things in motion.