add real-space RMG converter to convert4qmc #3351

kgasperich · 2021-08-09T20:48:50Z

Proposed changes

This PR adds RMG hdf5 output to the list of formats supported by convert4qmc (this hdf5 output is generated when the RMG input contains write_qmcpack_restart = "true").

Handling of pseudopotential files is not perfect, so this should be cleaned up in a later PR (the PP terms in the Hamiltonian in the generated *.qmc.in-wf{j,noj}.xml will always reference X.qmcpp.xml where X is the appropriate element symbol, but convert4qmc will not generate/link/move/copy/convert these from the PP files used by RMG).

What type(s) of changes does this code introduce?

New feature

Does this introduce a breaking change?

No

What systems has this change been tested on?

cooley (RHEL 7; x86_64) icc 19.1.0.166

Checklist

Update the following with a yes where the items apply. If you're unsure about any of them, don't hesitate to ask. This is
simply a reminder of what we are going to look for before merging your code.

Yes. This PR is up to date with current the current state of 'develop'
Yes. Code added or changed in the PR has been clang-formatted
Yes*. This PR adds tests to cover any new code, or to catch a bug that is being fixed
Yes. Documentation has been added (if appropriate)

qmc-robot · 2021-08-09T20:49:03Z

Can one of the admins verify this patch?

PDoakORNL · 2021-08-11T16:52:50Z

any tests at all?

kgasperich · 2021-08-11T18:20:45Z

I can add one (or more) to https://github.com/QMCPACK/qmcpack/tree/develop/tests/converter
The smallest ones I have right now (all spin-unpolarized) are 5, 80, 95, and 136 MB (size of the hdf5 output from RMG), and I can add any subset of those.
In terms of coverage, it might be good to have a spin-polarized system, but the smallest of those that I have right now is 1.3 GB; I can try to converge that on a coarser grid to get the size down.

PDoakORNL · 2021-08-11T20:49:37Z

#3354 is relevant to this. We probably don't want to hold the PR for that issue to be resolved.

kgasperich · 2021-08-11T21:44:45Z

Ok, I added two tests (2.5 and 5 MB), one spin-polarized and one unpolarized.
We can move things around as necessary depending on how #3354 is resolved

prckent

Thanks Kevin.

Questions on the test choices:

what does the k-point grid in the diamond test case achieve? can we save some filespace by using 1x1x1?

for the o atom case, can we make it even more distinct? it is spin polarized but can we readily support open boundary conditions? (relatively unique feature of rmg, hits more code paths)

prckent · 2021-08-19T20:05:23Z

Start testing in-house

prckent · 2021-08-20T13:27:25Z

Test this please

kgasperich · 2021-08-25T16:32:37Z

@prckent

what does the k-point grid in the diamond test case achieve? can we save some filespace by using 1x1x1?

In the interest of expediency I was just selecting a subset of the existing RMG example tests; I wanted to use the O atom for a spin-polarized example, and because that was 1x1x1 I thought it might provide better coverage to use a larger grid for the spin-unpolarized test (diamond). The diamond test uses a 4x4x4 grid, but I can change that to 1x1x1 if the grid isn't important (or to 1x1x2 or 2x2x2 if we want something smaller than what we have, but larger than 1x1x1).

for the o atom case, can we make it even more distinct? it is spin polarized but can we readily support open boundary conditions? (relatively unique feature of rmg, hits more code paths)

I'm not sure I understand the question; I was under the impression that RMG only supported periodic boundary conditions.
https://github.com/RMGDFT/rmgdft/blob/606ae210d50b1bf76e228ff98f1a9058e560c499/Input/ReadCommon.cpp#L423-L426
Should I add an option to convert4qmc to generate inputs with bconds set to n n n?

prckent · 2021-09-02T15:32:41Z

Use the smallest grid for file size. Forget my comments on non-periodic settings.

kgasperich · 2021-09-28T21:56:32Z

@prckent I added some ctest framework and a working example for RMG. I still need to figure out how/where to set the path to the RMG executable (maybe best to just add a cmake variable RMG_BIN analogous to QE_BIN) and I'll need to add some logic so that the RMG tests are added only if RMG is found (something similar to FindQE.cmake).

kgasperich · 2021-09-29T17:18:48Z

@prckent the RMG testing should be working now as long as RMG_BIN is passed to cmake. Should we add an RMG build step to the nightly test scripts?

prckent · 2021-09-29T18:47:38Z

Is it safe to use v4.2.2 of RMG or do we need to use develop?

prckent · 2021-10-15T22:21:22Z

This is now working for me. Will have minor comments only.

prckent

Please can you update the logging so that it only prints "Adding RMG tests" as we do with other test categories?

-- Adding integration tests for QMCPACK
Use --log-level=VERBOSE CMake option for details of which tests will be enabled.
Adding estimator tests for QMCPACK
Adding I/O tests for QMCPACK
Adding performance tests for QMCPACK
Adding tests for rmg -> convert4qmc -> QMCPACK workflow
workdir: /home/pk7/projects/qmc/git_QMCPACK_prckent/build_quick/tests/rmg/rmg-Diamond2-1x1x1-gamma-ccECP-np-1
rmg-Diamond2-1x1x1-gamma-ccECP-np-1
adding RMG tests
Adding system tests for QMCPACK
Python dependencies met. Adding PySCF workflow tests
Adding example tests for QMCPACK
Adding Nexus tests
-- Configuring done
-- Generating done
-- Build files have been written to: /home/pk7/projects/qmc/git_QMCPACK_prckent/build_quick

Not critical but was there a reason to not repurpose an existing pseudopotential file and use your C.qmcpp.xml ?
Add RMG_BIN in top level readme.md

prckent · 2021-10-18T17:55:34Z

src/QMCTools/RMGParser.h

+// This file is distributed under the University of Illinois/NCSA Open Source License.
+// See LICENSE file in top directory for details.
+//
+// Copyright (c) 2016 Jeongnim Kim and QMCPACK developers.


prckent · 2021-10-18T17:55:55Z

src/QMCTools/RMGParser.h

+  RMGParser(int argc, char** argv);
+
+  std::vector<std::string> ECP_names;
+  //std::vector<std::string> tags;


Remove commented lines, including those below

prckent · 2021-10-18T18:56:35Z

Working

pk7@sulfur build_quick]$ ctest -R rmg --output-on-failure
Test project /home/pk7/projects/qmc/git_QMCPACK_prckent/build_quick
      Start   62: converter_test_diamond2_rmg
 1/15 Test   #62: converter_test_diamond2_rmg ............................   Passed    0.39 sec
      Start   63: converter_test_O_rmg
 2/15 Test   #63: converter_test_O_rmg ...................................   Passed    0.36 sec
      Start  228: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-scf
 3/15 Test  #228: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-scf ................   Passed    6.70 sec
      Start  229: LINK_rmg-Diamond2-1x1x1-gamma-ccECP-np-1_h5_Waves
 4/15 Test  #229: LINK_rmg-Diamond2-1x1x1-gamma-ccECP-np-1_h5_Waves ......   Passed    0.05 sec
      Start  230: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-rmg2qmc
 5/15 Test  #230: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-rmg2qmc ............   Passed    0.29 sec
      Start  231: LINK_rmg-Diamond2-1x1x1-gamma-ccECP-np-1_TO_1-16
 6/15 Test  #231: LINK_rmg-Diamond2-1x1x1-gamma-ccECP-np-1_TO_1-16 .......   Passed    0.05 sec
      Start  232: COPY_rmg-Diamond2-1x1x1-gamma-ccECP-np-1_XML_TO_1-16
 7/15 Test  #232: COPY_rmg-Diamond2-1x1x1-gamma-ccECP-np-1_XML_TO_1-16 ...   Passed    0.08 sec
      Start  233: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16
 8/15 Test  #233: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16 ...............   Passed    7.54 sec
      Start  234: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16-kinetic
 9/15 Test  #234: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16-kinetic .......   Passed    0.07 sec
      Start  235: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16-samples
10/15 Test  #235: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16-samples .......   Passed    0.07 sec
      Start  236: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16-localecp
11/15 Test  #236: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16-localecp ......   Passed    0.07 sec
      Start  237: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16-nonlocalecp
12/15 Test  #237: rmg-Diamond2-1x1x1-gamma-ccECP-np-1-1-16-nonlocalecp ...   Passed    0.07 sec
      Start 2034: ntest_nexus_rmg_input
13/15 Test #2034: ntest_nexus_rmg_input ..................................   Passed    0.71 sec
      Start 2043: ntest_nexus_rmg_analyzer
14/15 Test #2043: ntest_nexus_rmg_analyzer ...............................   Passed    0.58 sec
      Start 2051: ntest_nexus_rmg_simulation
15/15 Test #2051: ntest_nexus_rmg_simulation .............................   Passed    0.56 sec

100% tests passed, 0 tests failed out of 15

prckent · 2021-10-18T19:01:42Z

We could discuss the diamond test names as well: Here they are "Diamond2". Elsewhere we have "diamond" and "diamondC". In the performance tests we also have atom and electron counts indicated, which is certainly useful.
Perhaps use diamondC-a2-e8, or is this overkill? We could update the other test names consistently over time (not here).

Edited to add: I suggest to change to just diamondC for now. Otherwise the names are unwieldy.

ye-luo · 2021-10-20T19:20:30Z

CMake/FindRMG.cmake

+# Locate rmg-cpu 
+# Take RMG_BIN as hint for location
+
+find_path(RMG_CPU_DIR rmg-cpu HINTS ${RMG_BIN})


find_program(RMG_CPU_EXE rmg-cpu HINTS ${RMG_BIN}) and use RMG_CPU_EXE in tests are more straight forward.

that does look cleaner; I've changed it

prckent · 2021-11-03T13:29:43Z

@kgasperich What are your plans to update this? I would really like to get it merged and in use in our nightlies. Additional updates can be done in later PRs. The nightly scripts are updated and rmg installed, so these tests should start to run once this is merged.

kgasperich · 2021-11-03T16:13:08Z

@prckent sorry, it fell off my radar after the last meeting, but I've updated it now and brought it up-to-date with develop.
The only issues mentioned above that I haven't addressed yet are the test names and the reuse of pseudopotential files

prckent · 2021-11-03T19:26:18Z

As just discussed at a meeting on this, the updates will be handled in later PRs so that we can get this in use today.

prckent · 2021-11-03T20:19:00Z

Start testing in-house

ye-luo · 2021-11-03T20:29:32Z

Test this please

ye-luo · 2021-11-03T20:36:48Z

I saw binary h5 files are added but we also have ways to generate them. Can we remove those h5 and bake converter test into the workflow test? If we are in favor of merging this as it is. Need to squash to make sure no intermediate files (I saw some in the history) stored in the repo. When these h5 files need updating, they should just be removed.

prckent · 2021-11-03T20:48:42Z

I think this is consistent with our QE tests. We can't test the converter without pregenerated files in the repo.

kgasperich added 4 commits May 3, 2021 08:08

started adding RMG to convert4qmc

fd6e334

Merge branch 'develop' into rmg-convert

2d77681

replace raw pointer with std::unique_ptr for RMGParser in convert4qmc

479f5d9

clang-format

ff1ad72

updated docs for RMG converter

170abe7

change supertwist handling in RMG converter

263c9af

kgasperich added 3 commits August 11, 2021 16:22

added tests for RMG converter

38f3919

removed larger rmg converter test

5687220

added spin-polarized RMG converter test

c18f20e

prckent reviewed Aug 19, 2021

View reviewed changes

Merge branch 'develop' into rmg-convert

c8a7290

kgasperich and others added 2 commits August 25, 2021 11:55

Merge branch 'QMCPACK:develop' into rmg-convert

63cc545

add override for two RMGParser functions

b95cf05

kgasperich added 2 commits September 2, 2021 16:27

Merge branch 'develop' into rmg-convert

eec4c09

added RMG ctest (WIP)

3e0b23b

kgasperich added 2 commits September 29, 2021 12:06

cleaned up RMG tests

d51aebf

shortened rmg vmc test

89d88e0

added rmg-diamond reference vals

c2297b1

prckent self-requested a review October 18, 2021 17:48

prckent requested changes Oct 18, 2021

View reviewed changes

ye-luo reviewed Oct 20, 2021

View reviewed changes

kgasperich added 3 commits November 3, 2021 11:02

minor cleanup

fa579b9

minor cleanup of cmake for RMG

fac5b5a

merge branch 'develop' into rmg-convert

138c5fc

prckent approved these changes Nov 3, 2021

View reviewed changes

prckent added 2 commits November 3, 2021 15:26

Merge branch 'develop' into rmg-convert

aafd0e9

Merge branch 'develop' into rmg-convert

e776074

ye-luo enabled auto-merge (squash) November 3, 2021 20:41

ye-luo merged commit 2c533a4 into QMCPACK:develop Nov 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add real-space RMG converter to convert4qmc #3351

add real-space RMG converter to convert4qmc #3351

kgasperich commented Aug 9, 2021 •

edited

Loading

qmc-robot commented Aug 9, 2021

PDoakORNL commented Aug 11, 2021

kgasperich commented Aug 11, 2021

PDoakORNL commented Aug 11, 2021

kgasperich commented Aug 11, 2021

prckent left a comment

prckent commented Aug 19, 2021

prckent commented Aug 20, 2021

kgasperich commented Aug 25, 2021 •

edited

Loading

prckent commented Sep 2, 2021

kgasperich commented Sep 28, 2021

kgasperich commented Sep 29, 2021

prckent commented Sep 29, 2021

prckent commented Oct 15, 2021

prckent left a comment •

edited

Loading

prckent Oct 18, 2021

kgasperich Nov 3, 2021

prckent Oct 18, 2021

kgasperich Nov 3, 2021

prckent commented Oct 18, 2021

prckent commented Oct 18, 2021 •

edited

Loading

ye-luo Oct 20, 2021

kgasperich Nov 3, 2021

prckent commented Nov 3, 2021 •

edited

Loading

kgasperich commented Nov 3, 2021 •

edited

Loading

prckent commented Nov 3, 2021

prckent commented Nov 3, 2021

ye-luo commented Nov 3, 2021

ye-luo commented Nov 3, 2021 •

edited

Loading

prckent commented Nov 3, 2021

add real-space RMG converter to convert4qmc #3351

add real-space RMG converter to convert4qmc #3351

Conversation

kgasperich commented Aug 9, 2021 • edited Loading

Proposed changes

What type(s) of changes does this code introduce?

Does this introduce a breaking change?

What systems has this change been tested on?

Checklist

qmc-robot commented Aug 9, 2021

PDoakORNL commented Aug 11, 2021

kgasperich commented Aug 11, 2021

PDoakORNL commented Aug 11, 2021

kgasperich commented Aug 11, 2021

prckent left a comment

Choose a reason for hiding this comment

prckent commented Aug 19, 2021

prckent commented Aug 20, 2021

kgasperich commented Aug 25, 2021 • edited Loading

prckent commented Sep 2, 2021

kgasperich commented Sep 28, 2021

kgasperich commented Sep 29, 2021

prckent commented Sep 29, 2021

prckent commented Oct 15, 2021

prckent left a comment • edited Loading

Choose a reason for hiding this comment

prckent Oct 18, 2021

Choose a reason for hiding this comment

kgasperich Nov 3, 2021

Choose a reason for hiding this comment

prckent Oct 18, 2021

Choose a reason for hiding this comment

kgasperich Nov 3, 2021

Choose a reason for hiding this comment

prckent commented Oct 18, 2021

prckent commented Oct 18, 2021 • edited Loading

ye-luo Oct 20, 2021

Choose a reason for hiding this comment

kgasperich Nov 3, 2021

Choose a reason for hiding this comment

prckent commented Nov 3, 2021 • edited Loading

kgasperich commented Nov 3, 2021 • edited Loading

prckent commented Nov 3, 2021

prckent commented Nov 3, 2021

ye-luo commented Nov 3, 2021

ye-luo commented Nov 3, 2021 • edited Loading

prckent commented Nov 3, 2021

kgasperich commented Aug 9, 2021 •

edited

Loading

kgasperich commented Aug 25, 2021 •

edited

Loading

prckent left a comment •

edited

Loading

prckent commented Oct 18, 2021 •

edited

Loading

prckent commented Nov 3, 2021 •

edited

Loading

kgasperich commented Nov 3, 2021 •

edited

Loading

ye-luo commented Nov 3, 2021 •

edited

Loading