make two-body Jastrow work in 2D #4289

Paul-St-Young · 2022-10-21T12:39:59Z

Proposed changes

Follow up on #3868, enable Jastrow in 2D.
The WavefunctionTester was modified to run in 2D (in addition to 3D) to test these changes.

What type(s) of changes does this code introduce?

New feature

Does this introduce a breaking change?

No

What systems has this change been tested on?

workstation

Checklist

Update the following with a yes where the items apply. If you're unsure about any of them, don't hesitate to ask. This is
simply a reminder of what we are going to look for before merging your code.

Yes. This PR is up to date with current the current state of 'develop'
No. Code added or changed in the PR has been clang-formatted
Yes. This PR adds tests to cover any new code, or to catch a bug that is being fixed

ye-luo

Could you please add a unit test. WaveFunctionTest is not the optimal way of making a test because it cannot provide enough safeguard. If the value, gradients are set zero, WaveFunctionTest cannot catch it.

prckent

Thanks Paul. Echoing Ye's comments, please stay away from wftester for formal testing. We have established routes for that via unit testing (by far preferred), followed by integration testing.

Background: wftester is a historical clearly not looked-after part of the code that we should not be increasing dependencies on. Other than this PR from you and a PR from me, there are no existing uses or tests. Perhaps we'll rewrite it, remove it, or trim it. Some days I have been tempted to just delete it, since it would be faster to start afresh with clean code. If there is something that is unclear how to write from a unit test please ask.

Paul-St-Young · 2022-10-21T15:59:38Z

@ye-luo @prckent The WavefunctionTester is by far the easiest route to test particle and parameter derivatives of a complicated wavefunction. When I doubt the correctness of a run, it's easy replace the <qmc method="vmc/dmc"> section with <qmc method="wftest"> for a quick test. I know that @rcclay and my self have made many uses of this feature, for example when debugging backflow. @markdewing also made the FiniteDifference class fairly clean.

WavefunctionTester is also my go-to when developing a new wavefunction component, for example when resurrecting 2D electron gas orbitals #4084.
Since we have no other facility for generally checking the correctness of analytical derivatives (either for single-particle orbital, or for the many-body wavefunction), I don't see why we should remove this feature.
Having to re-code finite-difference for every unit test is not productive IMO.

I am happy to put in a simple regression unit test, but I would like to keep the wftest to check for correctness.

markdewing · 2022-10-21T16:09:29Z

src/QMCWaveFunctions/Jastrow/RadialJastrowBuilder.cpp

@@ -223,7 +224,7 @@ std::unique_ptr<WaveFunctionComponent> RadialJastrowBuilder::createJ2(xmlNodePtr
 #if OHMMS_DIM == 1
        RealType dim_factor = 1.0 / (OHMMS_DIM + 1);
 #else
-        RealType dim_factor = (ia == ib) ? 1.0 / (OHMMS_DIM + 1) : 1.0 / (OHMMS_DIM - 1);
+        RealType dim_factor = (ia == ib) ? 0.5 / (ndim - 1) : 1.0 / (ndim - 1);


For 2 dimensions, the factor for the case ia==ib changes from 1/3 to 1/2. Is that change intended?

Yes, this is an intentional change.
The same-spin cusp should always be half of that of the opposite-spin cusp.
Did I miss something?

The previous code was added in #2806 , and it references equation 9 of https://arxiv.org/abs/2003.06506 (and footnote 45 for the 1-D case). According to that formula, it looks like the same-spin cusp is half of the opposite spin cusp for d=3, but not for d=2.

Yes, you are right. I must have made a mistake in my own derivation.
Thanks for catching this!

prckent · 2022-10-21T16:40:06Z

Fair point on not recoding the finite derivatives.

prckent · 2022-10-21T18:05:16Z

@Paul-St-Young You use <parameter name="ratio">deriv</parameter> or perhaps <parameter name="basic">yes</parameter>?

prckent · 2022-10-21T18:20:34Z

src/QMCDrivers/WaveFunctionTester.cpp

@@ -69,6 +70,12 @@ WaveFunctionTester::WaveFunctionTester(const ProjectData& project_data,

  deltaR.resize(w.getTotalNum());
  makeGaussRandom(deltaR);


Any reason not to make makeGaussRandom dimension aware and avoid the changes and logging on later lines?

The reason is that I wanted the change to be localized in the WaveFunctionTester.

Eventually, what you suggested is nicer. Although, I would have to touch a lot of files to make this API change.
I wonder if it's worth making another PR for that.

Hi Paul - In that case I agree that making another PR would be a good way of splitting this up

prckent

Just noticed: the tests are not being run in CI. Please can you rename them with the prefix deterministic- so that "ctest -L deterministic" will pick them up. I think the name is sufficient. Thanks.

edit: for consistency with the existing heg tests, how about "deterministic-heg2d-...". That avoids introducing a new abbreviation that people will have to learn the meaning of.

prckent · 2022-10-21T19:17:23Z

Test this please

ye-luo · 2022-10-22T00:25:30Z

@ye-luo @prckent The WavefunctionTester is by far the easiest route to test particle and parameter derivatives of a complicated wavefunction. When I doubt the correctness of a run, it's easy replace the <qmc method="vmc/dmc"> section with <qmc method="wftest"> for a quick test. I know that @rcclay and my self have made many uses of this feature, for example when debugging backflow. @markdewing also made the FiniteDifference class fairly clean.

WavefunctionTester is also my go-to when developing a new wavefunction component, for example when resurrecting 2D electron gas orbitals #4084. Since we have no other facility for generally checking the correctness of analytical derivatives (either for single-particle orbital, or for the many-body wavefunction), I don't see why we should remove this feature. Having to re-code finite-difference for every unit test is not productive IMO.

I am happy to put in a simple regression unit test, but I would like to keep the wftest to check for correctness.

You can use wftest for your development but the final test being added needs to be a unit test. wftest does random sampling which is not desired in unit tests. I don't see where you get the impression of the need of finite-difference in unit test. Unit test should check against stored values. See https://github.com/QMCPACK/qmcpack/blob/develop/src/QMCWaveFunctions/tests/test_polynomial_eeI_jastrow.cpp

Paul-St-Young · 2022-10-24T14:08:33Z

@Paul-St-Young You use <parameter name="ratio">deriv</parameter> or perhaps <parameter name="basic">yes</parameter>?

Yes, I use only two features of the WavefunctionTester: the "basic" test for particle derivatives and move ratios and the "derivative" test for parameter derivatives.

Paul-St-Young · 2022-10-24T14:09:10Z

Just noticed: the tests are not being run in CI. Please can you rename them with the prefix deterministic- so that "ctest -L deterministic" will pick them up. I think the name is sufficient. Thanks.

edit: for consistency with the existing heg tests, how about "deterministic-heg2d-...". That avoids introducing a new abbreviation that people will have to learn the meaning of.

Thanks fo advice! I have changed the name of the test.

prckent · 2022-10-24T14:11:52Z

Test this please

ye-luo · 2022-10-27T18:51:34Z

@Paul-St-Young It seems failing with address sanitizer https://github.com/QMCPACK/qmcpack/actions/runs/3315476545/jobs/5476115646

prckent · 2022-10-31T21:26:30Z

Paul - Are you able to reproduce any of these problems locally? e.g. Our gcc9 " no mpi and no omp " build is crashing with a segfault.

Paul-St-Young · 2022-11-01T02:20:21Z

Yes, I was able to reproduce the crash.
The problem is only in the real build.

The input has 2 up electrons in 2 PW orbitals.
The 2 complex PW orbitals 0 and e^ikr are fine, but there are 3 real PW orbitals: 0, sin(kr), and cos(kr).
The extra memory was not allocated in DiracDeterminant.

How may I resize DiracDeterminant from within an SPOSet builder?

ye-luo · 2022-11-01T02:28:03Z

DiracDeterminant query the size from SPOSet. Make sure it is correct at the first place.

Paul-St-Young · 2022-11-01T13:27:16Z

DiracDeterminant query the size from SPOSet. Make sure it is correct at the first place.

@ye-luo I'm confused by this.
In DiractDeterminant.cpp, resize is called with NumPtcls in both arguments.
If I make the following change, then my error goes way

-  resize(NumPtcls, NumPtcls);
+  const int norb = Phi->getOrbitalSetSize();
+  resize(NumPtcls, norb);

However, it breaks other tests.
Where does DiracDeterminant query the size from SPOSet to resize the determinant in the current version of the code?

prckent · 2022-11-01T13:36:26Z

I agree with Ye on this one. We should create diracdeterminants with the correct size to begin with. Cleaner and clearer. The old trope of changing the size of things after initial construction hurts the accessibility of the code and is not good for maintenance.

PDoakORNL · 2022-11-01T13:36:45Z

I feel like this is in the end a multiple sources of truth issue. The fix is not resizing a determinant on the fly.

Paul-St-Young · 2022-11-01T14:20:21Z

@prckent @PDoakORNL Yes, I agree with your view.
That that resize in the snippet above is from the constructor of DiracDeterminant, so it's not changing the size of things after initial construction.
The question is whether to initialize the determinants to (NumPtcls, NumPtcls) or (NumPtcls, norb).

ye-luo · 2022-11-01T14:39:38Z

The determinant matrices must be strictly square. (NumPtcls, NumPtcls). In the FreeOrbital input, you specified size 2 in the input. So the object should work for exactly or up to 2 orbitals. Based on what you mentioned that FreeOrbital creates 3 orbitals in the real case, then a truncation must be done at the construction of FreeOrbital object.

add some checks in FreeOrbitalBuilder

Paul-St-Young · 2022-11-01T20:39:36Z

Thanks @ye-luo! Your explanation was very helpful.

I think the fundamental problem was that I set up a system that truly needs a complex wavefunction.
The 2 up 2 dn system at Gamma should have a complex single determinant wavefunction, because there is no -k PW to pair up with the +k PW to make sin and cos.

I added some checks in FreeOrbitalBuilder to abort in a situation as described above.

I also edited the example to contain 5 up 5 dn (closed-shell in 2D) and now things run through.

ye-luo

Minor changes requested.

ye-luo · 2022-11-01T21:31:12Z

src/QMCDrivers/WaveFunctionTester.cpp

-  FiniteDifference fd(FiniteDifference::FiniteDiff_LowOrder);
-  //FiniteDifference fd(FiniteDifference::FiniteDiff_Richardson);
+  FiniteDifference fd(ndim, FiniteDifference::FiniteDiff_LowOrder);
+  //FiniteDifference fd(ndim, FiniteDifference::FiniteDiff_Richardson);


Remove this commented line

ye-luo · 2022-11-01T21:33:12Z

src/QMCDrivers/WaveFunctionTester.cpp

  //RealType delta = 1.0;
-  //FiniteDifference fd(FiniteDifference::FiniteDiff_Richardson);
+  //FiniteDifference fd(ndim, FiniteDifference::FiniteDiff_Richardson);


Remove commented lines.

ye-luo · 2022-11-01T21:34:23Z

src/QMCWaveFunctions/ElectronGas/FreeOrbitalBuilder.cpp

+    std::ostringstream msg;
+    msg << "norb = " << norb << " npw = " << npw;
+    msg << " cannot be ran in real PWs (sin, cos)" << std::endl;
+    msg << "either use complex build or change the size of SPO set" << std::endl;


Do you have recommendation to print about how to "change the size of SPO set"?

ye-luo · 2022-11-01T21:36:03Z

src/QMCWaveFunctions/ElectronGas/FreeOrbitalBuilder.cpp

@@ -31,6 +34,14 @@ std::unique_ptr<SPOSet> FreeOrbitalBuilder::createSPOSetFromXML(xmlNodePtr cur)
  app_log() << "twist cartesian = " << tvec << std::endl;
 #else
  npw = std::ceil((norb + 1.0) / 2);


Remove int npw in line 15 and put
int npw = here and line 31
Also please do a clang-formatting.

ye-luo · 2022-11-01T21:38:18Z

src/QMCWaveFunctions/Jastrow/RadialJastrowBuilder.cpp

        RealType qq       = species(chargeInd, ia) * species(chargeInd, ib);
        RealType red_mass = species(massInd, ia) * species(massInd, ib) / (species(massInd, ia) + species(massInd, ib));
 #if OHMMS_DIM == 1
        RealType dim_factor = 1.0 / (OHMMS_DIM + 1);
 #else
-        RealType dim_factor = (ia == ib) ? 1.0 / (OHMMS_DIM + 1) : 1.0 / (OHMMS_DIM - 1);
+        RealType dim_factor = (ia == ib) ? 1.0 / (ndim + 1) : 1.0 / (ndim - 1);


Remove #if OHMMS_DIM == 1

ye-luo

Please add random seeds in both determinsitic test xml.

Paul-St-Young · 2022-11-02T01:16:08Z

@ye-luo I think I addressed all of your comments. Please see if this is satisfactory now.
The macos error seems to be an h5py install error, so not related to this PR

ye-luo · 2022-11-02T01:23:46Z

Test this please

prckent · 2022-11-02T14:57:39Z

I reran the failed macos test and it passed. Seems to have failed due to a transient infrastructure problem, nothing to do with this PR.

ye-luo

LGTM
@prckent do you still have remaining concerns?

prckent · 2022-11-02T15:00:55Z

Thanks @Paul-St-Young !

make two-body Jastrow work in 2D

e6c548e

Paul-St-Young requested a review from ye-luo October 21, 2022 12:39

Paul-St-Young self-assigned this Oct 21, 2022

ye-luo requested changes Oct 21, 2022

View reviewed changes

prckent self-requested a review October 21, 2022 14:28

prckent requested changes Oct 21, 2022

View reviewed changes

markdewing reviewed Oct 21, 2022

View reviewed changes

prckent reviewed Oct 21, 2022

View reviewed changes

prckent requested changes Oct 21, 2022

View reviewed changes

Paul-St-Young added 4 commits October 24, 2022 09:42

restore same-spin cusp condition

efe4288

disable jastrow hessian in 2d

dc51ca5

add unit test for 2D jastrow

08a1768

rename heg2d tests

254f5f6

Paul-St-Young added 3 commits October 24, 2022 10:29

fix real and mixed builds

dac5877

move 2d jastrow test

1479219

use gamma in wftest

f6996e9

run real PWs in closed shell system

aaf9739

add some checks in FreeOrbitalBuilder

Merge branch 'develop' into develop

af6432d

ye-luo reviewed Nov 1, 2022

View reviewed changes

ye-luo requested changes Nov 1, 2022

View reviewed changes

Paul-St-Young added 6 commits November 1, 2022 19:34

add random seed to deterministic tests

644b5bd

remove commented lines

7be1094

recomment size

375c421

incorporate 1D cusp

8d359e4

create npw at first use

21d4895

clang format

80419a1

ye-luo approved these changes Nov 2, 2022

View reviewed changes

ye-luo enabled auto-merge November 2, 2022 14:59

prckent approved these changes Nov 2, 2022

View reviewed changes

ye-luo merged commit e6169b5 into QMCPACK:develop Nov 2, 2022

Paul-St-Young mentioned this pull request Nov 2, 2022

How to resurrect true 2D capability? #3868

Open

5 tasks

		@@ -69,6 +70,12 @@ WaveFunctionTester::WaveFunctionTester(const ProjectData& project_data,

		deltaR.resize(w.getTotalNum());
		makeGaussRandom(deltaR);

make two-body Jastrow work in 2D #4289

make two-body Jastrow work in 2D #4289

Conversation

Paul-St-Young commented Oct 21, 2022

Proposed changes

What type(s) of changes does this code introduce?

Does this introduce a breaking change?

What systems has this change been tested on?

Checklist

ye-luo left a comment

Choose a reason for hiding this comment

prckent left a comment • edited Loading

Choose a reason for hiding this comment

Paul-St-Young commented Oct 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prckent commented Oct 21, 2022

prckent commented Oct 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prckent left a comment • edited Loading

Choose a reason for hiding this comment

prckent commented Oct 21, 2022

ye-luo commented Oct 22, 2022

Paul-St-Young commented Oct 24, 2022

Paul-St-Young commented Oct 24, 2022

prckent commented Oct 24, 2022

ye-luo commented Oct 27, 2022

prckent commented Oct 31, 2022

Paul-St-Young commented Nov 1, 2022

ye-luo commented Nov 1, 2022

Paul-St-Young commented Nov 1, 2022

prckent commented Nov 1, 2022

PDoakORNL commented Nov 1, 2022

Paul-St-Young commented Nov 1, 2022

ye-luo commented Nov 1, 2022

Paul-St-Young commented Nov 1, 2022

ye-luo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ye-luo left a comment

Choose a reason for hiding this comment

Paul-St-Young commented Nov 2, 2022

ye-luo commented Nov 2, 2022

prckent commented Nov 2, 2022

ye-luo left a comment

Choose a reason for hiding this comment

prckent commented Nov 2, 2022

prckent left a comment •

edited

Loading

Paul-St-Young commented Oct 21, 2022 •

edited

Loading

prckent commented Oct 21, 2022 •

edited

Loading

prckent left a comment •

edited

Loading