Safer distance table implementation #2177

ye-luo · 2020-01-01T05:35:06Z

This PR focuses on ParticleSet and DistanceTableData implementation.
Changes are:

Mark most DistanceTableData members with protected and public. All the members belonging to SoA are converged.
Remove ParticleSet::setActive. setActive prepare old distances and displacements before a particle is moved. This is part of a state machine and only safe to use in certain scenarios because setActive may overwrite extra memory. After examining its use, its role can be moved inside the makeMove function. When a move is proposed, making sure the old value is updated in the right place. If a DT consumer needs the old value before the call to propose a move, it should request full table availability via addTable.
ParticleSet::makeMoveXXX functions support a maybe_accept flag. If the caller knows the fact that no move will be accepted. Setting maybe_accept = false can save a lot of computation. For example in NLPP.
Make ParticleSet::acceptMove safe. The old acceptMove was safe only if move of all the particles are proposed once and in order. This is a rather unsafe assumption. In this PR, acceptMove are made always maintaining up-to-date distance tables. Only in specific cases, forward = true optimization can be requested for example during p-by-p random walking. In the NLPP force evaluation, update() full table is no more needed after an acceptMove.

Overall, less safe codepath for optimization are more tightly controlled. It allows to write working but non-optimal code easier because state machines are less exposed.

Since the base class DistanceTableData is more of an abstraction and actual data are moved to derived class. DistanceTable is a more appropriate name. I will do renaming in a separate PR.

markdewing · 2020-01-02T18:54:37Z

The name 'forward' for the flag seems like it could be more descriptive. For me it invokes images of mail forwarding or C++ move operations.

Thinking through the flag, what it does, and when it is used:
It is used when an algorithm moves through the particle indices in a particular sequence. This enables an optimization to reduce the amount of work to be done where the table only needs to be partially updated at each step.

Perhaps 'partial_update' or 'partial_table_update'?

PDoakORNL

Overall change is a step forward. There are a few typos to fix and some missed opportunities to modernize/clarify naming. Some of these could be address in a future PR.

src/OhmmsPETE/TinyVector.h

src/Particle/AsymmetricDistanceTableData.h

PDoakORNL · 2020-01-02T20:42:06Z

src/Particle/DistanceTableData.h

+  using IndexType    = QMCTraits::IndexType;
+  using RealType     = QMCTraits::RealType;
+  using PosType      = QMCTraits::PosType;
+  using DistRowType  = Vector<RealType, aligned_allocator<RealType>>;


Type does not seem needed, DistRow with the type formatting sufficient.

Removed Type suffix.

src/Particle/DistanceTableData.h

PDoakORNL · 2020-01-02T21:09:31Z

src/Particle/DistanceTableData.h


  /** Displacements[N_targets]x[3][N_sources]
-   *  Note: This is a memory view using the memory space allocated in memoryPool
+   *  Note: Derived classes decide if it is a memory view or the actaully storage


PDoakORNL · 2020-01-02T21:24:12Z

src/Particle/SoaDistanceTableAA.h

  int Ntargets_padded;

+  ///actual memory for Displacements
+  aligned_vector<RealType> memoryPool_displs_;


strangely named method or variable? memory_pool_displs_

PDoakORNL · 2020-01-02T21:52:12Z

src/Particle/DistanceTableData.h

+   * @param P the target particle set
+   * @param rnew proposed new position
+   * @param iat the particle to be moved
+   * @param prepare_old if true, prepare old distances and displacements for using getOldDists and getOldDispls functions later.


hopefully temporary

The old values should be temporarily used only.

src/QMCHamiltonians/LocalECPotential.h

PDoakORNL · 2020-01-02T22:39:15Z

src/Particle/DistanceTableData.h

+
+  /** return the temporary distances when a move is proposed
+   */
+  const DistRowType& getTemporalDists() const { return Temp_r; }


getTempDists() is clearer to me. Temporal is not a common synonym of temporary.

All changed.

src/Sandbox/diff_distancetables.cpp

ye-luo · 2020-01-03T05:09:02Z

@markdewing I changed DTD forward to partial_update and ParticleSet forward to partial_table_update as you suggested.

prckent · 2020-01-03T16:03:22Z

This is moving in a good direction.

It is clear that the design is still not right - it is too complex still, although this is definitely an improvement. Perhaps indicating where state is involved in the function names might help.

ye-luo added 23 commits November 14, 2019 22:32

DistanceTable change draft

7c9329e

Merge branch 'develop' into safer-DT

b11de6f

Mark more protected data in DistanceTableData.h

c0e97d8

Code cleaning.

487472a

Reduce pointer exposure.

c201af2

More protected data.

67a75ee

Move memory to the derived classes.

954f87c

Fix sandbox

756da24

Plan to simplify setActive.

11ba4bb

Remove setActive fully. Reduce state machines.

cbd85ec

Add AA DT tests.

531e4e0

Merge branch 'safer-DT' into add-DT-AA-tests

a61efd9

Expand AA DT tests.

64d26de

Refine timers.

f71de6b

Attempt recompute.

b54d155

Make a try.

2c5d7b8

Add forward mode in acceptMove

84ec4f7

Introduce back need_full_table_ in DT.

e0b9a51

Code format.

5317838

Reduce compiler warning.

b33ef46

Code formating.

55d1e44

More comments.

8d7c215

Avoid accessing whole table.

89580ca

ye-luo mentioned this pull request Jan 1, 2020

Fix NLPP derivative in SoA #2179

Merged

Add forward in batched interfaces.

84b280b

ye-luo force-pushed the safer-DT branch from 3e8f893 to 84b280b Compare January 1, 2020 15:00

ye-luo requested a review from rcclay January 2, 2020 01:54

ye-luo mentioned this pull request Jan 2, 2020

Make delayed update active only by request #2180

Merged

PDoakORNL requested changes Jan 2, 2020

View reviewed changes

ye-luo added 9 commits January 3, 2020 10:04

Rename opposite to inverse.

99275b5

Remove Type in type names.

fc06511

Temporal to Temp for shorter names.

7fac7cd

Formatting.

0f5637b

Merge remote-tracking branch 'origin/develop' into safer-DT

8400076

Change forward to partial_update and partial_table_update.

952feab

Rename variables.

2ab956b

Change ture to true.

520875f

Follow naming rules.

939bbfa

PDoakORNL approved these changes Jan 3, 2020

View reviewed changes

prckent merged commit 9f7ad03 into QMCPACK:develop Jan 3, 2020

ye-luo deleted the safer-DT branch January 3, 2020 23:23

mcbennet mentioned this pull request Apr 9, 2020

Final DMC energy varies with equilibration strategy #2330

Closed

ye-luo mentioned this pull request Apr 12, 2020

Fix last bit of #2330 #2377

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Safer distance table implementation #2177

Safer distance table implementation #2177

ye-luo commented Jan 1, 2020

markdewing commented Jan 2, 2020

PDoakORNL left a comment

PDoakORNL Jan 2, 2020

ye-luo Jan 3, 2020

PDoakORNL Jan 2, 2020

ye-luo Jan 3, 2020

PDoakORNL Jan 2, 2020

ye-luo Jan 3, 2020

PDoakORNL Jan 2, 2020

ye-luo Jan 3, 2020

PDoakORNL Jan 2, 2020

ye-luo Jan 3, 2020

ye-luo commented Jan 3, 2020

prckent commented Jan 3, 2020

Safer distance table implementation #2177

Safer distance table implementation #2177

Conversation

ye-luo commented Jan 1, 2020

markdewing commented Jan 2, 2020

PDoakORNL left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ye-luo commented Jan 3, 2020

prckent commented Jan 3, 2020