Add primitive support for constructor custom pushforward functions #909

parth-07 · 2024-05-26T09:59:04Z

This commit adds primitive support for custom pushforward functions for
constructors. Custom constructor pushforward function support will enable the
below features:

Class differentiation support for classes whose constructor Clad
cannot automatically differentiate. Now, We can enable differentiation of
entire C++ standard library by providing custom derivatives.
Remove the restriction of default-constructible for class types. This
was a troublesome restriction. Now, the only restriction for class
types is to have a sensible copy-constructor. That is, copy
constructor should copy the class members and after copy-construction,
both the objects should be equivalent, mathematically speaking.

Constructor pushforward functions differ from ordinary pushforward
functions in two important ways:

Constructor pushforward functions initialize the primal class object
and the corresponding derivative object. Ordinary member function
pushforwards takes an already-existing primal class object and the
corresponding derivative object as inputs.
Constructor pushforward functions return a value even though
constructor do not return anything. Constructor pushforward functions
return initialized primal object and the derivative object. These are
then used to initialize primal object and the derivative in the
derivative function code.

How to write custom constructor pushforward functions

Let's see how to write custom pushforward function for a
constructor:

Custom constructor pushforwards must have the name constructor_pushforward
Custom constructor pushforwards must be defined in
::clad::custom_derivatives::class_functions namespace.
The parameters of the custom constructor pushforward must be:
{::clad::Identify<Class>, original params..., derivative params...}.

'original parameters...' and 'derivative parameters...' is same as what
we have for other pushforward functions. We will soon see why do we need
::clad::Identify<T> for constructor custom pushforwards.

Let's see a basic example of how to write custom constructor
pushforward.

class Coordinates {
  Coordinates(double px, double py, double pz) :
    x(px), y(py), z(pz) {}

  public:
  double x, y, z;
}

namespace clad {
namespace custom_derivatives {
namespace class_functions {
// custom constructor pushforward function
clad::ValueAndPushforward<Coordinates, Coordinates>
constructor_pushforward(clad::Identify<Coordinates>, double x, double y,
                        double z, double d_x, double d_y, double d_z) {
  return {Coordinates(x, y, z), Coordinates(d_x, d_y, d_z) };
}
} // namespace class_functions
} // namespace custom_derivatives
} // namespace clad

// custom constructor pushforward is used as follows:
// primal code
Constructor c(u, v, w);

// derivative code
clad::ValueAndPushforward<Coordinates, Coordinates> _t0 =
  constructor_pushforward(clad::Identify<Coordinates>, u, v, w, _d_u,
    _d_v, _d_w);
Coordinates _d_c = _t0.pushforward;
Coordinates c = _t0.value;

Now, let's see a bit advanced example based on std::vector constructor.

namespace clad {
namespace custom_derivatives {
namespace class_functions {
// Custom pushforward for: vector(size_t n, const typename
::std::vector<T>::allocator_type alloc)
template <typename T>
clad::ValueAndPushforward<::std::vector<T>, ::std::vector<T>>
constructor_pushforward(
    Identify<::std::vector<T>>, size_t n,
    const typename ::std::vector<T>::allocator_type alloc, size_t d_n,
    const typename ::std::vector<T>::allocator_type d_alloc) {
  ::std::vector<T> v(n, alloc);
  ::std::vector<T> d_v(n, 0, alloc);
  return {v, d_v};
}

// Custom pushfoward for: vector(size_t n, T val, const typename
::std::vector<T>::allocator_type alloc)
template <typename T>
clad::ValueAndPushforward<::std::vector<T>, ::std::vector<T>>
constructor_pushforward(
    Identify<::std::vector<T>>, size_t n, T val,
    const typename ::std::vector<T>::allocator_type alloc, size_t d_n, T d_val,
    const typename ::std::vector<T>::allocator_type d_alloc) {
  ::std::vector<T> v(n, val, alloc);
  ::std::vector<T> d_v(n, d_val, alloc);
  return {v, d_v};
}
} // namespace class_functions
} // namespace custom_derivatives
} // namespace clad

// The custom constructor pushforwards is used as follows:

// Primal code:
std::vector<double> v(10, u);

// Derivative code:
clad::ValueAndPushforward<std::vector<double>, std::vector<double>> _t0 =
  clad::custom_derivatives::class_functions::constructor_pushforward(
    clad::Identify<std::vector<double> >(), 10, u, allocator_type(),
    0, _d_u, allocator_type());
std::vector<double> d_v = _t0.pushforward;
std::vector<double> v = _t0.value;

Why `clad::Identify<T>`?

So, why do we need clad::Identify?

For a constructor that takes two parameters of types size_t and double,
the custom pushforward will have the following signature if we do not
include ::clad::Identify<T>:

clad::ValueAndPushforward<Class, Class> constructor_pushforward(size_t n,
  double val, size_t d_n, double d_val);

Now, the question is: How to distinguish custom constructor pushforwards
for different classes?

MyClassA a(3, 5.0);
MyClassB b(7, 9.0);

There is no way for overload resolution selector to distinguish
constructor_pushforward for classes MyClassA and MyClassB.

clad::Identify<T> is used to identify the class for which
custom constructor pushforward is defined.

Please note that we cannot use the same strategy which we use for custom
member function pushforwards because member function pushforwards
always have parameters of the class type which are used for identifying
the class.

We also cannot simply ask users to define the pushforwards inside the
declaration context of the class because it may not always be feasible
to modify the source code of external libraries.

If anyone has better ideas for interface for defining custom constructor
pushforward function, then definitely share them!

github-actions · 2024-05-26T10:02:18Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-05-26T14:13:32Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-05-26T14:19:03Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-05-26T19:24:40Z

clang-tidy review says "All clean, LGTM! 👍"

codecov · 2024-05-26T19:28:24Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.94%. Comparing base (22b2590) to head (86128ef).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #909      +/-   ##
==========================================
+ Coverage   93.92%   93.94%   +0.02%     
==========================================
  Files          55       55              
  Lines        7932     7965      +33     
==========================================
+ Hits         7450     7483      +33     
  Misses        482      482

Files	Coverage Δ
...clude/clad/Differentiator/BaseForwardModeVisitor.h	`100.00% <ø> (ø)`
include/clad/Differentiator/BuiltinDerivatives.h	`100.00% <ø> (ø)`
include/clad/Differentiator/VisitorBase.h	`100.00% <ø> (ø)`
lib/Differentiator/BaseForwardModeVisitor.cpp	`98.98% <100.00%> (+0.02%)`	⬆️
lib/Differentiator/CladUtils.cpp	`93.64% <100.00%> (+0.03%)`	⬆️
lib/Differentiator/VisitorBase.cpp	`96.70% <100.00%> (+0.05%)`	⬆️

Files	Coverage Δ
...clude/clad/Differentiator/BaseForwardModeVisitor.h	`100.00% <ø> (ø)`
include/clad/Differentiator/BuiltinDerivatives.h	`100.00% <ø> (ø)`
include/clad/Differentiator/VisitorBase.h	`100.00% <ø> (ø)`
lib/Differentiator/BaseForwardModeVisitor.cpp	`98.98% <100.00%> (+0.02%)`	⬆️
lib/Differentiator/CladUtils.cpp	`93.64% <100.00%> (+0.03%)`	⬆️
lib/Differentiator/VisitorBase.cpp	`96.70% <100.00%> (+0.05%)`	⬆️

github-actions · 2024-06-01T19:46:53Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-06-01T20:05:34Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-06-01T20:19:21Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-06-01T20:26:51Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-06-01T21:10:47Z

clang-tidy review says "All clean, LGTM! 👍"

lib/Differentiator/BaseForwardModeVisitor.cpp

github-actions

clang-tidy made some suggestions

github-actions · 2024-06-02T16:25:08Z

lib/Differentiator/BaseForwardModeVisitor.cpp

+      GetPushForwardFunctionSuffix();
+  Expr* pushforwardCall = m_Builder.BuildCallToCustomDerivativeOrNumericalDiff(
+      customPushforwardName, customPushforwardArgs, getCurrentScope(),
+      const_cast<DeclContext*>(CE->getConstructor()->getDeclContext()));


warning: do not use const_cast [cppcoreguidelines-pro-type-const-cast]

const_cast<DeclContext*>(CE->getConstructor()->getDeclContext())); ^

Don't we need to operate on the cloned CE?

We do not need to clone CXXConstructExpr* CE here because we are not (directly) using the CXXConstructExpr in the derivative code. We are using CE here to get the class (declaration context) of the constructor.

github-actions · 2024-06-02T16:25:08Z

lib/Differentiator/BaseForwardModeVisitor.cpp

+      customPushforwardName, customPushforwardArgs, getCurrentScope(),
+      const_cast<DeclContext*>(CE->getConstructor()->getDeclContext()));
+  if (pushforwardCall) {
+    auto valueAndPushforwardE = StoreAndRef(pushforwardCall);


warning: 'auto valueAndPushforwardE' can be declared as 'auto *valueAndPushforwardE' [llvm-qualified-auto]

Suggested change

auto valueAndPushforwardE = StoreAndRef(pushforwardCall);

auto *valueAndPushforwardE = StoreAndRef(pushforwardCall);

lib/Differentiator/VisitorBase.cpp

vgvassilev

@PetroZarytskyi, can you take a look?

include/clad/Differentiator/BuiltinDerivatives.h

vgvassilev · 2024-06-03T17:03:43Z

include/clad/Differentiator/STLBuiltins.h

+template <typename T>
+clad::ValueAndPushforward<::std::vector<T>, ::std::vector<T>>
+constructor_pushforward(
+    Identify<::std::vector<T>>, typename ::std::vector<T>::size_type n,


The tag dispatcher generally goes last in the function signature.

I am not too sure about this. If we put the tag dispatcher (ConstructorPushforwardTag) at end, then it might be difficult to find constructor pushforwards for a class which has 5 constructors with different number of parameters. I feel that it is easy to spot (dare I say,Identify :) ) constructor pushforwards for a particular class by seeing the first parameter of constructor pushforwards as compared to the last parameter. Please share your thoughts on this.

Okay, I do not have a strong preference here. I believe being a first argument will work out, too because what we do here is slightly different than the canonical tag dispatch practice in C++.

lib/Differentiator/BaseForwardModeVisitor.cpp

vgvassilev · 2024-06-03T17:05:36Z

lib/Differentiator/BaseForwardModeVisitor.cpp

+      GetPushForwardFunctionSuffix();
+  Expr* pushforwardCall = m_Builder.BuildCallToCustomDerivativeOrNumericalDiff(
+      customPushforwardName, customPushforwardArgs, getCurrentScope(),
+      const_cast<DeclContext*>(CE->getConstructor()->getDeclContext()));


Don't we need to operate on the cloned CE?

lib/Differentiator/VisitorBase.cpp

github-actions

clang-tidy made some suggestions

include/clad/Differentiator/STLBuiltins.h

include/clad/Differentiator/VisitorBase.h

lib/Differentiator/BaseForwardModeVisitor.cpp

vgvassilev

LGTM! Please address the clang-tidy reports they seem good. The const_cast report can be suppressed with:

// FIXME: We should not use const_cast to get the decl context here.
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)

include/clad/Differentiator/BaseForwardModeVisitor.h

This commit adds primitive support for custom pushforward functions for constructors. Custom constructor pushforward function support will enable the below features: - Class differentiation support for classes whose constructor Clad cannot automatically differentiate. Now, We can enable differentiation of entire C++ standard library by providing custom derivatives. - Remove the restriction of default-constructible for class types. This was a troublesome restriction. Now, the only restriction for class types is to have a sensible copy-constructor. That is, copy constructor should copy the class members and after copy-construction, both the objects should be equivalent, mathematically speaking. Constructor pushforward functions differ from ordinary pushforward functions in two important ways: - Constructor pushforward functions initialize the primal class object and the corresponding derivative object. Ordinary member function pushforwards takes an already-existing primal class object and the corresponding derivative object as inputs. - Constructor pushforward functions return a value even though constructor do not return anything. Constructor pushforward functions return initialized primal object and the derivative object. These are then used to initialize primal object and the derivative in the derivative function code. How to write custom constructor pushforward functions ---------------------------------------- Let's see how to write custom pushforward function for a constructor: - Custom constructor pushforwards must have the name `constructor_pushforward` - Custom constructor pushforwards must be defined in `::clad::custom_derivatives::class_functions` namespace. - The parameters of the custom constructor pushforward must be: {`::clad::ConstructorPushforwardTag<Class>`, original params..., derivative params...}. 'original parameters...' and 'derivative parameters...' is same as what we have for other pushforward functions. We will soon see why do we need `::clad::ConstructorPushforwardTag<T>` for constructor custom pushforwards. Let's see a basic example of how to write custom constructor pushforward. ```cpp class Coordinates { Coordinates(double px, double py, double pz) : x(px), y(py), z(pz) {} public: double x, y, z; } namespace clad { namespace custom_derivatives { namespace class_functions { // custom constructor pushforward function clad::ValueAndPushforward<Coordinates, Coordinates> constructor_pushforward(clad::ConstructorPushforwardTag<Coordinates>, double x, double y, double z, double d_x, double d_y, double d_z) { return {Coordinates(x, y, z), Coordinates(d_x, d_y, d_z) }; } } // namespace class_functions } // namespace custom_derivatives } // namespace clad // custom constructor pushforward is used as follows: // primal code Constructor c(u, v, w); // derivative code clad::ValueAndPushforward<Coordinates, Coordinates> _t0 = constructor_pushforward(clad::ConstructorPushforwardTag<Coordinates>, u, v, w, _d_u, _d_v, _d_w); Coordinates _d_c = _t0.pushforward; Coordinates c = _t0.value; ``` Now, let's see a bit advanced example based on `std::vector` constructor. ```cpp namespace clad { namespace custom_derivatives { namespace class_functions { // Custom pushforward for: vector(size_t n, const typename ::std::vector<T>::allocator_type alloc) template <typename T> clad::ValueAndPushforward<::std::vector<T>, ::std::vector<T>> constructor_pushforward( ConstructorPushforwardTag<::std::vector<T>>, size_t n, const typename ::std::vector<T>::allocator_type alloc, size_t d_n, const typename ::std::vector<T>::allocator_type d_alloc) { ::std::vector<T> v(n, alloc); ::std::vector<T> d_v(n, 0, alloc); return {v, d_v}; } // Custom pushfoward for: vector(size_t n, T val, const typename ::std::vector<T>::allocator_type alloc) template <typename T> clad::ValueAndPushforward<::std::vector<T>, ::std::vector<T>> constructor_pushforward( ConstructorPushforwardTag<::std::vector<T>>, size_t n, T val, const typename ::std::vector<T>::allocator_type alloc, size_t d_n, T d_val, const typename ::std::vector<T>::allocator_type d_alloc) { ::std::vector<T> v(n, val, alloc); ::std::vector<T> d_v(n, d_val, alloc); return {v, d_v}; } } // namespace class_functions } // namespace custom_derivatives } // namespace clad // The custom constructor pushforwards is used as follows: // Primal code: std::vector<double> v(10, u); // Derivative code: clad::ValueAndPushforward<std::vector<double>, std::vector<double>> _t0 = clad::custom_derivatives::class_functions::constructor_pushforward( clad::ConstructorPushforwardTag<std::vector<double> >(), 10, u, allocator_type(), 0, _d_u, allocator_type()); std::vector<double> d_v = _t0.pushforward; std::vector<double> v = _t0.value; ``` Why `clad::ConstructorPushforwardTag<T>`? ------------------------ So, why do we need clad::ConstructorPushforwardTag<T>? For a constructor that takes two parameters of types `size_t` and `double`, the custom pushforward will have the following signature if we do not include `::clad::ConstructorPushforwardTag<T>`: ```cpp clad::ValueAndPushforward<Class, Class> constructor_pushforward(size_t n, double val, size_t d_n, double d_val); ``` Now, the question is: How to distinguish custom constructor pushforwards for different classes? ```cpp MyClassA a(3, 5.0); MyClassB b(7, 9.0); ``` There is no way for overload resolution selector to distinguish constructor_pushforward for classes `MyClassA` and `MyClassB`. `clad::ConstructorPushforwardTag<T>` is used to identify the class for which custom constructor pushforward is defined. Please note that we cannot use the same strategy which we use for custom member function pushforwards because member function pushforwards always have parameters of the class type which are used for identifying the class. We also cannot simply ask users to define the pushforwards inside the declaration context of the class because it may not always be feasible to modify the source code of external libraries. -------------------------------------------- Fixes #965

vgvassilev

LGTM!

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch 2 times, most recently from 66e30a8 to b839cfa Compare May 26, 2024 14:13

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from b839cfa to bf6bdbf Compare May 26, 2024 14:15

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from bf6bdbf to b906096 Compare May 26, 2024 19:21

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from b906096 to 98901dd Compare June 1, 2024 19:43

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from 98901dd to 2f26fad Compare June 1, 2024 20:02

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from 2f26fad to 428ecc7 Compare June 1, 2024 20:15

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from 428ecc7 to 8c0cb99 Compare June 1, 2024 20:22

parth-07 requested a review from vgvassilev June 1, 2024 20:27

parth-07 marked this pull request as ready for review June 1, 2024 20:30

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from 8c0cb99 to 4073353 Compare June 1, 2024 21:07

vgvassilev force-pushed the FMConstructorCustomDerivativeFinal branch from 4073353 to dd598f2 Compare June 2, 2024 16:11

vgvassilev reviewed Jun 2, 2024

View reviewed changes

lib/Differentiator/BaseForwardModeVisitor.cpp Show resolved Hide resolved

github-actions bot reviewed Jun 2, 2024

View reviewed changes

vgvassilev reviewed Jun 3, 2024

View reviewed changes

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from dd598f2 to cdadd0a Compare July 1, 2024 19:29

parth-07 requested a review from vgvassilev July 1, 2024 19:36

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from cdadd0a to f1f7ced Compare July 1, 2024 19:38

github-actions bot reviewed Jul 1, 2024

View reviewed changes

include/clad/Differentiator/STLBuiltins.h Outdated Show resolved Hide resolved

include/clad/Differentiator/VisitorBase.h Outdated Show resolved Hide resolved

lib/Differentiator/BaseForwardModeVisitor.cpp Show resolved Hide resolved

vgvassilev approved these changes Jul 2, 2024

View reviewed changes

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from f1f7ced to 34568d5 Compare July 2, 2024 06:53

vgvassilev reviewed Jul 2, 2024

View reviewed changes

include/clad/Differentiator/BaseForwardModeVisitor.h Outdated Show resolved Hide resolved

parth-07 force-pushed the FMConstructorCustomDerivativeFinal branch from 34568d5 to 86128ef Compare July 2, 2024 18:59

parth-07 requested a review from vgvassilev July 2, 2024 19:14

vgvassilev approved these changes Jul 2, 2024

View reviewed changes

vgvassilev merged commit 645d2b6 into master Jul 2, 2024
89 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add primitive support for constructor custom pushforward functions #909

Add primitive support for constructor custom pushforward functions #909

parth-07 commented May 26, 2024 •

edited

Loading

github-actions bot commented May 26, 2024

github-actions bot commented May 26, 2024

github-actions bot commented May 26, 2024

github-actions bot commented May 26, 2024

codecov bot commented May 26, 2024 •

edited

Loading

github-actions bot commented Jun 1, 2024

github-actions bot commented Jun 1, 2024

github-actions bot commented Jun 1, 2024

github-actions bot commented Jun 1, 2024

github-actions bot commented Jun 1, 2024

github-actions bot left a comment

github-actions bot Jun 2, 2024

vgvassilev Jun 3, 2024

parth-07 Jul 1, 2024

github-actions bot Jun 2, 2024

vgvassilev left a comment

vgvassilev Jun 3, 2024

parth-07 Jun 7, 2024

vgvassilev Jul 2, 2024

vgvassilev Jun 3, 2024

github-actions bot left a comment

vgvassilev left a comment

vgvassilev left a comment

	auto valueAndPushforwardE = StoreAndRef(pushforwardCall);
	auto *valueAndPushforwardE = StoreAndRef(pushforwardCall);

Add primitive support for constructor custom pushforward functions #909

Add primitive support for constructor custom pushforward functions #909

Conversation

parth-07 commented May 26, 2024 • edited Loading

How to write custom constructor pushforward functions

Why clad::Identify<T>?

github-actions bot commented May 26, 2024

github-actions bot commented May 26, 2024

github-actions bot commented May 26, 2024

github-actions bot commented May 26, 2024

codecov bot commented May 26, 2024 • edited Loading

Codecov Report

github-actions bot commented Jun 1, 2024

github-actions bot commented Jun 1, 2024

github-actions bot commented Jun 1, 2024

github-actions bot commented Jun 1, 2024

github-actions bot commented Jun 1, 2024

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Jun 2, 2024

Choose a reason for hiding this comment

vgvassilev Jun 3, 2024

Choose a reason for hiding this comment

parth-07 Jul 1, 2024

Choose a reason for hiding this comment

github-actions bot Jun 2, 2024

Choose a reason for hiding this comment

vgvassilev left a comment

Choose a reason for hiding this comment

vgvassilev Jun 3, 2024

Choose a reason for hiding this comment

parth-07 Jun 7, 2024

Choose a reason for hiding this comment

vgvassilev Jul 2, 2024

Choose a reason for hiding this comment

vgvassilev Jun 3, 2024

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

vgvassilev left a comment

Choose a reason for hiding this comment

vgvassilev left a comment

Choose a reason for hiding this comment

parth-07 commented May 26, 2024 •

edited

Loading

Why `clad::Identify<T>`?

codecov bot commented May 26, 2024 •

edited

Loading