Det &Slogdet #34992

huangxu96 · 2021-08-18T06:54:14Z

PR types

New features

PR changes

APIs

Describe

Add new API : paddle.linalg.det & paddle.linalg.slogdet

API Alias：paddle.det& paddle.slogdet

Example:

paddle.det():

Calculates the determinant value of a square matrix or batches of square matrices.

Supports the input of float, double

Args:
x (Tensor): input (Tensor): the input matrix of size (n, n) or the batch of matrices of size
(*, n, n) where * is one or more batch dimensions.

Returns:
y (Tensor): the determinant value of a square matrix or batches of square matrices.

Example: 
    .. code-block:: python

    import paddle

    x =  paddle.randn([3,3,3])

    A = paddle.det(x)

    print(A)

    # [ 0.02547996,  2.52317095, -6.15900707])

paddle.slogdet():

Calculates the sign and natural logarithm of the absolute value of a square matrix's or batches square matrices' determinant.
The determinant can be computed with ``sign * exp(logabsdet)

Supports the input of float, double

Note that for matrices that have zero determinant, this returns (0, -inf)

Args:
x (Tensor): the batch of matrices of size : math:(*, n, n)
where math:* is one or more batch dimensions.

Returns:
y (Tensor): A tensor containing the sign of the determinant and the natural logarithm
of the absolute value of determinant, respectively.

Example:
.. code-block:: python

    import paddle

    x =  paddle.randn([3,3,3])

    A = paddle.slogdet(x)

    print(A)

    # [[ 1.        ,  1.        , -1.        ],
    # [-0.98610914, -0.43010661, -0.10872950]])

…e determinant value.

paddle-bot-old · 2021-08-18T06:54:16Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… slogdet

JamesLim-sy · 2021-09-09T11:49:54Z

paddle/fluid/operators/determinant_op.cu

+  if (tid < numel) {
+    out[tid] = static_cast<T>(1);
+  }
+}


这个CUDA kernel可以用set_constant替代

这部分后面会被删除。

JamesLim-sy · 2021-09-09T11:51:58Z

paddle/fluid/operators/determinant_op.cu

+    auto input_dim_size = input->dims().size();
+
+    std::vector<int64_t> res_in = vectorize(framework::stride(input->dims()));
+    paddle::framework::Tensor input_stride_tensor;


line21 已经声明了using Tensor = framework::Tensor;

JamesLim-sy · 2021-09-09T11:52:40Z

paddle/fluid/operators/determinant_op.cu

+    auto output_dim_size = output->dims().size();
+
+    int batch_count = 1;
+    for (int i = 0; i < input->dims().size() - 2; i++) {


line39 声明了auto input_dim_size = input->dims().size();

JamesLim-sy · 2021-09-09T12:08:54Z

paddle/fluid/operators/determinant_op.h

+      auto end_idx = input_vec.begin() + (i + 1) * rank * rank;
+      std::vector<T> sub_vec(begin_idx,
+                             end_idx);  // get every square matrix data
+      Eigen::MatrixXf matrix(rank, rank);


Eigen::MatrixXf 这个看着仅能支持fp32类型的计算，注册类型应该是不止fp32类型的，可以仿照这种方式做一下。
https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/amp/fp16_type_traits.h

Quite curious about we can only copy data to use Eigen::MatrixXf::determinant. Have you seen https://eigen.tuxfamily.org/dox/group__TutorialMapClass.html ? I am not sure whether the link would help, but I think that we can have a try.

JamesLim-sy · 2021-09-09T12:10:51Z

paddle/fluid/operators/determinant_op.h

+    auto* output = context.Output<framework::Tensor>("Out");
+
+    int batch_count = 1;
+    for (int i = 0; i < input->dims().size() - 2; i++) {


这里加一行注释解释一下为什么要减去2

Also add a Paddle Enforce to make sure the input->dims().size() >= 2

Both are done.

JamesLim-sy · 2021-09-09T12:13:26Z

paddle/fluid/operators/determinant_op.h

+      auto end_idx = input_vec.begin() + (i + 1) * rank * rank;
+      std::vector<T> sub_vec(begin_idx,
+                             end_idx);  // get every square matrix data
+      Eigen::MatrixXf matrix(rank, rank);


这里的问题同上

JamesLim-sy · 2021-09-09T12:16:46Z

paddle/fluid/operators/determinant_op.h

+      Eigen::MatrixXf matrix(rank, rank);
+      for (int i = 0; i < rank; ++i) {
+        for (int j = 0; j < rank; ++j) {
+          matrix(i, j) = sub_vec[rank * i + j];


可以尝试按行填值，而不是elmentwise级别填值

JamesLim-sy · 2021-09-09T12:20:08Z

paddle/fluid/operators/determinant_op.h

+    int batch_count = 1;
+    for (int i = 0; i < input->dims().size() - 2; i++) {
+      batch_count *= input_dim[i];
+    }


这块函数调用了很多次，封装一下吧

… slogdet

paddle/fluid/operators/determinant_op.cc

paddle/fluid/operators/determinant_op.cu

zhhsplendid · 2021-09-10T07:15:49Z

paddle/fluid/operators/determinant_op.h

+    auto* output = context.Output<framework::Tensor>("Out");
+
+    int batch_count = 1;
+    for (int i = 0; i < input->dims().size() - 2; i++) {


Also add a Paddle Enforce to make sure the input->dims().size() >= 2

paddle/fluid/operators/determinant_op.h

python/paddle/fluid/tests/unittests/test_determinant_op.py

zhhsplendid · 2021-09-10T08:01:22Z

python/paddle/fluid/tests/unittests/test_determinant_op.py

+    def init_data(self):
+        self.case = np.random.randn(3, 3, 3, 3).astype('float32')
+        self.inputs = {'Input': self.case}
+        self.target = np.array(np.linalg.slogdet(self.inputs['Input']))


Same as above

python/paddle/fluid/tests/unittests/test_determinant_op.py

python/paddle/fluid/tests/unittests/op_test.py

thisjiang · 2021-09-10T08:58:49Z

paddle/fluid/operators/determinant_op.cc

+
+  void InferShape(framework::InferShapeContext *ctx) const override {
+    OP_INOUT_CHECK(ctx->HasInput("Input"), "Input", "Input",
+                   "DeterminantGradOp");


反向还需要Out的值，所以这儿麻烦加上

OP_INOUT_CHECK(ctx->HasInput("Out"), "Input", "Out", "DeterminantGradOp");

#34992 (comment)

thisjiang · 2021-09-10T08:58:51Z

paddle/fluid/operators/determinant_op.cc

+ protected:
+  void Apply(GradOpPtr<T> grad_op) const override {
+    grad_op->SetType("determinant_grad");
+    grad_op->SetInput("Input", this->Input("Input"));


反向还需要Out的值，所以这儿麻烦加上

grad_op->SetInput("Out", this->Output("Out"));

thisjiang · 2021-09-10T08:58:53Z

paddle/fluid/operators/determinant_op.cc

+
+  void InferShape(framework::InferShapeContext *ctx) const override {
+    OP_INOUT_CHECK(ctx->HasInput("Input"), "Input", "Input",
+                   "SlogDeterminantGradOp");


thisjiang · 2021-09-10T08:58:54Z

paddle/fluid/operators/determinant_op.cc

+ protected:
+  void Apply(GradOpPtr<T> grad_op) const override {
+    grad_op->SetType("slogdeterminant_grad");
+    grad_op->SetInput("Input", this->Input("Input"));


thisjiang · 2021-09-14T09:42:59Z

paddle/fluid/operators/determinant_op.cc

+                  ops::SlogDeterminantGradOpMaker<paddle::imperative::OpBase>);
+
+REGISTER_OPERATOR(slogdeterminant_grad,
+                  ops::DeterminantGradOp)  // reuse det grad op


这儿应该是ops::SlogDeterminantGradOp

这里暂时复用 DeterminantGradOp.

sneaxiy · 2021-09-16T04:25:27Z

paddle/fluid/operators/determinant_op.cc

+
+  void InferShape(framework::InferShapeContext *ctx) const override {
+    OP_INOUT_CHECK(ctx->HasInput("Input"), "Input", "Input", "determinant");
+    OP_INOUT_CHECK(ctx->HasOutput("Out"), "Output", "Out", "determinant");


Should do infer shape at compile time at least?

… slogdet

…rsion

zhhsplendid

LGTM

zhangting2020

LGTM for skip_check_grad_ci

lanxianghit

LGTM for API change

Add new API : paddle.linalg.det & paddle.linalg.slogdet API Alias：paddle.det& paddle.slogdet

This PR added det and slogdet API to release/2.2 It is cherry-pick from #34992 and #36013

Add new API : paddle.linalg.det & paddle.linalg.slogdet API Alias：paddle.det& paddle.slogdet

split batched matrixs to single matrix, invoke eigen libary to get th…

b991e1f

…e determinant value.

add cuda code of determinant_op

548534f

huangxu96 force-pushed the slogdet branch from 8350917 to 6d15b5e Compare August 25, 2021 06:42

huangxu96 force-pushed the slogdet branch from 6d15b5e to fa4ad12 Compare August 26, 2021 13:39

huangxu96 force-pushed the slogdet branch from fa4ad12 to 4845e31 Compare August 27, 2021 03:09

Add API of slogdet

c04f269

huangxu96 force-pushed the slogdet branch from 4845e31 to c04f269 Compare August 27, 2021 03:30

huangxu96 added 3 commits August 30, 2021 13:15

add unittest and fix some bug

3fc793b

add unittest and fix some bug

04e186c

add unittest and cuda code

9da88ba

huangxu96 changed the title ~~Slogdet~~ Det &Slogdet Sep 8, 2021

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

ca90942

… slogdet

huangxu96 force-pushed the slogdet branch from 46981be to ca90942 Compare September 8, 2021 09:06

modified code style

56845ed

JamesLim-sy reviewed Sep 9, 2021

View reviewed changes

huangxu96 added 2 commits September 10, 2021 06:57

fix bug in unittest

224250d

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

8d75e54

… slogdet

zhhsplendid reviewed Sep 10, 2021

View reviewed changes

python/paddle/fluid/tests/unittests/op_test.py Outdated Show resolved Hide resolved

thisjiang reviewed Sep 10, 2021

View reviewed changes

fix as review

c0ddcfe

thisjiang reviewed Sep 14, 2021

View reviewed changes

huangxu96 force-pushed the slogdet branch from f6ee719 to 2693c96 Compare September 17, 2021 02:51

huangxu96 force-pushed the slogdet branch from 2693c96 to 47c2191 Compare September 17, 2021 06:09

sneaxiy reviewed Sep 17, 2021

View reviewed changes

modify as review

ae11334

huangxu96 force-pushed the slogdet branch from 47c2191 to ae11334 Compare September 17, 2021 09:10

huangxu96 added 2 commits September 17, 2021 11:17

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

ceb2c43

… slogdet

use std::abs instead gcc abs to avoid bugs occurs in different gcc ve…

200562a

…rsion

zhhsplendid previously approved these changes Sep 18, 2021

View reviewed changes

zhangting2020 previously approved these changes Sep 18, 2021

View reviewed changes

modify as TPM's review

20b5c87

huangxu96 dismissed stale reviews from zhangting2020 and zhhsplendid via 20b5c87 September 22, 2021 02:48

TCChenlong approved these changes Sep 22, 2021

View reviewed changes

zhangting2020 approved these changes Sep 22, 2021

View reviewed changes

zhhsplendid approved these changes Sep 22, 2021

View reviewed changes

lanxianghit approved these changes Sep 22, 2021

View reviewed changes

zhhsplendid merged commit 9ce45dd into PaddlePaddle:develop Sep 22, 2021

huangxu96 mentioned this pull request Sep 22, 2021

[Cherry-pick]Det &Slogdet (#34992) #35922

Closed

huangxu96 added a commit to huangxu96/Paddle that referenced this pull request Sep 24, 2021

Det &Slogdet (PaddlePaddle#34992)

3bfc3d3

Add new API : paddle.linalg.det & paddle.linalg.slogdet API Alias：paddle.det& paddle.slogdet

ghost pushed a commit to piotrekobi/Paddle that referenced this pull request Sep 24, 2021

Det &Slogdet (PaddlePaddle#34992)

b660bfc

Add new API : paddle.linalg.det & paddle.linalg.slogdet API Alias：paddle.det& paddle.slogdet

zhhsplendid pushed a commit to zhhsplendid/Paddle that referenced this pull request Sep 26, 2021

Det &Slogdet (PaddlePaddle#34992)

928f937

Add new API : paddle.linalg.det & paddle.linalg.slogdet API Alias：paddle.det& paddle.slogdet

zhhsplendid mentioned this pull request Sep 26, 2021

[cherry-pick] Add Det and Slogdet API to Release 2.2 #36083

Merged

lanxianghit pushed a commit that referenced this pull request Sep 26, 2021

[cherry-pick] Add Det and Slogdet API to Release 2.2 (#36083)

ba2a1bb

This PR added det and slogdet API to release/2.2 It is cherry-pick from #34992 and #36013

AnnaTrainingG pushed a commit to AnnaTrainingG/Paddle that referenced this pull request Sep 29, 2021

Det &Slogdet (PaddlePaddle#34992)

01e71ff

Add new API : paddle.linalg.det & paddle.linalg.slogdet API Alias：paddle.det& paddle.slogdet

Det &Slogdet #34992

Det &Slogdet #34992

Conversation

huangxu96 commented Aug 18, 2021 • edited Loading

PR types

PR changes

Describe

Example:

paddle-bot-old bot commented Aug 18, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhhsplendid left a comment

Choose a reason for hiding this comment

zhangting2020 left a comment

Choose a reason for hiding this comment

lanxianghit left a comment

Choose a reason for hiding this comment

huangxu96 commented Aug 18, 2021 •

edited

Loading