[NewIR]Split python api and vjp #56518

xiaoguoguo626807 · 2023-08-22T01:32:17Z

PR types

others

PR changes

others

Description

pcard-67164
split算子前向调用+反向调用链代码开发

…e/Paddle into support_prim_in_new_ir

… support_prim_in_new_ir

… upstream/czy-gen

…to upstream/czy-gen

… support_prim_in_new_ir

…ub.com/PaddlePaddle/Paddle into split_vjp2

…into split_vjp2

Aurelius84

LGTM overall

Aurelius84 · 2023-08-25T01:48:43Z

paddle/fluid/ir/dialect/paddle_dialect/ir/pd_manual_api.cc

@@ -18,5 +18,16 @@
 #include "paddle/ir/core/builtin_op.h"

 namespace paddle {
-namespace dialect {}  // namespace dialect
+namespace dialect {
+ir::OpResult split_grad(std::vector<ir::OpResult> out_grads,


Suggested change

ir::OpResult split_grad(std::vector<ir::OpResult> out_grads,

ir::OpResult split_grad(const std::vector<ir::OpResult>& out_grads,

@0x45f 我看 pd_api.h 里的 concat、add_n 的vector 入参也是值copy，这个是不是可以优化为 const &？

这个在build 函数手写完备后，可加入自动代码生成一起完善

Aurelius84 · 2023-08-25T01:53:46Z

paddle/fluid/ir/dialect/paddle_dialect/ir/pd_manual_op.h

@@ -51,9 +51,28 @@ class AddNOp : public ir::Op<AddNOp, OpYamlInfoInterface> {
  static void InferMeta(phi::InferMetaContext *infer_meta);
 };

+class SplitGradOp : public ir::Op<SplitGradOp, OpYamlInfoInterface> {


是不需要有 paddle::dialect::InferMetaInterface 接口么？我看pd_op.h 里其他的GradOp 都是有的

这个splitgrad yaml 中没有infermeta, 使用的是invoke算子concat 的infermeta, 已补充

zhangbo9674 · 2023-08-25T02:01:39Z

paddle/fluid/ir/dialect/paddle_dialect/ir/pd_manual_op.cc

@@ -14,6 +14,7 @@

 #include "paddle/fluid/ir/dialect/paddle_dialect/ir/pd_manual_op.h"
 #include "paddle/fluid/ir/dialect/paddle_dialect/ir/pd_attribute.h"
+#include "paddle/fluid/ir/dialect/paddle_dialect/ir/pd_op.h"


这里是不是可以按需 include 必要的头文件

splitgrad 调用了full op, 需要引入此头文件
int axis = axis_.owner()
->dyn_castpaddle::dialect::FullOp()
.attributes()
.at("value")
.dyn_castpaddle::dialect::ScalarAttribute()
.data()
.to();

zhangbo9674 · 2023-08-25T02:03:19Z

paddle/fluid/ir/dialect/paddle_dialect/ir/pd_manual_op.cc

+                 .dyn_cast<paddle::dialect::ScalarAttribute>()
+                 .data()
+                 .to<int>();
+  (void)axis;


不需要添加这行代码吧？

zhangbo9674 · 2023-08-25T02:05:07Z

paddle/fluid/ir/dialect/paddle_dialect/ir/pd_manual_op.cc

+  argument.AddOutputs(argument_outputs.begin(), argument_outputs.end());
+}
+
+void SplitGradOp::Verify() {}


根据前面GetOpInfo的描述，split 存在两个输入：out_grad 和 axis，Verify 中有必要做一下校验

zhangbo9674 · 2023-08-25T02:06:02Z

paddle/fluid/ir/dialect/paddle_dialect/ir/pd_manual_op.cc

+                  false,
+                  false),
+      OpInputInfo(
+          "axis", "paddle::dialect::ScalarAttribute", false, false, true)};


axis 是可变 attribute 的话，是不是有必要添加一个axis 为Scalar类型的Build 接口？

done，此处build接口无使用样例，存在覆盖率问题

… split_vjp2

Charles-hit · 2023-08-25T10:31:36Z

test/cpp/prim/test_vjp.cc

@@ -402,5 +402,74 @@ TEST(VJP, Add_BackwardTest) {
  ASSERT_EQ(dx.data<float>()[0], 1.0);
  ASSERT_EQ(dy.data<float>()[0], 1.0);
 }
+
+TEST(VJP, SplitBackwardTest) {


这儿单测能否在python端用新执行器去测试？ c++这种直接在InterpreterCore 中取值的方案不是很推荐。

* support ir api form prim * convert vector of int to intarray * add reference of lbfgs * add reference of lbfgs * support ir api for prim * Add more gen api * concat python api to concat_grad * fix gen conflict * support vjp prim mode in new ir * remove useless code * add vjp autogen v1.0 * add test for prim * resolve type conflict * modify utils * remove useless code * add split op and modify some bug of vectorType * fix conflict * add concat python test * add split python api to vjp * modify build bug * modify run bug * fix conflict bug * build bug fix * modify python api bug * modify test * fix conflict * fluid backward recover * recover conflict * reply review comments * modify opruntimeinfo num --------- Co-authored-by: cyber-pioneer <chenzhuo@tju.edu.cn> Co-authored-by: Charles-hit <wanghao107@baidu.com> Co-authored-by: 0x45f <wangzhen45@baidu.com> Co-authored-by: chenzhiyang <1792266893@qq.com> Co-authored-by: Chen Zhiyang <chenzhiyang99@126.com>

cyber-pioneer and others added 30 commits August 10, 2023 11:42

support ir api form prim

4b21c66

convert vector of int to intarray

37125f9

add reference of lbfgs

2c0166c

add reference of lbfgs

37883b2

Merge commit 'refs/pull/56162/head' of https://github.com/PaddlePaddl…

58dd125

…e/Paddle into support_prim_in_new_ir

support ir api for prim

8834e65

Merge commit 'refs/pull/56162/head' of https://github.com/PaddlePaddl…

d286e7f

…e/Paddle into support_prim_in_new_ir

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

79aa356

… support_prim_in_new_ir

Add more gen api

0cc4336

concat python api to concat_grad

0afe2ed

fix conflict

00379a9

fix conflict

3537f91

fix gen conflict

afd61ea

fix conflict

d95467a

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

adc040f

… support_prim_in_new_ir

support vjp prim mode in new ir

2f3a72a

resolve conflict

b88cc54

remove useless code

6b6fc8e

add vjp autogen v1.0

bfbb0e8

vjp codegen resolve conflict

c428ff6

Merge branch 'PaddlePaddle:develop' into upstream/czy-gen

ece1b32

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

ff5aa2c

… upstream/czy-gen

add test for prim

065c011

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

578e08b

… upstream/czy-gen

resolve type conflict

0c8db6e

Merge branch 'upstream/czy-gen' of github.com:changeyoung98/Paddle in…

873ec07

…to upstream/czy-gen

rename desctensor to lazytensor

7939ab2

modify utils

ffc71f2

remove useless code

c161003

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

95642e3

… support_prim_in_new_ir

xiaoguoguo626807 added 12 commits August 23, 2023 01:28

fix conflict

6879097

fix conflict bug

c21b912

fix conflict

a3e947e

build bug fix

98e6beb

Merge branch 'develop', commit 'refs/pull/56518/head' of https://gith…

7dca72b

…ub.com/PaddlePaddle/Paddle into split_vjp2

modify python api bug

f5d60fb

modify test

47097e8

Merge branch 'split_vjp' of https://github.com/xiaoguoguo626807/Paddle …

877262d

…into split_vjp2

fix conflict

6054bde

fix conflict

a6b5af3

fluid backward recover

2fc525f

recover conflict

1072cdd

Aurelius84 changed the title ~~【newir】Split python api and vjp~~ [NewIR]Split python api and vjp Aug 25, 2023

fix conflict

71d1c09

Aurelius84 reviewed Aug 25, 2023

View reviewed changes

zhangbo9674 reviewed Aug 25, 2023

View reviewed changes

YuanRisheng previously approved these changes Aug 25, 2023

View reviewed changes

reply review comments

0ded8f3

xiaoguoguo626807 dismissed YuanRisheng’s stale review via 0ded8f3 August 25, 2023 03:41

xiaoguoguo626807 added 2 commits August 25, 2023 06:05

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

d342f2e

… split_vjp2

modify opruntimeinfo num

5dcecd9

Aurelius84 approved these changes Aug 25, 2023

View reviewed changes

Ligoml approved these changes Aug 25, 2023

View reviewed changes

Charles-hit reviewed Aug 25, 2023

View reviewed changes

Charles-hit approved these changes Aug 25, 2023

View reviewed changes

jeff41404 approved these changes Aug 25, 2023

View reviewed changes

zhangbo9674 approved these changes Aug 25, 2023

View reviewed changes

xiaoguoguo626807 merged commit 7995a38 into PaddlePaddle:develop Aug 28, 2023

xiaoguoguo626807 deleted the split_vjp branch June 12, 2024 02:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[NewIR]Split python api and vjp #56518

[NewIR]Split python api and vjp #56518

xiaoguoguo626807 commented Aug 22, 2023

Aurelius84 left a comment

Aurelius84 Aug 25, 2023

xiaoguoguo626807 Aug 25, 2023 •

edited

Loading

Aurelius84 Aug 25, 2023

xiaoguoguo626807 Aug 25, 2023 •

edited

Loading

zhangbo9674 Aug 25, 2023

xiaoguoguo626807 Aug 25, 2023

zhangbo9674 Aug 25, 2023

xiaoguoguo626807 Aug 25, 2023

zhangbo9674 Aug 25, 2023

xiaoguoguo626807 Aug 25, 2023

zhangbo9674 Aug 25, 2023

xiaoguoguo626807 Aug 25, 2023

Charles-hit Aug 25, 2023

	ir::OpResult split_grad(std::vector<ir::OpResult> out_grads,
	ir::OpResult split_grad(const std::vector<ir::OpResult>& out_grads,

[NewIR]Split python api and vjp #56518

[NewIR]Split python api and vjp #56518

Conversation

xiaoguoguo626807 commented Aug 22, 2023

PR types

PR changes

Description

Aurelius84 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xiaoguoguo626807 Aug 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xiaoguoguo626807 Aug 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xiaoguoguo626807 Aug 25, 2023 •

edited

Loading

xiaoguoguo626807 Aug 25, 2023 •

edited

Loading