[DRR] C++ DRR (Declarative Rewrite Rule) of Paddle #55859

yuanlehome · 2023-08-01T03:39:06Z

PR types

New features

PR changes

Others

Description

一、背景与目标

背景
目前Paddle正在进行IR升级工作，其核心目标是设计一套能够由多方共用且优于当前的IR基础设施。引入的新IR会统一整个Paddle体系内的IR表示。
Pass作为对IR进行优化（常量折叠、死代码消除、运算融合等）的关键组件，也需要基于新IR重新进行设计并解决原Pass体系中存在的各种问题。通过对Paddle内现有Pass进行统计分类，发现DAG->DAG PatternRewrite类型的Pass数量占比过半。为了提升用户在新IR上开发Pass的使用体验并且降低后续全量Pass迁移的成本，我们有必要对PatternRewrite这一类型的Pass做进一步的设计优化，使用户通过简单的接口调用即可完成在新IR上的模式匹配替换Pass。

目标
针对DAG->DAG PatternRewrite的Pass场景，提供一套简洁易用，用户开发成本较低的API接口，用来实现在新IR上子图匹配替换Pass。

DRR(Declarative Rewrite Rule) Pass API并不是IR，而是对IR的统一封装，目的是让用户集中在对优化逻辑的处理上，而不需要关心对底层IR的处理。其C++形式示例如下：

void SimpliedRemoveRedundentReshapePass(DrrPassContext* ctx) {
// Source patterns：待匹配的子图
  SourcePattern pat = ctx->SourcePattern();
  const auto& reshape = pat.Op("pd_op.reshape");
  // 匹配两个重叠reshape, 对输入输出张量分别设置为 arg0 和 ret，方便后面加约束
  pat.Tensor("ret") = reshape(reshape(pat.Tensor("arg0")));

  // Constrains：约束规则
  // 要求 arg0 和 ret 的 shape 相同， 其中 Tensor.shape的类型是抽象层面的PatternShape, 而不是普通 Shape
  // RequireEqual:封装的通用API，用户无需关心基础数据结构
  pat.RequireEqual(pat.Tensor("ret").shape(), pat.Tensor("arg0").shape());
  // RequireNativeCall:用于处理基础API不能提供的操作，其中的lambda表达式会在Pass运行期执行，因此会与底层数据结构相关
  pat.RequireNativeCall([](MatchContext* match_ctx) -> bool {
    // MatchContext拿到的已经是实际值，而非PatternShape，因此需指定实际类型
    return match_ctx->Tensor("ret").shape<std::vector<int>>() == match_ctx->Tensor("arg0").shape<std::vector<int>>();
  });
 
  // Result patterns：要替换为的子图
  ResultPattern res = pat.ResultPattern();
  // 使用 arg0 替换 ret 
  // 所有 ret 参数均在Source Pattern中使用，对 ret 的替换等同于对 ret 的 producer op的删除
  res.Tensor("ret").Assign(res.Tensor("arg0"));
}

二、设计方案

整体概览

DRR Pass是在新IR上，针对DAG->DAG PatternRewrite的Pass场景，提供一套简洁易用，用户开发成本较低的API接口。主要支持的方向包括分布式训练、编译器前端以及推理在新IR上的Pass优化。

主体设计

本方案采用了分层的设计思想，将整个模块从上到下分为了三层：

最上层是用户接口，提供用户使用DRR的API接口
中间是DAG子图表示，用于将用户通过DRR API接口开发的代码转化为内部的DAG数据结构
最底层是新IR Program的匹配替换，根据上一层得到的中间DAG子图在新IR Program上做匹配，子图匹配成功后进行替换。

Others

Pcard-71500

… develop

…evelop

paddle-bot · 2023-08-01T03:39:32Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[DRR] Add Basic Class

[DRR] Add Test Case

[DRR] Fix complile bug

[DRR] Add MatchContext Class

change smart to pointor Replace 'weak_ptr' with pointer modify weak_ptr use count==0 judgment logic Replace the declaration and call of weakptr with pointer

[DRR] Replace 'weak_ptr' with pointer

CLAassistant · 2023-08-09T11:56:36Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ gongshaotian
✅ yuanlehome
❌ zyfncg
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

… drr

vivienfanghuagood · 2023-10-17T03:52:48Z

paddle/fluid/pir/drr/api/drr_pattern_context.cc

+Tensor& Op::operator()(const Tensor& arg1, const Tensor& arg2) const {
+  std::vector<const Tensor*> inputs{&arg1, &arg2};
+  auto& out = pattern_graph_->AddTmpTensor(std::shared_ptr<Tensor>(new Tensor(
+      prefix + op_type_name_ + "_" + std::to_string(count++), pattern_graph_)));


count是static变量，多线程场景是否会有访问冲突问题？

已修改为thread_local

Refine code format

… drr

zhangbo9674 · 2023-10-17T08:14:10Z

paddle/fluid/pir/drr/CMakeLists.txt

+    ${PADDLE_SOURCE_DIR}/paddle/fluid/pir/dialect/op_generator/op_creator_drr_gen.py
+)
+set(op_compat_yaml_file ${PADDLE_SOURCE_DIR}/paddle/phi/api/yaml/op_compat.yaml)
+set(op_forward_yaml_file1


目前在如下目录中还存在一个 yaml 文件，该 yaml 文件中定义了 pir 单独定义的算子，是否需要纳入该自动生成体系中？：paddle/fluid/pir/dialect/operator/ir/ops.yaml

paddle/fluid/pir/dialect/operator/ir/ops.yaml应该是加入进来了，对应的是这两个解析后的文件吧？

可能需要，这个我们后面加上～

paddle/pir/core/value.cc

winter-wang · 2023-10-17T09:33:09Z

test/cpp/pir/pattern_rewrite/CMakeLists.txt

-cc_test_old(pattern_rewrite_test SRCS pattern_rewrite_test.cc DEPS
-            ${PATTERN_REWRITE_TEST_DEPS})
+if(NOT APPLE)
+  cc_test_old(pattern_rewrite_test SRCS pattern_rewrite_test.cc DEPS


建议使用paddle_test

使用paddle_test单测链接在win上会有问题

Make count variable as thread_local in Op class

winter-wang

LGTM

winter-wang

LGTM

XiaoguangHu01

LGTM

* fix cudnn 8.7+ bug on cudnnConvolutionBiasActivationForward * add drr_rewrite_pattern.h and remove_redundent_reshape demo * add drr_context and pattern_graph class * add test case * fix cmake file * fix compile bug * fix runtime bug and refine code * add MatchContext * update code * add impl of tensor_interface * fix compile bug * change smart ptr to pointor * change smart to pointor * change smart to pointor * Replace 'weak_ptr' with pointer * modify weak_ptr use count==0 judgment logic * change smart to pointor change smart to pointor Replace 'weak_ptr' with pointer modify weak_ptr use count==0 judgment logic Replace the declaration and call of weakptr with pointer * add match * add match * remove OperationInterface * update * Add Rewrite impl of DrrRewritePattern * refine code * rename ir_value to get in IrValue * fix header include * add CreateOperation template demo * Add GraphTopo class in pattern_graph * Reimplementing the GraphTopo class using queue * Reimplementing the GraphTopo class using queue * Optimize the access method of visited tensor * Considering that the inputs of opcall may be empty * Overloading the operator() method of Op, supporting dual tensor inputs * support attr * 1. Add Op class support for multi input and multi output function. 2. Add DRR duplicate TransposeOp merge testing code * 1. Add transferOP in createOption func * fix bug * fix NotifyOperationRemoved * refine code * Fix axis bug in perm * mupdate share_ptr * update * refine drr_test ut * Modify according to review * modify reshape_op * format code * support vector<int> for attr * fix drr test * refine code * Resolve compilation loop dependencies * add RequireNativeCall * support native_call in drr api * temp tensor prefix fix * refine code * suport Tensor Assgin API in ResultPattern * refine test code * refactor ther drr_pattern class * refine test case * rename DrrPatternBuilder to DrrPatternBase * fix compile bug * adjust include * Add log info in DrrRewritePattern * use ir::get_type_name * use ir::get_type_name * support compute attrbute in drr pattern * refine code * Add fusion testing code for fullOp and expandOp * Standardize code format * Replace IR_THROW() with PADDLE_THROW() * refine code * add attention fuse demo * update * fix compile error * add multihead_matmul fuse pattern * fix multihead_matmul * Update drr_attention_fuse_test.cc add buildprogram * fix drr_attention_fuse_test compile * add fused_gemm_epilogue in drr * attr support std::vector<int64_t> * add debug log * update * fix some bug * fix confilct * support subgraph replace in source pattern graph for drr * Improve the implementation of Drr and multihead_matmul_fuse_pass * add ReorderBlockOpsPass * fix drr_attention_fuse_pass * update * update reorder_block_ops_pass * revert fusedgemm * update * Add Bottom2UpMatch() func * merge code * fix bug * add log & fix bug * refine cpp type trait * using oprand() & num_oprand() replace oprands() * fix conflict * fix compile * fix pd.xxx to pd_op.xxx * fix bug of delete op in drr * add PatternGraphMatchV2 & FindOutputOp func * refactor ir operation creator * fix include pir * fix ir * merging * Split out dfsvisitor func from FindOutputOp func * fix bug * fix output op in source pattern bug * Debugging drr_test drr_attention_fuse_test passed! * Debugging drr_fuse_linear_test passed! * Optimize the PatternGraphMatchV2 function interface and overload the operator= method in MatchContextImpl * Modify comments and function names * auto code-gen for creating ir operation in drr * delete debug log * optimize the interface of MatchFromOutputToInput() * Optimize SourcePatternGraph::OutputNodes judgment logic * polish code * using default operator=() in MatchContextImpl * fix merge conflict * create test case: drr_same_name_test * fix duplicate binding of ir op bug * Rename drr_same_name_test to drr_same_type_binding_test & Add graphical notes * refactor logic of insert point for creating new operation in drr * update * fix compile error * fix some bug * fix codestyle * fix bug * Update anchor node judgment logic * fix bug of link pir * fix codestyle * self review v1 * refine code format * set thread_local for count in op class * fix compile on mac * remove unused .h in value.cc * fix compile --------- Co-authored-by: zyfncg <zhangyunfei07@baidu.com> Co-authored-by: gongshaotian <gstian5555@outlook.com> Co-authored-by: gongshaotian <> Co-authored-by: gongshaotian <141618702+gongshaotian@users.noreply.github.com>

yuanlehome added 7 commits July 13, 2023 08:15

fix cudnn 8.7+ bug on cudnnConvolutionBiasActivationForward

e41ee6d

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

9ecefe9

… develop

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

d092636

… develop

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

b74a6df

… develop

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

9ffdf8e

… develop

Merge branch 'develop' of https://github.com/yuanlehome/Paddle into d…

2ebb74e

…evelop

add drr_rewrite_pattern.h and remove_redundent_reshape demo

4350ddb

zyfncg and others added 20 commits August 1, 2023 08:58

add drr_context and pattern_graph class

665458d

Merge pull request #2 from zyfncg/my_drr

5068af5

[DRR] Add Basic Class

add test case

577161d

fix cmake file

4d90c86

Merge pull request #3 from zyfncg/my_drr

37d179b

[DRR] Add Test Case

fix compile bug

445210e

fix runtime bug and refine code

29c69f0

Merge pull request #4 from zyfncg/my_drr

4691895

[DRR] Fix complile bug

add MatchContext

ac346c0

update code

984ca9d

add impl of tensor_interface

0dc6c0c

fix compile bug

b86c832

Merge pull request #6 from zyfncg/drr_attr

727a5c1

[DRR] Add MatchContext Class

change smart ptr to pointor

61ff236

change smart to pointor

b988494

change smart to pointor

e5deff3

Replace 'weak_ptr' with pointer

4608b5f

modify weak_ptr use count==0 judgment logic

2d4df1a

change smart to pointor

dd6bd6d

change smart to pointor Replace 'weak_ptr' with pointer modify weak_ptr use count==0 judgment logic Replace the declaration and call of weakptr with pointer

Merge pull request #7 from gongshaotian/drr

7279d74

[DRR] Replace 'weak_ptr' with pointer

Merge branch 'develop' into drr

344645a

yuanlehome added 3 commits October 13, 2023 08:34

fix codestyle

a21c96b

self review v1

6330309

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

8a0f5b4

… drr

vivienfanghuagood reviewed Oct 17, 2023

View reviewed changes

zyfncg and others added 7 commits October 17, 2023 03:56

refine code format

2c6fb1e

Merge branch 'drr' of https://github.com/yuanlehome/Paddle into drr_pass

2866cb5

Merge pull request #41 from zyfncg/drr_pass

c80faa8

Refine code format

Merge branch 'drr' of https://github.com/yuanlehome/Paddle into drr

b0f9ce0

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

e9bb397

… drr

set thread_local for count in op class

55573f8

fix compile on mac

ad3764f

yuanlehome force-pushed the drr branch from 989e97b to ad3764f Compare October 17, 2023 06:47

zhangbo9674 reviewed Oct 17, 2023

View reviewed changes

winter-wang reviewed Oct 17, 2023

View reviewed changes

paddle/pir/core/value.cc Outdated Show resolved Hide resolved

remove unused .h in value.cc

6d4ab15

winter-wang reviewed Oct 17, 2023

View reviewed changes

yuanlehome added 2 commits October 17, 2023 17:34

Merge pull request #42 from zyfncg/drr_pass

0430ba0

Make count variable as thread_local in Op class

Merge branch 'drr' of https://github.com/yuanlehome/Paddle into drr

a1c093b

winter-wang previously approved these changes Oct 17, 2023

View reviewed changes

fix compile

61e4d31

yuanlehome dismissed winter-wang’s stale review via 61e4d31 October 17, 2023 13:04

zhangbo9674 approved these changes Oct 18, 2023

View reviewed changes

winter-wang approved these changes Oct 18, 2023

View reviewed changes

vivienfanghuagood approved these changes Oct 18, 2023

View reviewed changes

XiaoguangHu01 approved these changes Oct 18, 2023

View reviewed changes

zyfncg approved these changes Oct 18, 2023

View reviewed changes

yuanlehome merged commit cce81ea into PaddlePaddle:develop Oct 18, 2023
28 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DRR] C++ DRR (Declarative Rewrite Rule) of Paddle #55859

[DRR] C++ DRR (Declarative Rewrite Rule) of Paddle #55859

yuanlehome commented Aug 1, 2023 •

edited

Loading

paddle-bot bot commented Aug 1, 2023

CLAassistant commented Aug 9, 2023 •

edited

Loading

vivienfanghuagood Oct 17, 2023

yuanlehome Oct 17, 2023

zhangbo9674 Oct 17, 2023

zyfncg Oct 18, 2023

yuanlehome Oct 18, 2023

winter-wang Oct 17, 2023

yuanlehome Oct 17, 2023

winter-wang left a comment

winter-wang left a comment

XiaoguangHu01 left a comment

[DRR] C++ DRR (Declarative Rewrite Rule) of Paddle #55859

[DRR] C++ DRR (Declarative Rewrite Rule) of Paddle #55859

Conversation

yuanlehome commented Aug 1, 2023 • edited Loading

PR types

PR changes

Description

一、背景与目标

二、设计方案

Others

paddle-bot bot commented Aug 1, 2023

CLAassistant commented Aug 9, 2023 • edited Loading

vivienfanghuagood Oct 17, 2023

Choose a reason for hiding this comment

yuanlehome Oct 17, 2023

Choose a reason for hiding this comment

zhangbo9674 Oct 17, 2023

Choose a reason for hiding this comment

zyfncg Oct 18, 2023

Choose a reason for hiding this comment

yuanlehome Oct 18, 2023

Choose a reason for hiding this comment

winter-wang Oct 17, 2023

Choose a reason for hiding this comment

yuanlehome Oct 17, 2023

Choose a reason for hiding this comment

winter-wang left a comment

Choose a reason for hiding this comment

winter-wang left a comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

yuanlehome commented Aug 1, 2023 •

edited

Loading

CLAassistant commented Aug 9, 2023 •

edited

Loading