[3/4] CUDNNv8 ResNet Fusion: Add fused_donv_drelu_dbn OP #58986

Tom-Zheng · 2023-11-14T08:11:07Z

PR types

New features

PR changes

OPs

Description

This PR adds fused_dconv_drelu_dbn op, which is part of the CUDNNv8 ResNet fusion integration.

paddle-bot · 2023-11-14T08:11:13Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

Tom-Zheng · 2023-11-14T08:15:00Z

@zyfncg Hi, would you please take a look?

Tom-Zheng · 2023-11-24T05:07:55Z

@zyfncg The CI failures will be approved manually and should not block review process. Please take a look.

onecatcn · 2023-11-30T02:25:20Z

2023-11-23 11:07:36 ****************
2023-11-23 11:07:36 0. You must have raindrops2sea or XiaoguangHu01 approval for change 20+ files or add than 1000+ lines of content.
2023-11-23 11:07:36 1. You must have one RD (XiaoguangHu01,chenwhql,zhiqiu,Xreki,luotao1,qili93,Aurelius84) approval for the usage of const_cast.
2023-11-23 11:07:36 2. paddle::get is not recommended for direct use, because it may throw an bad_variant_access exception without any stack information, so please use PADDLE_GET(_**)(dtype, value) series macros here. If these macros cannot meet your needs, please use try-catch to handle paddle::get and request chenwhql (Recommend), luotao1 or lanxianghit or Aurelius84 review and approve.
2023-11-23 11:07:36 3. Unittest is not allowed to be disabled.
2023-11-23 11:07:36 You must have one RD (kolinwei(Recommend), wanghuancoder, luotao1, QingshuChen, qili93 or ZzSean or Aurelius84) approval for the usage of @unittest.skip or @unittest.skipIf.
2023-11-23 11:07:36 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-23 11:07:36 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-23 11:07:36 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-23 11:07:36 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-23 11:07:36 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-23 11:07:36 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-23 11:07:36 4. It is an Op accuracy problem, please take care of it. You must have one RD (zhangting2020 (Recommend), luotao1 or phlrain, qili93, QingshuChen or Aurelius84) approval for the usage (either add or delete) of @skip_check_grad_ci. For more information, please refer to: https://github.com/PaddlePaddle/Paddle/wiki/Gradient-Check-Is-Required-for-Op-Test. The corresponding lines are as follows:

2023-11-24 11:27:18 [tools/ci_op_benchmark.sh:280] [INFO] Or you can apply for one RD (ZzSean(Recommend), JamesLim-sy, Xreki, luotao1) approval to pass this PR.

2023-11-23 12:51:03 Please find RD for approval first, and then find TPM for approval.
2023-11-23 12:51:03 0. You must have one RD (inference[ vivienfanghuagood(Recommend), yuanlehome, qingqing01 ] or slim[ wanghaoshuang(Recommend), qingqing01 ] or train[ chenwhql(Recommend), phlrain ]) approval for the changes of Inputs/Output/Attrs of OPs. The changes of OPs will cause that the new version inference fails to load model trained by the old version. Please modify your code.
2023-11-23 12:51:03 For more details, please click [https://github.com/PaddlePaddle/Paddle/wiki/OP-Input-Output-Attribute-Compatibility-Modification].

wanghuancoder

LGTM for @unittest.skip or @unittest.skipIf

vivienfanghuagood

LGTM for add API

zyfncg · 2023-11-30T02:34:50Z

paddle/phi/infermeta/fusion.cc

+  grad_weight->set_dims(weight.dims());
+  grad_bn1_input->set_dims(bn1_input.dims());
+  grad_bn1_gamma->set_dims(bn1_gamma.dims());
+  grad_bn1_beta->set_dims(bn1_beta.dims());


please set dtype like this

zyfncg · 2023-11-30T02:36:29Z

paddle/phi/kernels/fusion/gpu/fused_dconv_drelu_dbn_kernel.cu

@@ -0,0 +1,1162 @@
+/* Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.


2022 -> 2023

zyfncg · 2023-11-30T02:40:09Z

test/legacy_test/test_fused_dconv_drelu_dbn_op.py

@@ -0,0 +1,478 @@
+#   Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.


2022 ->2023

Tom-Zheng · 2023-12-04T02:33:41Z

2023-11-30 11:48:08 0. You must have raindrops2sea or XiaoguangHu01 approval for change 20+ files or add than 1000+ lines of content.
2023-11-30 11:48:08 1. You must have one RD (XiaoguangHu01,chenwhql,zhiqiu,Xreki,luotao1,qili93,Aurelius84) approval for the usage of const_cast.
2023-11-30 11:48:08 2. paddle::get is not recommended for direct use, because it may throw an bad_variant_access exception without any stack information, so please use PADDLE_GET(_**)(dtype, value) series macros here. If these macros cannot meet your needs, please use try-catch to handle paddle::get and request chenwhql (Recommend), luotao1 or lanxianghit or Aurelius84 review and approve.
2023-11-30 11:48:08 3. Unittest is not allowed to be disabled.
2023-11-30 11:48:08 You must have one RD (kolinwei(Recommend), wanghuancoder, luotao1, QingshuChen, qili93 or ZzSean or Aurelius84) approval for the usage of @unittest.skip or @unittest.skipIf.
2023-11-30 11:48:08 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-30 11:48:08 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-30 11:48:08 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-30 11:48:08 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-30 11:48:08 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-30 11:48:08 +@unittest.skipIf(skip_unit_test(), skip_msg)
2023-11-30 11:48:08 4. It is an Op accuracy problem, please take care of it. You must have one RD (zhangting2020 (Recommend), luotao1 or phlrain, qili93, QingshuChen or Aurelius84) approval for the usage (either add or delete) of @skip_check_grad_ci. For more information, please refer to: https://github.com/PaddlePaddle/Paddle/wiki/Gradient-Check-Is-Required-for-Op-Test. The corresponding lines are as follows:
2023-11-30 11:48:08 test/legacy_test/test_fused_dconv_drelu_dbn_op.py
2023-11-30 11:48:08 +	@skip_check_grad_ci(reason="no grap op")
2023-11-30 11:48:08 +	@skip_check_grad_ci(reason="no grap op")
2023-11-30 11:48:08 +	@skip_check_grad_ci(reason="no grap op")
2023-11-30 11:48:08 +	@skip_check_grad_ci(reason="no grap op")
2023-11-30 11:48:08 +	@skip_check_grad_ci(reason="no grap op")
2023-11-30 11:48:08 +	@skip_check_grad_ci(reason="no grap op")
2023-11-30 11:48:08 There are 5 approved errors.

@XiaoguangHu01 @chenwhql @ZzSean @zhangting2020 Would you please approve?

Tom-Zheng · 2023-12-04T02:34:02Z

@qingqing01 Would you please approve for PR-CI-Static-Check?

XiaoguangHu01

LGTM

Tom-Zheng · 2023-12-04T12:57:25Z

@tianshuo78520a Would you please help approve PR-CI-Coverage

Tom-Zheng · 2023-12-04T13:05:01Z

@qingqing01 @heavengate would you please approve for PR-CI-Static-Check?

Tom-Zheng · 2023-12-05T06:25:50Z

@zyfncg Hi, would you please merge it?

paddle-bot bot added the contributor External developers label Nov 14, 2023

Tom-Zheng added the NVIDIA label Nov 14, 2023

onecatcn assigned zyfncg Nov 15, 2023

Tom-Zheng force-pushed the gh_add_dconv_drelu_dbn branch from 99c2b4a to 876270d Compare November 17, 2023 09:02

jeng1220 mentioned this pull request Nov 23, 2023

fix /usr/lib/x86_64-linux-gnu/libcudnn.so #59193

Merged

Tom-Zheng force-pushed the gh_add_dconv_drelu_dbn branch from 876270d to 6d61be1 Compare November 23, 2023 03:04

Tom-Zheng closed this Nov 24, 2023

Tom-Zheng reopened this Nov 24, 2023

onecatcn requested a review from XiaoguangHu01 November 30, 2023 02:25

onecatcn requested review from chenwhql, wanghuancoder, zhangting2020, ZzSean and vivienfanghuagood November 30, 2023 02:25

zhangting2020 previously approved these changes Nov 30, 2023

View reviewed changes

wanghuancoder previously approved these changes Nov 30, 2023

View reviewed changes

vivienfanghuagood approved these changes Nov 30, 2023

View reviewed changes

zyfncg previously approved these changes Nov 30, 2023

View reviewed changes

Tom-Zheng added 4 commits November 30, 2023 03:26

Rename output

4e2b1ad

Add fused_dconv_drelu_dbn_op

287998e

Add to CI test

2226cf9

Review changes

58a38a0

Tom-Zheng dismissed stale reviews from zyfncg, wanghuancoder, and zhangting2020 via 58a38a0 November 30, 2023 03:37

Tom-Zheng force-pushed the gh_add_dconv_drelu_dbn branch from 6d61be1 to 58a38a0 Compare November 30, 2023 03:37

onecatcn requested a review from zyfncg December 2, 2023 03:50

zyfncg approved these changes Dec 3, 2023

View reviewed changes

zhangting2020 approved these changes Dec 4, 2023

View reviewed changes

wanghuancoder approved these changes Dec 4, 2023

View reviewed changes

chenwhql approved these changes Dec 4, 2023

View reviewed changes

XiaoguangHu01 approved these changes Dec 4, 2023

View reviewed changes

onecatcn requested a review from heavengate December 5, 2023 01:56

heavengate approved these changes Dec 5, 2023

View reviewed changes

zyfncg merged commit 1099b8f into PaddlePaddle:develop Dec 5, 2023
29 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[3/4] CUDNNv8 ResNet Fusion: Add fused_donv_drelu_dbn OP #58986

[3/4] CUDNNv8 ResNet Fusion: Add fused_donv_drelu_dbn OP #58986

Tom-Zheng commented Nov 14, 2023

paddle-bot bot commented Nov 14, 2023

Tom-Zheng commented Nov 14, 2023

Tom-Zheng commented Nov 24, 2023

onecatcn commented Nov 30, 2023

wanghuancoder left a comment

vivienfanghuagood left a comment

zyfncg Nov 30, 2023

Tom-Zheng Nov 30, 2023

zyfncg Nov 30, 2023

Tom-Zheng Nov 30, 2023

zyfncg Nov 30, 2023

Tom-Zheng Nov 30, 2023

Tom-Zheng commented Dec 4, 2023

Tom-Zheng commented Dec 4, 2023

XiaoguangHu01 left a comment

Tom-Zheng commented Dec 4, 2023

Tom-Zheng commented Dec 4, 2023

Tom-Zheng commented Dec 5, 2023

		@@ -0,0 +1,1162 @@
		/* Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.

		@@ -0,0 +1,478 @@
		# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.

[3/4] CUDNNv8 ResNet Fusion: Add fused_donv_drelu_dbn OP #58986

[3/4] CUDNNv8 ResNet Fusion: Add fused_donv_drelu_dbn OP #58986

Conversation

Tom-Zheng commented Nov 14, 2023

PR types

PR changes

Description

paddle-bot bot commented Nov 14, 2023

Tom-Zheng commented Nov 14, 2023

Tom-Zheng commented Nov 24, 2023

onecatcn commented Nov 30, 2023

wanghuancoder left a comment

Choose a reason for hiding this comment

vivienfanghuagood left a comment

Choose a reason for hiding this comment

zyfncg Nov 30, 2023

Choose a reason for hiding this comment

Tom-Zheng Nov 30, 2023

Choose a reason for hiding this comment

zyfncg Nov 30, 2023

Choose a reason for hiding this comment

Tom-Zheng Nov 30, 2023

Choose a reason for hiding this comment

zyfncg Nov 30, 2023

Choose a reason for hiding this comment

Tom-Zheng Nov 30, 2023

Choose a reason for hiding this comment

Tom-Zheng commented Dec 4, 2023

Tom-Zheng commented Dec 4, 2023

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

Tom-Zheng commented Dec 4, 2023

Tom-Zheng commented Dec 4, 2023

Tom-Zheng commented Dec 5, 2023