Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Inference] auto mixed precision inference support white list #56535

Conversation

yuanlehome
Copy link
Contributor

@yuanlehome yuanlehome commented Aug 22, 2023

PR types

New features

PR changes

Others

Description

For custom op, auto mixed precision inference support white list.

  • 对于一阶段混合精度推理,新增Exp_EnableMixedPrecisionOps c++ api和exp_enable_mixed_precision_ops python api用来设置AMP白名单
  • 对于二阶段混合精度推理,原convert_to_mixed_precision python api新增white_list可选参数来设置AMP白名单

白名单中的算子会不经过内部逻辑的判断直接认为默认支持低精度,黑名单与之相反。

NOTE: API更改均是兼容升级。

除此之外,本PR还做了如下工作:

  1. convert_to_mixed_precision底层增加后处理逻辑,跑identity_op_clean_pass;
  2. identity_op_clean_pass内新增clean连续两个级联cast op的逻辑;
  3. 补充identity_op_clean_pass内相关单测;

Related issue: #56491
Related doc: PaddlePaddle/Paddle-Inference-Demo#463

Others

Pcard-71500
Bug fix: #56664

@paddle-bot
Copy link

paddle-bot bot commented Aug 22, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@yuanlehome yuanlehome changed the title auto mixed precision inference support white list [Inference] auto mixed precision inference support white list Aug 22, 2023
@yuanlehome yuanlehome force-pushed the convert_mixed_precision_support_custom_op branch from d94f4f1 to 69b8ac6 Compare August 22, 2023 08:55
@yuanlehome yuanlehome force-pushed the convert_mixed_precision_support_custom_op branch from ef5072a to ac4f334 Compare August 24, 2023 03:52
// cast_op_2_out
// ->
// pre_op -> cast_op_2_out
struct FindTwoCastOpPattern : public PatternBase {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为啥会带上这个PASS的修改?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

见issue讨论

Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yuanlehome yuanlehome merged commit ecff21e into PaddlePaddle:develop Aug 25, 2023
BeingGod pushed a commit to BeingGod/Paddle that referenced this pull request Sep 9, 2023
…Paddle#56535)

* auto mixed precision inference support white list

* update

* update

* update

* move down identity_op_clean_pass

* fix code style
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants