Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Random op] remove FLAGS_use_curand of all Random OP's CUDA implementation #41308

Merged
merged 1 commit into from
Apr 7, 2022
Merged

[Random op] remove FLAGS_use_curand of all Random OP's CUDA implementation #41308

merged 1 commit into from
Apr 7, 2022

Conversation

zhwesky2010
Copy link
Contributor

@zhwesky2010 zhwesky2010 commented Apr 2, 2022 •
edited
Loading

PR types

Performance optimization

PR changes

OPs

Describe

对 随机数系列OP 进行了整体升级，采用性能优的CUDA并行式 随机数算法curandStatePhilox4_32_10_t，并添加了不兼容保护开关FLAGS_use_curand，共涉及到以下PR：

  1. [Random op]optimize CUDA implementation of dropout OP #40874
  2. [Random op]optimize CUDA implementation of multinomial OP #40752
  3. [Random op]optimize CUDA implementation of randperm OP #40464
  4. [Random op]optimize CUDA implementaion of randint OP #39952
  5. [Random op]optimize CUDA implementaion of bernoulli OP #39732
  6. [Random op]optimize CUDA implementaion of uniform/gaussian OP #38611
  7. [Random op]add new API/OP:paddle.Tensor.exponential_ #38256
  8. [Random op]add new API/OP: paddle.poisson #38117

该PR是上面所有PR的最终升级PR，将移除不兼容保护开关FLAGS_use_curand，对性能有2~145倍提升，具体性能加速数据如下：

infoflow 2022-04-06 15-45-22

注：该PR仅改变上述API/OP的随机数采样方式，仅改变采样值，概率密度函数不变，理论上对模型不产生收敛影响。

@paddle-bot-old
Copy link

paddle-bot-old bot commented Apr 2, 2022

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@zhwesky2010 zhwesky2010 changed the title 1 remove FLAGS_use_curand and change all random op CUDA implementation remove FLAGS_use_curand and change all random op CUDA implementation Apr 2, 2022
@zhwesky2010 zhwesky2010 changed the title remove FLAGS_use_curand and change all random op CUDA implementation [Random op] remove FLAGS_use_curand for all random op's CUDA implementation Apr 2, 2022
@zhwesky2010 zhwesky2010 changed the title [Random op] remove FLAGS_use_curand for all random op's CUDA implementation [Random op] remove FLAGS_use_curand for all Random OP's CUDA implementation Apr 2, 2022
@zhwesky2010 zhwesky2010 mentioned this pull request Apr 6, 2022
Merged
@zhwesky2010 zhwesky2010 changed the title [Random op] remove FLAGS_use_curand for all Random OP's CUDA implementation [Random op] remove FLAGS_use_curand of all Random OP's CUDA implementation Apr 6, 2022
XiaoguangHu01
XiaoguangHu01 approved these changes Apr 7, 2022
View reviewed changes
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

jeff41404
jeff41404 approved these changes Apr 7, 2022
View reviewed changes
Copy link
Contributor

@jeff41404 jeff41404 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhwesky2010 zhwesky2010 merged commit 9714878 into PaddlePaddle:develop Apr 7, 2022
douch pushed a commit to douch/Paddle that referenced this pull request Apr 10, 2022
@zhwesky2010
remove FLAGS_use_curand and change all random op CUDA implementation (P…
0372a4d 
…addlePaddle#41308)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lanxianghit lanxianghit lanxianghit approved these changes

@XiaoguangHu01 XiaoguangHu01 XiaoguangHu01 approved these changes

@jeff41404 jeff41404 jeff41404 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@zhwesky2010 @jeff41404 @XiaoguangHu01 @lanxianghit