Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Inference] Add TryShrinkMemory interface. #28409

Merged
merged 10 commits into from
Nov 11, 2020
Merged

[Inference] Add TryShrinkMemory interface. #28409

merged 10 commits into from
Nov 11, 2020

Conversation

jiweibo
Copy link
Contributor

@jiweibo jiweibo commented Nov 4, 2020 •
edited
Loading

PR types

New features

PR changes

Others

Describe

该PR增加了TryShrinkMemory接口，具体功能及底层逻辑如下：

  • Predictor新增TryShrinkMemory接口，该接口调用底层memory::Release(place)接口，来释放空余的内/显存chunk。
  • Predictor析构的时候，会主动析构sub_scope_，此时调用memory::Release(place)接口，减小显存池。
  • Predictor内部包含了Shared_ptr scope，修改该scope的默认deleter，当最后一个Predictor析构后，scope调用其deleter，权重所占显存归还显存池，调用底层memory::Release接口，释放显存池。

TryShrinkMemory逻辑如下：在sub_scope_中清除tmp tensor，其所占空间归还内/显存池；memory::Release(place)接口查看是否有没有被占用的chunk，若有则释放，从而降低显/内存占用。

demo： https://github.com/PaddlePaddle/Paddle-Inference-Demo/tree/master/c%2B%2B/test/shrink_memory
Memory底层增加Release接口的pr见：#28355 #28456

@paddle-bot-old
Copy link

paddle-bot-old bot commented Nov 4, 2020

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@jiweibo jiweibo changed the title Add ShrinkMemory interface. [Inference] Add ShrinkMemory interface. Nov 4, 2020
@jiweibo jiweibo mentioned this pull request Nov 4, 2020
Merged
Shixiaowei02
Shixiaowei02 previously approved these changes Nov 5, 2020
View reviewed changes
@jiweibo jiweibo changed the title [Inference] Add ShrinkMemory interface. [Inference] Add TryShrinkMemory interface. Nov 6, 2020
Superjomn
Superjomn approved these changes Nov 11, 2020
View reviewed changes
Copy link
Contributor

@Superjomn Superjomn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

paddle/fluid/inference/api/analysis_predictor.h Outdated Show resolved Hide resolved
paddle/fluid/inference/api/analysis_predictor_tester.cc Outdated Show resolved Hide resolved
@jiweibo jiweibo merged commit 1bf4836 into PaddlePaddle:develop Nov 11, 2020
@jiweibo jiweibo deleted the shrink_memory branch November 11, 2020 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Shixiaowei02 Shixiaowei02 Shixiaowei02 approved these changes

@Superjomn Superjomn Superjomn approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jiweibo @Superjomn @Shixiaowei02