Fix the problem when rnns compatible with cudnn's parameters are created again #31800
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
iclementine
changed the title
iclementine
pushed a commit
to iclementine/Paddle
that referenced
this pull request
Mar 29, 2021
XiaoguangHu01
pushed a commit
that referenced
this pull request
Mar 30, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Bug fixes
PR changes
Others
Describe
Test a typical case in static graph usage: create two nearly identical program with a shared startup program to share their parameters.
When creating a parameter, the name is checked. If there is already a parameter with the same name, which is the output of a operator(i.e. its creator), its re-creation is skipped.
But if that parameter has been the output of more than one operators, an exception is raised. For special cases, white list is added. Flattening rnn's parameters for the need to call cudnn kernel is such a case. When a SimpleRNN/GRU/LSTM is compatible with cudnn kernel, the parameters are packed together for the need of cudnn kernel. These parameters are first created by their creators, then they are also the ouput of the packing operator (a.k.a coalesce_tensor).
The parameters' data are possibly copied to a new memory block(packed together), a new tensor pointing to the memory block is created. Also the original parameters' data pointer is pointed to the new address.
So we need to add a white list when creating parameter with the same name twice.