add weight_norm & remove_weight_norm #26131
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
|
✅ This PR's description meets the template requirements! |
jzhang533
reviewed
Aug 16, 2020
| __all__ = ['weight_norm', 'remove_weight_norm'] | ||
|
|
||
|
|
||
| def l2_norm(x, axis, epsilon=1e-12, name=None): |
There was a problem hiding this comment.
不可以.. layers. l2_normalize只返回了out，但是这里是要用到另一个norm输出..
python/paddle/fluid/param_attr.py
Outdated
| @@ -216,6 +216,8 @@ class WeightNormParamAttr(ParamAttr): | |||
| It is recommended to use ``minimize(loss, grad_clip=clip)`` to clip gradient. | |||
| There are three clipping strategies: :ref:`api_fluid_clip_GradientClipByGlobalNorm` , | |||
| :ref:`api_fluid_clip_GradientClipByNorm` , :ref:`api_fluid_clip_GradientClipByValue` . | |||
|
|
|||
| Please use 'paddle.nn.utils.weight_norm' in dygraph mode. | |||
There was a problem hiding this comment.
move this line to the beginning of the docstring would be better. (i.e.: Line 206)
| .. code-block:: python | ||
|
|
||
| import numpy as np | ||
| import paddle.fluid as fluid |
| conv = Conv2D(3, 5, 3) | ||
| wn = weight_norm(conv) | ||
| remove_weight_norm(conv) | ||
| print(conv.weight.shape) |
There was a problem hiding this comment.
这个print能体现remove_weight_norm生效了么？
There was a problem hiding this comment.
换成 print conv.weight_g 了，谢谢~
|
|
||
| def weight_norm(layer, name='weight', dim=0): | ||
| """ | ||
| This weight_norm layer apply weight normalization to a parameter according to the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
APIs
Describe
Add weight_norm and remove_weight_norm for dygraph