forked from PaddlePaddle/Paddle
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add mixed precision training doc (PaddlePaddle#2)
* add mixed_precision doc
- Loading branch information
1 parent
f487345
commit b680968
Showing
2 changed files
with
29 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -33,6 +33,6 @@ PLSC具备以下特点: | |
|
||
### 高级功能 | ||
|
||
* [混合精度训练] | ||
* [混合精度训练](docs/mixed_precision.md) | ||
* [分布式参数转换] | ||
* [Base64格式图像预处理] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
# 混合精度训练 | ||
|
||
PLSC支持混合精度训练。使用混合精度训练可以提升训练的速度,同时减少训练使用的内存。 | ||
|
||
可以通过下面的代码设置开启混合精度训练: | ||
|
||
```python | ||
from __future__ import print_function | ||
import plsc.entry as entry | ||
|
||
def main(): | ||
ins = entry.Entry() | ||
ins.set_mixed_precision(True, 1.0) | ||
ins.train() | ||
|
||
if __name__ == "__main__": | ||
main() | ||
``` | ||
其中,`set_mixed_precision`函数介绍如下: | ||
|
||
| API | 描述 | 参数说明 | | ||
| :------------------- | :--------------------| :---------------------- | | ||
| set_mixed_precision(use_fp16, loss_scaling) | 设置混合精度训练 | `use_fp16`为是否开启混合精度训练,默认为False;`loss_scaling`为初始的损失缩放值,默认为1.0| | ||
|
||
- `use_fp16`:bool类型,当想要开启混合精度训练时,可将此参数设为True即可。 | ||
- `loss_scaling`:float类型,为初始的损失缩放值,这个值有可能会影响混合精度训练的精度,建议设为默认值1.0。 | ||
|
||
为了提高混合精度训练的稳定性和精度,默认开启了动态损失缩放机制。更多关于混合精度训练的介绍可参考:[混合精度训练](https://arxiv.org/abs/1710.03740) |