Skip to content

Commit

Permalink
add mixed precision training doc (PaddlePaddle#2)
Browse files Browse the repository at this point in the history
* add mixed_precision doc
  • Loading branch information
danleifeng authored and lilong12 committed Dec 17, 2019
1 parent f487345 commit b680968
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ PLSC具备以下特点:

### 高级功能

* [混合精度训练]
* [混合精度训练](docs/mixed_precision.md)
* [分布式参数转换]
* [Base64格式图像预处理]
28 changes: 28 additions & 0 deletions docs/mixed_precision.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# 混合精度训练

PLSC支持混合精度训练。使用混合精度训练可以提升训练的速度,同时减少训练使用的内存。

可以通过下面的代码设置开启混合精度训练:

```python
from __future__ import print_function
import plsc.entry as entry

def main():
ins = entry.Entry()
ins.set_mixed_precision(True, 1.0)
ins.train()

if __name__ == "__main__":
main()
```
其中,`set_mixed_precision`函数介绍如下:

| API | 描述 | 参数说明 |
| :------------------- | :--------------------| :---------------------- |
| set_mixed_precision(use_fp16, loss_scaling) | 设置混合精度训练 | `use_fp16`为是否开启混合精度训练,默认为False;`loss_scaling`为初始的损失缩放值,默认为1.0|

- `use_fp16`:bool类型,当想要开启混合精度训练时,可将此参数设为True即可。
- `loss_scaling`:float类型,为初始的损失缩放值,这个值有可能会影响混合精度训练的精度,建议设为默认值1.0。

为了提高混合精度训练的稳定性和精度,默认开启了动态损失缩放机制。更多关于混合精度训练的介绍可参考:[混合精度训练](https://arxiv.org/abs/1710.03740)

0 comments on commit b680968

Please sign in to comment.