Feature/cpu profiling #5895
Merged
Feature/cpu profiling #5895
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gongweibao
reviewed
Nov 24, 2017
|
|
||
| ```text | ||
| 4696 12.040 0.003 12.040 0.003 {built-in method run} | ||
| 300005 0.874 0.000 1.681 0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/dataset/mnist.py:38(reader) |
There was a problem hiding this comment.
/home/yuyang/perf_test/.env
这个路径是不是没有必要暴露出来？另外其实也挺长的。
There was a problem hiding this comment.
恩。这个事实际上profile程序的输出。我直接拷贝过来的。确实有点长，不过
- 这个路径是挺标准的linux路径，也不太涉密。
- 这个路径确实说明了 mnist.py这个文件38行的某个函数的耗时。
先这样吧。
chengduoZH
reviewed
Nov 26, 2017
| 使用`pip install cprofilev`安装`cprofilev`工具，进而使用如下命令开启HTTP服务 | ||
|
|
||
| ```bash | ||
| cprofilev -a 0.0.0.0 -p 3214 -f profile.out main.py |
There was a problem hiding this comment.
- 这里不指定pi和端口也是可以的，默认会
http://127.0.0.1:4000。 - 也可以这样使用
python -m cprofilev main.py，会将结果输出到http://127.0.0.1:4000页面中，不过因为程序还在运行，所以需要手动刷新页面。
There was a problem hiding this comment.
本地运行是可以的。。但是，我们经常会在服务器上调试程序，-a 0.0.0.0还是挺必要的。
| | cumtime | 函数总时间。包含这个函数调用其他函数的时间 | | ||
| | percall | cumtime的每次调用平均时间 | | ||
| | filename:lineno(function) | 文件名, 行号，函数名 | | ||
|
|
There was a problem hiding this comment.
查看性能分析结果也可以借用gprof2dot工具，该工具可将性能分析文件转成一张图，同时会通过颜色标记图中最耗时的结点了路径。
优点：更直观
缺点：只是一张图，无法查看更具体的调用关系
There was a problem hiding this comment.
Cool。要不然我这个PR merge了之后，麻烦你写一下gprof2dot的使用和效果。
@gongweibao Develop image加入性能调优的工具是挺有用的。不过，
这个PR里面的所有实验都是直接在Linux下进行的。没有进docker容器。 |
|
@typhoonzero 请教下在docker内部做profiler，得到数据是准确的吗？有没有坑需要注意 |
|
@dzhwinter 我们在docker内外测过CPU的性能，觉得误差基本可以忽略不计（最大是1.7%的diff）。#5148 (comment) GPU的还未实验过。 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #5878