Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

关于GPU集群训练log的疑问 #674

Closed
qubingxin opened this issue Nov 30, 2016 · 2 comments
Closed

关于GPU集群训练log的疑问 #674

qubingxin opened this issue Nov 30, 2016 · 2 comments
Assignees
@backyes
Labels
Cluster

Comments

@qubingxin
Copy link

训练集样本共计31W左右，测试集样本共计2.4W左右。用2个结点进行训练，其中一个结点的日志如下：
I1130 17:48:22.723963 32583 TrainerInternal.cpp:180] Pass=2 Batch=309 samples=158208 AvgCost=0.134166 Eval: classification_error_evaluator=0.0493528
I1130 18:01:22.744444 32583 Tester.cpp:101] Test samples=51200 cost=0.34299 Eval: classification_error_evaluator=0.160273

  1. 其中训练样本的samples是15W多，是否可以理解为31W除以2个结点得到的数字？
  2. Test samples在log中是5W多，是原始数据的2倍多，请问为何有这样的数据量差异？
@backyes
Copy link
Contributor

backyes commented Nov 30, 2016

Check #448, 看看是否有关系

我们也在简化这块的逻辑，#411

@backyes backyes self-assigned this Nov 30, 2016
@luotao1 luotao1 added the Cluster label Dec 1, 2016
@reyoung
Copy link
Collaborator

reyoung commented Dec 16, 2016

这个issue已经fix了。

原来Paddle在测试的时候，测试数据可以配置使用部分数据，也可以使用更多的数据。每一个测试周期实用的测试数据数量，和用户提供的测试数据总数没有什么关系。并且都使用--test_period参数控制。

目前将这块的逻辑修改成了每次测试全量的数据，--test_period用来控制测试间隔。

@reyoung reyoung closed this as completed Dec 16, 2016
zhhsplendid pushed a commit to zhhsplendid/Paddle that referenced this issue Sep 25, 2019
@tink2123 @shanyi15
rm fluid.core in desigin_idea (PaddlePaddle#674)
dbecff8 
* Update fluid_design_idea.md

* Update fluid_design_idea_en.md
zhhsplendid pushed a commit to zhhsplendid/Paddle that referenced this issue Sep 25, 2019
@shanyi15
Update0309 (PaddlePaddle#681)
669ad78 
* synchronize with develop (PaddlePaddle#642)

* update_commitid1.3 (PaddlePaddle#641)

* update inference c++ API doc (PaddlePaddle#634)

* update inference c++ API doc

* fix link

* thorough clean for doc (PaddlePaddle#644)

* thorough clean

* delete_DS_Store

* Cherrypick1.3 (PaddlePaddle#652)

* thorough clean

* delete_DS_Store

* [Don't merge now]update_install_doc (PaddlePaddle#643)

* update_install_doc

* follow_comments

* add maxdepth (PaddlePaddle#646)

* upload_md (PaddlePaddle#649)

* update_version (PaddlePaddle#650)

* Translation of 16 new apis (PaddlePaddle#651)

* fix_windows

* Final update 1.3 (PaddlePaddle#653)

* thorough clean

* delete_DS_Store

* update_1.3

* Deadlink fix (PaddlePaddle#654)

* fix_deadlinks

* update_docker

* Update release_note.rst

* Update index_cn.rst

* update_Paddle (PaddlePaddle#658)

* fix pic (PaddlePaddle#659)

* [to 1.3] cn api debug (PaddlePaddle#655) (PaddlePaddle#661)

* debug

* fix 2 -conv2d

* "锚" ==> anchor(s)

* Weekly cherrypick0302 (PaddlePaddle#668)

* Update programming_guide.md (PaddlePaddle#664)

* Update programming_guide.md

* Update programming_guide_en.md

* Update cn api to 1.3 (PaddlePaddle#663)

* Update cn api to 1.3 fluid & layers

* Rest to 1.3

* Weeklyupdate 0301 (PaddlePaddle#666)

* Tables_rm_op

* update_op

* update_index

* update_book_0302 (PaddlePaddle#667)

* fix_format (PaddlePaddle#669) (PaddlePaddle#670)

* fix_format

* Update Tables.md

* Update Tables_en.md

* add dataset api_cn (PaddlePaddle#673)

* rm fluid.core in desigin_idea (PaddlePaddle#674)

* Update fluid_design_idea.md

* Update fluid_design_idea_en.md

* Fix array_read code example error. (PaddlePaddle#671)

Signed-off-by: zhaoyuchen <zhaoyuchen01@baidu.com>

* add data_reader_cn (PaddlePaddle#676)

* fix doc error (PaddlePaddle#675)

* update_book_commitid (PaddlePaddle#680)

* update_book_commitid

* commitid0309

* fix typo

* book indexes (PaddlePaddle#677)
wangxicoding pushed a commit to wangxicoding/Paddle that referenced this issue Dec 9, 2021
@linjieccc
Merge pull request PaddlePaddle#674 from linjieccc/textcnn
7be95b5 
[TextCNN] Add TextCNN example for sentiment analysis
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@backyes backyes

Labels
Cluster
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@reyoung @backyes @luotao1 @qubingxin