Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

updates to the training loop / logging #172

Merged
merged 4 commits into from
Jun 28, 2017
Merged

updates to the training loop / logging #172

merged 4 commits into from
Jun 28, 2017

Conversation

alexholdenmiller
Copy link
Member

added reset_metrics, metrics.report() rounds floats, default params for train_loop use mostly -1 now (params are ignored when set to -1)

@alexholdenmiller alexholdenmiller changed the title updates to the training loop / lopgging updates to the training loop / logging Jun 27, 2017
jaseweston
jaseweston suggested changes Jun 27, 2017
View reviewed changes
examples/train_model.py
@@ -33,14 +33,17 @@
import math
import os

def run_eval(agent, opt, datatype, still_training=False):
def run_eval(agent, opt, datatype, still_training=False, max_exs=-1):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't that be in the opt?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I put it there to make it explicit, since even if you set max-valid on the command line you probably don't want to use it at the very end when you're doing the final validation/testing (you can see below it's only used for one run_eval but not the other two).

I guess that coincides with the use of still_training, so I could interpret from that instead, this is just more explicit

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i see

jaseweston
jaseweston approved these changes Jun 28, 2017
View reviewed changes
Copy link
Contributor

@jaseweston jaseweston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suspicious of you changing so much, but i dont see anything wrong..

examples/train_model.py
@@ -33,14 +33,17 @@
import math
import os

def run_eval(agent, opt, datatype, still_training=False):
def run_eval(agent, opt, datatype, still_training=False, max_exs=-1):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i see

@alexholdenmiller
Copy link
Member Author

just making everything better!

@alexholdenmiller alexholdenmiller merged commit fdadbdd into master Jun 28, 2017
@alexholdenmiller alexholdenmiller deleted the train_loop branch June 28, 2017 15:05
huzefasiyamwala added a commit to huzefasiyamwala/ParlAI that referenced this pull request Aug 17, 2017
@huzefasiyamwala
Merging master branch (#1)
1270f80 
* added MTurk update to NEWS.md

* added more MTurk news

* added more explanation for multi-assignment design

* fix virtualenv path error

* Fixed bug with vqa_v2 teacher's len (facebookresearch#164)

* Change MTurk local db to be in-memory

* remote agent fixes, switch model param to default None (facebookresearch#166)

* remote fixes (facebookresearch#167)

* Added Personalized Dialog dataset (facebookresearch#163)

* Fixed bug on building fb data (facebookresearch#171)

* ExecutableWorld,  designed to also work with BatchWorld (facebookresearch#170)

* small

* exec world

* small

* blah

* mm

* index

* index

* index

* small batch fixes

* small batch fixes

* Update NEWS.md

* updates to the training loop / logging (facebookresearch#172)

* add sigfig rounding and unit tests for utils (facebookresearch#173)

* train fixes (facebookresearch#174)

* fixes to train and dict (facebookresearch#177)

* bug fixes in drqa (facebookresearch#176)

* Add image feature extraction modules and fix minor bugs.  (facebookresearch#169)

* fix vqa_v1 and vqa_v2 testset image source.
* add image_featurizer
* add image_featurizers and examples.
* update image_featurizers.py and dialog_teacher.py based on the discussion.

* Fixes to ParsedRemoteAgent (facebookresearch#178)

* Capitalize 'all' to 'All' to follow naming convention

* Fixed missing img path and step size (facebookresearch#180)

* Fixing bibtex citation (facebookresearch#186)

Missing braces

* Added from scratch section to task tutorial (facebookresearch#184)

* vqa fixes (facebookresearch#183)

* Lazy Torch requirement (facebookresearch#182)

* fix a typo (facebookresearch#187)

* Update README.md

* Refactor MTurk html to make it more modular

* Add shutdowns (facebookresearch#191)

* Update run_tests_short.sh

* add insuranceqa as a task (facebookresearch#193)

* Save parameters of all agents through calling world.save()

* Move print statement

* Fix docstring

* mturk improvement and cleanup

* fixed path bug

* Adding cmdline args to task agents as well

* Make add_task_args a separate function

* updating save functions (facebookresearch#195)

* Added download resuming (facebookresearch#194)

* Support multiple tasks

* implemented send messages in bulk

* added HIT auto-test script

* better error message

* fixed message duplication error and other issues

* [MTurk] able to approve/reject work individually, block workers, and pay bonus; improved database status checking

* [MTurk] fixed auto test bug

* [MTurk] ignore abandoned HIT

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* world.save_agents fix (facebookresearch#198)

* save agents fix

* moar

* Update eval_model.py

* [MTurk] changed hit approval flow

* [MTurk] added unique_request_token for send_bonus

* fixed comment

* Clean up dir only on version update (facebookresearch#197)

* Update NEWS.md

* Update README.md

* group args when printing (facebookresearch#201)

* Rehosted COCO-IMG to allow for resuming download (facebookresearch#202)

* [MTurk] removed unnecessary wait

* [WIP] trying to increase max concurrent polling operations

* change poll test

* [MTurk] moved to us-east-1 (N. Virginia) for higher Lambda max connections

* turned off debug

* [MTurk] fixed bugs

* [MTurk] block_worker works

* elapsed time

* avg elapsed

* [MTurk] clean up

* [MTurk] added retry for ajax request

* Modified task tutorial (facebookresearch#206)

* Delete memnn_luatorch_cpu.rst

* Update README.md

* Update agents.py

* fix test for init files to exclude mturk html dir (facebookresearch#203)

* add task directory to mturk manager

* fix wikimovies kb teacher (facebookresearch#207)

* Default evaluation uses 'valid' datatype (facebookresearch#210)

* Add MS MARCO dataset, and modify insuranceQA to accept version (V1 or V2) (facebookresearch#200)

* Added retrying with exp backoff to downloads (facebookresearch#209)

* added reason for reject_work(); added MTurkWorld as parent class

* Update worlds.py

* added opt to MTurkWorld.__init__

* better mturk cost checking

* Fixed typo

* Update agents.py

* moved sync hit assignment info to a better place

* Fixed dialog data shared cands bug (facebookresearch#212)

* fix to acts indexing in batchworld (facebookresearch#215)

* fix insuranceqa bug (facebookresearch#211)

* Update NEWS.md

* Update NEWS.md

* Update NEWS.md

* Update NEWS.md

* minor comment and print statement fixes (facebookresearch#217)

* update basic tutorial (facebookresearch#219)

* Add TriviaQA task (facebookresearch#204)

* Update NEWS.md

* Tutorial for seq2seq agent, updated agent (facebookresearch#222)

* Add start of sentence token to dict.py (facebookresearch#221)

* minor comment and print statement fixes

* add start of sentence token to dict.py

* return original order of special tokens; change names of start and end tokens

* quick eos fix

* Update NEWS.md

* Update NEWS.md

* update seq2seq to use "END" like dictionary (facebookresearch#227)

* add hred to parlai (facebookresearch#228)

* Add placeholder agent for HRED model (facebookresearch#229)

* added email_worker API

* refactored init_aws

* setting assignment duration

* getting worker id earlier in time

* hide implementation detail of conversation_id to discourage change

* removed unnecessary create_hit_type_lock

* Added CLEVR task (facebookresearch#233)

* Exception to alert user of possible mistake when providing label and label candidate (facebookresearch#232)

* better handling for email_worker error case

* turn off keypress trigger if send button is disabled

* added abandoned HIT handling

* clean up

* disabled approve/reject for abandoned HITs

* added gating for pay_bonus

* fixed pay bonus bug

* fixed mturk agent act() and observe() bug

* fixed bug

* fixed submit HIT handling

* remove quotes around model path (facebookresearch#239)

The command line option doesn't require quotes, even if /tmp/model is a placeholder it's slightly misleading

* readme, example, and test fixes (facebookresearch#234)

* Update README.md

* bug where wrong word embedding is zeroed (facebookresearch#242)

word_dict['<NULL>'] returns the index for UNK rather than NULL and thus the wrong embedding is being zeroed

* Update README.md

* Added candidates to clevr (facebookresearch#243)

* Allow newlines in FbDialog format by using '\n' in the text file messages (facebookresearch#246)

* Simple task that just loads the specified FbDialogData file (facebookresearch#245)

* fromfile task

* again

* comment

* init py

* fromfile task: add docstring (facebookresearch#247)

* Fixed dialog_babi task 6 candidates (facebookresearch#248)

* Added image args to parlai args (facebookresearch#249)

* weak ranking system for seq2seq (facebookresearch#235)

* Added MemNN agent (facebookresearch#251)

* add references to msmarco and insuranceqa

* update msmarco description

* task list

* Added clevr to task list and item on news (facebookresearch#259)

* Update README.md

* fixed error in json format (facebookresearch#262)

* print() is a function in Python 3 (facebookresearch#256)

* print() is a function in Python 3 (facebookresearch#255)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@jaseweston jaseweston jaseweston approved these changes

Assignees
No one assigned
Labels
CLA Signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alexholdenmiller @jaseweston @facebook-github-bot