Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Count updates of DQN #341

Merged
merged 2 commits into from
Oct 25, 2018
Merged

Count updates of DQN #341

merged 2 commits into from
Oct 25, 2018

Conversation

muupan
Copy link
Member

@muupan muupan commented Oct 24, 2018

This PR will add n_updates to the statistics of DQN. It is helpful for debugging.

I confirmed that

python examples/gym/train_dqn_gym.py --gpu -1 --replay-start-size 1000 --update-interval 1 --steps 2000

gets ('n_updates', 1000)

INFO:chainerrl.experiments.train_agent:outdir:results/20181024T204226.214715 step:2000 episode:9 R:-1.3913073996433227
INFO:chainerrl.experiments.train_agent:statistics:[('average_q', 0.3017159130785429), ('average_loss', 0.002146864709680402), ('n_updates', 1000)]

, and

python examples/gym/train_dqn_gym.py --gpu -1 --replay-start-size 1000 --update-interval 2 --steps 2000

gets ('n_updates', 500)

INFO:chainerrl.experiments.train_agent:outdir:results/20181024T204321.815223 step:2000 episode:9 R:-1.1019894359275235
INFO:chainerrl.experiments.train_agent:statistics:[('average_q', 0.25655420187569583), ('average_loss', 0.002052869248714256), ('n_updates', 500)]

.

@muupan muupan mentioned this pull request Oct 24, 2018
5 tasks
Copy link
Member

@toslunar toslunar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.optimizer.t maintains this value and possibly more robust in serialization.

@muupan
Copy link
Member Author

muupan commented Oct 25, 2018

Good point. It uses self.optimizer.t now, and I confirmed that the printed statistics are the same as above.

Copy link
Member

@toslunar toslunar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@toslunar toslunar merged commit a54268a into chainer:master Oct 25, 2018
@muupan muupan modified the milestone: v0.5 Nov 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants