Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

merge latest #1

Merged
merged 76 commits into from
Apr 26, 2019
Merged

merge latest #1

merged 76 commits into from
Apr 26, 2019

Commits on Aug 30, 2018

  1. Update snakecase to camelcase for certain variables. 

    PiperOrigin-RevId: 210592195
    @smoitra-g @psc-g
    smoitra-g authored and psc-g committed Aug 30, 2018
    Configuration menu
    Copy the full SHA
    c154865 View commit details
    Browse the repository at this point in the history

  2. Fix docstring, as it is not returning is_episode_over. 

    PiperOrigin-RevId: 210983763
    @psc-g
    psc-g committed Aug 30, 2018
    Configuration menu
    Copy the full SHA
    37fa491 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2018

  1. BEGIN: Add "apt-get update" to install instructions. 

    PiperOrigin-RevId: 211382454
    @smoitra-g
    smoitra-g committed Sep 3, 2018
    Configuration menu
    Copy the full SHA
    e97a3cd View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2018

  1. Add a .gitignore file. 

    PiperOrigin-RevId: 211460953
    @psc-g
    psc-g committed Sep 4, 2018
    Configuration menu
    Copy the full SHA
    98ef0f5 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2018

  1. Add Machado et al. reference to main README, and update some referenc… 

    …es to their published links.

PiperOrigin-RevId: 211953206
    @psc-g
    psc-g committed Sep 7, 2018
    Configuration menu
    Copy the full SHA
    2664814 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2018

  1. Update install instructions for tensorflow with GPU support. 

    PiperOrigin-RevId: 212782732
    @psc-g
    psc-g committed Sep 13, 2018
    Configuration menu
    Copy the full SHA
    46222d7 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2018

  1. Add support for in-iteration Tensorboard reporting. 

    PiperOrigin-RevId: 213423885
    @psc-g
    psc-g committed Sep 18, 2018
    Configuration menu
    Copy the full SHA
    fb3f376 View commit details
    Browse the repository at this point in the history

  2. Add a What's New section to the main README and include the latest in… 

    …-iteration change.

PiperOrigin-RevId: 213449527
    @psc-g
    psc-g committed Sep 18, 2018
    Configuration menu
    Copy the full SHA
    a59d5d6 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2018

  1. Update utils.py 

    Typo
    @tlwillke
    tlwillke authored Sep 28, 2018
    Configuration menu
    Copy the full SHA
    11b435f View commit details
    Browse the repository at this point in the history

  2. Removed redundant target q-value computation and added option for dou… 

    …ble dqn.

PiperOrigin-RevId: 213507746
    @psc-g
    Dopamine Team authored and psc-g committed Sep 28, 2018
    Configuration menu
    Copy the full SHA
    7d7828d View commit details
    Browse the repository at this point in the history

  3. Add visibility for IQN double-DQN functionality to the README. 

    PiperOrigin-RevId: 213597916
    @psc-g
    psc-g committed Sep 28, 2018
    Configuration menu
    Copy the full SHA
    70b81a1 View commit details
    Browse the repository at this point in the history

  4. Observation shape can be a tuple, Circular memory buffer's logging sh… 

    …ould allow for this.

PiperOrigin-RevId: 213686185
    @psc-g
    Dopamine Team authored and psc-g committed Sep 28, 2018
    Configuration menu
    Copy the full SHA
    94e1850 View commit details
    Browse the repository at this point in the history

  5. Add explicit unit tests for get_range(). 

    PiperOrigin-RevId: 213818042
    @psc-g
    psc-g committed Sep 28, 2018
    Configuration menu
    Copy the full SHA
    aaf334b View commit details
    Browse the repository at this point in the history

  6. Fixed quantile reshaping bug and set terminal_on_life_loss = True f… 

    …or the ICML gin config file.

PiperOrigin-RevId: 214080101
    @psc-g
    Dopamine Team authored and psc-g committed Sep 28, 2018
    Configuration menu
    Copy the full SHA
    2de70a4 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2018

  1. Update JSON files, links and colab utils to use the IQN runs after th… 

    …e bug-fix.

PiperOrigin-RevId: 217402619
    @psc-g
    psc-g committed Oct 16, 2018
    Configuration menu
    Copy the full SHA
    29661e3 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2018

  1. Update colabs to use new IQN data. 

    PiperOrigin-RevId: 217561622
    @psc-g
    psc-g committed Oct 17, 2018
    Configuration menu
    Copy the full SHA
    5259379 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2018

  1. update the assertion clause in Runner 

    PiperOrigin-RevId: 218741713
    @psc-g
    Dopamine Team authored and psc-g committed Oct 29, 2018
    Configuration menu
    Copy the full SHA
    38b1fd2 View commit details
    Browse the repository at this point in the history

  2. Provide the graph definition to Tensorboard. 

    PiperOrigin-RevId: 219163382
    @psc-g
    psc-g committed Oct 29, 2018
    Configuration menu
    Copy the full SHA
    4be166d View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2018

  1. Add download links for each of the individual checkpoints, to avoid h… 

    …aving to download all of them as a single .tar.gz file.

PiperOrigin-RevId: 219607104
    @psc-g
    psc-g committed Nov 1, 2018
    Configuration menu
    Copy the full SHA
    6665652 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2018

  1. Merge branch 'master' into patch-1

    @psc-g
    psc-g authored Nov 2, 2018
    Configuration menu
    Copy the full SHA
    e5d8465 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2018

  1. Merge pull request #32 from tlwillke/patch-1 

    Fix typo in utils.py
    @psc-g
    psc-g authored Nov 9, 2018
    Configuration menu
    Copy the full SHA
    7a58561 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2018

  1. internal 

    PiperOrigin-RevId: 219810600
    @psc-g
    Dopamine Team authored and psc-g committed Nov 13, 2018
    Configuration menu
    Copy the full SHA
    688a7e2 View commit details
    Browse the repository at this point in the history

  2. internal merge of PR #32 

    PiperOrigin-RevId: 220686551
    @tlwillke @psc-g
    tlwillke authored and psc-g committed Nov 13, 2018
    Configuration menu
    Copy the full SHA
    7ca1f2d View commit details
    Browse the repository at this point in the history

  3. internal 

    PiperOrigin-RevId: 221077222
    @psc-g
    Dopamine Team authored and psc-g committed Nov 13, 2018
    Configuration menu
    Copy the full SHA
    bea54bb View commit details
    Browse the repository at this point in the history

  4. Generalize agent constructors to accept different specifications of o… 

    …bservation shape/type, and the stack size.

PiperOrigin-RevId: 221138027
    @psc-g
    Dopamine Team authored and psc-g committed Nov 13, 2018
    Configuration menu
    Copy the full SHA
    8d6cbb3 View commit details
    Browse the repository at this point in the history

  5. Add a thin wrapper around Gym environments to make them conformant to… 

    … the API expected by Dopamine.

PiperOrigin-RevId: 221243592
    @psc-g
    Dopamine Team authored and psc-g committed Nov 13, 2018
    Configuration menu
    Copy the full SHA
    292f797 View commit details
    Browse the repository at this point in the history

  6. internal. 

    PiperOrigin-RevId: 221268516
    @psc-g
    Dopamine Team authored and psc-g committed Nov 13, 2018
    Configuration menu
    Copy the full SHA
    4eec4a8 View commit details
    Browse the repository at this point in the history

  7. internal. 

    PiperOrigin-RevId: 221311567
    @psc-g
    Dopamine Team authored and psc-g committed Nov 13, 2018
    Configuration menu
    Copy the full SHA
    32d23e1 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2018

  1. Remove TF dependency in setup and increase version number. 

    PiperOrigin-RevId: 221502319
    @psc-g
    Dopamine Team authored and psc-g committed Nov 14, 2018
    Configuration menu
    Copy the full SHA
    1f54124 View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2018

  1. Generalize observation_shape specification, to allow for non-square s… 

    …hapes.

PiperOrigin-RevId: 221709966
    @psc-g
    Dopamine Team authored and psc-g committed Nov 16, 2018
    Configuration menu
    Copy the full SHA
    0d9afcf View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2018

  1. Increase version number for a new PyPi release. 

    PiperOrigin-RevId: 222120861
    @psc-g
    psc-g committed Nov 19, 2018
    Configuration menu
    Copy the full SHA
    06d64fc View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2018

  1. Fix the handling of custom observation shapes and types. This include… 

    …s enforcing shapes to be passed in as tuples.

PiperOrigin-RevId: 222560771
    @psc-g
    psc-g committed Nov 22, 2018
    Configuration menu
    Copy the full SHA
    bc66570 View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2019

  1. Add bibtex entry for white paper.

    @psc-g
    psc-g authored Jan 4, 2019
    Configuration menu
    Copy the full SHA
    b4341f9 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2019

  1. Refactor atari.run_experiment into common.run_experiment to enable su… 

    …pport for non-Atari Gym environments.

PiperOrigin-RevId: 224811437
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    584dd30 View commit details
    Browse the repository at this point in the history

  2. Refactor atari.run_experiment into common.run_experiment to enable su… 

    …pport for non-Atari Gym environments.

PiperOrigin-RevId: 224819061
    @psc-g
    Dopamine Team authored and psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    163a9c0 View commit details
    Browse the repository at this point in the history

  3. Adding missing life loss parameter to some baseline configuration fil… 

    …es. Changing the IQN learning rate and Adam epsilon in the "apples to apples" comparison configuration as the algorithm seems unstable with the smaller Rainbow epsilon.

PiperOrigin-RevId: 225007621
    @psc-g
    Marc G. Bellemare authored and psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    d5f7db7 View commit details
    Browse the repository at this point in the history

  4. Refactor atari.run_experiment into common.run_experiment to enable su… 

    …pport for non-Atari Gym environments.

PiperOrigin-RevId: 225040877
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    31799e9 View commit details
    Browse the repository at this point in the history

  5. Fix observation_dtype argument in WrappedPrioritizedReplayBuffer whic… 

    …h wasn't getting passed along.

PiperOrigin-RevId: 225234742
    @psc-g
    Dopamine Team authored and psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    ce1086e View commit details
    Browse the repository at this point in the history

  6. Refactor network specifications into discrete_domains.atari_lib and a… 

    …dd an initial network for Cartpole DQN.

PiperOrigin-RevId: 225351561
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    a53ff8a View commit details
    Browse the repository at this point in the history

  7. Fix typo in dqn_cartpole.gin. 

    PiperOrigin-RevId: 225364589
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    09080db View commit details
    Browse the repository at this point in the history

  8. Make the identity_epsilon function conform to the existing linearly_d… 

    …ecaying epsilon function.

PiperOrigin-RevId: 225414171
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    4b8a6f6 View commit details
    Browse the repository at this point in the history

  9. Add a simple config for Acrobot with DQN. 

    PiperOrigin-RevId: 225568024
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    6f6a638 View commit details
    Browse the repository at this point in the history

  10. Make acrobot min/max vals numpy arrays. 

    PiperOrigin-RevId: 225575570
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    d69f3c9 View commit details
    Browse the repository at this point in the history

  11. Add BibTeX entry for Dopamine white paper. 

    PiperOrigin-RevId: 227857505
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    d4acd20 View commit details
    Browse the repository at this point in the history

  12. Refactor dopamine circular and prioritized replay buffer to support v… 

    …ectorized action and rewards and also add next action to storage.

PiperOrigin-RevId: 227942879
    @jingconan @psc-g
    jingconan authored and psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    cb1e248 View commit details
    Browse the repository at this point in the history

  13. Some minor fixes for prioritized_replay_buffer.py and related files. 

    PiperOrigin-RevId: 228429755
    @psc-g
    Dopamine Team authored and psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    da6bbc7 View commit details
    Browse the repository at this point in the history

  14. This CL ensures that RainbowAgent.__init__ always calls the right fun… 

    …ction with an explicit call to dqn_agent.DQNAgent.__init__

PiperOrigin-RevId: 228805979
    @psc-g
    Dopamine Team authored and psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    87b5eef View commit details
    Browse the repository at this point in the history

  15. Add rainbow support for cartpole and acrobot. 

    PiperOrigin-RevId: 230914146
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    206c57d View commit details
    Browse the repository at this point in the history

  16. Finish migration of Dopamine to using discrete_domains instead of atari. 

    PiperOrigin-RevId: 231432169
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    034e0ab View commit details
    Browse the repository at this point in the history

  17. Remove useless lines. 

    PiperOrigin-RevId: 231517673
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    eb4b102 View commit details
    Browse the repository at this point in the history

  18. Fix the the dopamine circular_replay_buffer for case in which reward … 

    …is a vector

PiperOrigin-RevId: 231540267
    @jingconan @psc-g
    jingconan authored and psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    b658c5e View commit details
    Browse the repository at this point in the history

  19. Set reasonable default settings for rainbow on cartpole and acrobot. 

    PiperOrigin-RevId: 231598699
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    f3653b5 View commit details
    Browse the repository at this point in the history

  20. Tidy up documentation for Dopamine 2.0. 

    PiperOrigin-RevId: 231661808
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    19b0348 View commit details
    Browse the repository at this point in the history

  21. Update api_docs for Dopamine 2.0. 

    PiperOrigin-RevId: 231771911
    @psc-g
    psc-g committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    11dee79 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2019

  1. Add agents using Fourier basis and linear Q function approximation 

    PiperOrigin-RevId: 232039653
    @psc-g
    Dopamine Team authored and psc-g committed Feb 2, 2019
    Configuration menu
    Copy the full SHA
    202fa9e View commit details
    Browse the repository at this point in the history

  2. Adding a colab demonstrating how to train agents on Cartpole. 

    PiperOrigin-RevId: 232073385
    @psc-g
    psc-g committed Feb 2, 2019
    Configuration menu
    Copy the full SHA
    9dbd94a View commit details
    Browse the repository at this point in the history

  3. Remove unused debug_mode flag and update documentation. 

    PiperOrigin-RevId: 232074103
    @psc-g
    psc-g committed Feb 2, 2019
    Configuration menu
    Copy the full SHA
    753a243 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2019

  1. removed the comment regarding the time limit, as it's wrong 

    PiperOrigin-RevId: 232499569
    @psc-g
    Dopamine Team authored and psc-g committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    e873d7c View commit details
    Browse the repository at this point in the history

  2. Remove atari-py and cmake installs for cartpole colab. 

    PiperOrigin-RevId: 232524026
    @psc-g
    psc-g committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    5fad7e9 View commit details
    Browse the repository at this point in the history

  3. Make default number of checkpoints saved consistent between dqn_agent… 

    … and checkpointer.

PiperOrigin-RevId: 232544723
    @psc-g
    psc-g committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    53d3450 View commit details
    Browse the repository at this point in the history

  4. Fix docstring 

    PiperOrigin-RevId: 232547295
    @smoitra-g @psc-g
    smoitra-g authored and psc-g committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    4799c3a View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2019

  1. Update Cartpole configuration files and colab to match default Gym ma… 

    …x episode length.

PiperOrigin-RevId: 232791515
    @psc-g
    psc-g committed Feb 7, 2019
    Configuration menu
    Copy the full SHA
    79b024f View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2019

  1. Update the agents colab to the new Dopamine 2.0 interface. 

    PiperOrigin-RevId: 233077734
    @psc-g
    psc-g committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    d7049fe View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2019

  1. internal. 

    PiperOrigin-RevId: 233593997
    @psc-g
    psc-g committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    349261c View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2019

  1. Update documentation to support Python 3.6. 

    PiperOrigin-RevId: 234139600
    @psc-g
    psc-g committed Feb 15, 2019
    Configuration menu
    Copy the full SHA
    0ce0709 View commit details
    Browse the repository at this point in the history

  2. Update create_agent docstring. 

    PiperOrigin-RevId: 234151608
    @smoitra-g @psc-g
    smoitra-g authored and psc-g committed Feb 15, 2019
    Configuration menu
    Copy the full SHA
    711e188 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2019

  1. Add close method to AtariPreprocessing. 

    PiperOrigin-RevId: 235068759
    @psc-g
    Dopamine Team authored and psc-g committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    00a5b32 View commit details
    Browse the repository at this point in the history

  2. Adding a comment to clarify why Rainbow uses the same data structure … 

    …for both replay schemes

PiperOrigin-RevId: 237079830
    @psc-g
    Marc G. Bellemare authored and psc-g committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    33c4f61 View commit details
    Browse the repository at this point in the history

  3. Update DQNAgent's eval_mode. Allow setting it in constructor and remo… 

    …ve it from the bundle.

PiperOrigin-RevId: 238107489
    @psc-g
    Dopamine Team authored and psc-g committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    2639e7d View commit details
    Browse the repository at this point in the history

  4. Remove unused dependency on tf.slim in agents 

    PiperOrigin-RevId: 238608544
    @psc-g
    Marc G. Bellemare authored and psc-g committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    75fc33a View commit details
    Browse the repository at this point in the history

  5. Changing CartPole/Acrobot observation type to match what is returned … 

    …by Gym

PiperOrigin-RevId: 241769847
    @psc-g
    Marc G. Bellemare authored and psc-g committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    f6dc339 View commit details
    Browse the repository at this point in the history

  6. Fix bug whereby batches of size larger than 1000 would never get 

    PiperOrigin-RevId: 242851617
    @psc-g
    Marc G. Bellemare authored and psc-g committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    2435916 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2019

  1. Propagating observation dtype to the replay buffer in RainbowAgent si… 

    …milarly to DQNAgent.

PiperOrigin-RevId: 243108947
    @psc-g
    Dopamine Team authored and psc-g committed Apr 16, 2019
    Configuration menu
    Copy the full SHA
    2ce82ea View commit details
    Browse the repository at this point in the history

  2. Fix ValueError issue when reloading baselines and increase minor vers… 

    …ion number for new release.

PiperOrigin-RevId: 243803145
    @psc-g
    psc-g committed Apr 16, 2019
    Configuration menu
    Copy the full SHA
    834c4b8 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2019

  1. Fix remaining instance of ValueError when numerics not cast as float6… 

    …4s, and increase minor version number for new release.

PiperOrigin-RevId: 243966695
    @psc-g
    psc-g committed Apr 17, 2019
    Configuration menu
    Copy the full SHA
    01abb82 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2019

  1. Only load training runs, as the colab uses TrainRunner, which doesn't… 

    … generate eval curves.

PiperOrigin-RevId: 244255414
    @psc-g
    psc-g committed Apr 23, 2019
    Configuration menu
    Copy the full SHA
    826f172 View commit details
    Browse the repository at this point in the history