Solve the Gridworld Problem with Reinforcement Learning

Based on the book --- Reinforcement Learning: An Introduction (2nd ed, 2018) by Sutton and Barto

For the Reinforcement Learning course Assignment 2 (see 'Gridworld Problem 1.pdf') at Memorial University of Newfoundland, Jul. 18, 2024

For Gridworld Problem 2, see https://github.com/zyxsjdy/Solve-the-Gridworld-Problem-with-Reinforcement-Learning-Cont

Environment

python 3.11.9

Packages: numpy, cvxpy, copy

How to run

Use Jupyter Notebook to open the '.ipynb' file, and click Run All to run the file.

'P1.ipynb' for part 1, 'P2.ipynb' for part 2, and 'environment.py' for the gridworld environment.

See explanation for results in 'Report.pdf'.

Included topics

Reinforcement Learning

Gridworld problem

Part 1:

Bellman equation

Bellman optimality equation

Iterative policy evaluation

Value iteration

Policy iteration

Policy improvement

Part 2:

Monte Carlo method

On-policy Monte Carlo ES (exploring starts)

On-policy first-visit Monte Carlo control for ϵ-soft policy

Off-policy Monte Carlo control

Off-policy Monte Carlo prediction (policy evaluation)

References

https://marcinbogdanski.github.io/rl-sketchpad

https://github.com/BY571/Medium_Code_Examples