This package unites the PyGame Framework with the Open AI Gym Framework to build a custom environment for training reinforcement learning models. Specifically, it implements the custom-built "Kuiper Escape" game.
The objective of the game is to live as long as you can, while avoiding the asteroids in the Kuiper Belt. The player starts centered in the screen, an can move in any direction using the keyboard arrows. Asteroids are generated with random sizes, speeds, and starting locations. The game ends once all player lives have expired.
The user has the following discrete actions:
- 0: Don't move
- 1: Up
- 2: Right
- 3: Down
- 4: Left
- 5: Up/Right Diagnal (optional)
- 6: Right/Down Diagnal (optional)
- 7: Down/Left Diagnal (optional)
- 8: Left/Up Diagnal (optional)
Note: The diagonal directions are optional in order to conveniently enable smaller action spaces (e.g. Discrete(5)
vs. Discrete(9)
)
The state/observation is a "virtual" lidar system. It sends off virtual beams of light in all directions to gather an array of points describing the distance and characteristics of nearby objects. The size of the lidar array and resulting observation/state space is configurable when the environment is initialized
The observation data (for each beam in the lidar array):
- Distance (i.e. radial distance from player to terminating point of lidar beam)
- Collision detection
- 0 if terminated at edge of screen, or at max radius distance
- 1 if collided with a rock
Example Visualizations of State
Note: The yellow dots (1 collide state) represent contact with a rock, the green dots (0 collide state) represent contact with wall or open space.
The environment will provide the following rewards:
- Reward of 1 for each step without losing life
- No reward is given if the player is in the corners of the screen
First it is recommended to setup a virtual environment:
python -m venv .env
source .env/bin/activate
Update pip and wheel to ensure a smooth installation process:
pip install --upgrade pip
pip install --upgrade wheel
Finally, install package locally with pip:
git clone https://github.com/jdegregorio/gym-kuiper-escape.git
pip install -e gym-kuiper-escape
To uninstall, use the following:
pip uninstall gym-kuiper-escape
- Python Version: 3.8.10
- Operating System: Ubuntu 20.04.3 LTS
To play the game without installing the package, run the following at the command line:
git clone https://github.com/jdegregorio/gym-kuiper-escape.git
cd gym-kuiper-escape
python -m venv .env
source .env/bin/activate
pip install -r requirements.txt
python gym_kuiper_escape/envs/env_base.py
To play after installing the package, enter the following python commands:
import gym
import gym_kuiper_escape
env = gym.make('kuiper-escape-base-v0', mode='human'))
env.game.play()
See this gym in action by checking out the GitHub repository using this gym to train an agent using reinforcement learning.
Kuiper Escape Reinforcement Learning Repo
Open AI Gym provides a standardized framework for training reinforcement learning models. The framework has numerous built-in environments (often games) for experimentation, but also enables users to define their own custom environments.
- Open AI Gym Documentation
- Creating Customer Environments
- Example Custom Environment
- Core Open AI Gym Clases
PyGame is a framework for developing games within python.
This tutorial is a great primer for getting started.