Skip to content

Latest commit

 

History

History
81 lines (54 loc) · 1.68 KB

README.md

File metadata and controls

81 lines (54 loc) · 1.68 KB

Decision transformer experiments

Setup

Step 1: set up conda environment

conda create -n dt-atari python=3.7.9
conda activate dt-atari

conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch

# confirm that torch has been installed correctly
python -c 'import torch; print(torch.cuda.is_available()); a = torch.zeros(5); a.to("cuda:0"); print(a)'
python -c 'import torch; '

pip install tqdm
pip install atari-py
pip install opencv-python
pip install blosc

git clone https://github.com/google/dopamine
pip install absl-py
pip install gin-config
pip install tensorflow==1.15
pip install protobuf==3.20.3

python -m atari_py.import_roms Atari-2600-VCS-ROM-Collection/ROMS/

Step 2: download dataset

mkdir dataset

pip install gsutil
gsutil -m cp -R gs://atari-replay-datasets/dqn/Seaquest dataset

Running the experiments

Training

Linear:

sh scripts/train/exps_linear.sh

Fourier:

for fourier_frequencies in 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
do
    sh scripts/train/exps_fourier.sh $fourier_frequencies
done

Evaluation

Once the linear and Fourier models finish training, we can run this script which will compute the smoothness values for the saved multinomials.

python scripts/eval/compute_smoothness_dict.py

Then, we can graph the metrics (normalized returns, smoothness) for all of our frequencies.

python scripts/eval/atari_graph_varying_freqs.py

Next, we can graph the learned PMFs side by side. Right now, it'll generate a graph for every saved multinomial in the test split for Fourier-8 and Linear.

python scripts/eval/atari_graph_PMFs.py