By Prarthana Bhattacharyya, Chengjie Huang and Krzysztof Czarnecki.
We provide code support for the paper: SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving.
Our code is based on Lane-GCN, which is a clean open-sourced project for motion forecasting methods.
In this study, we report the first systematic exploration and assessment of incorporating self-supervision into motion forecasting. We first propose to investigate four novel self-supervised learning tasks for motion forecasting with theoretical rationale and quantitative and qualitative comparisons on the challenging large-scale Argoverse dataset. Secondly, we point out that our auxiliary SSL-based learning setup not only outperforms forecasting methods which use transformers, complicated fusion mechanisms and sophisticated online dense goal candidate optimization algorithms in terms of performance accuracy, but also has low inference time and architectural complexity. Lastly, we conduct several experiments to understand why SSL improves motion forecasting.
For this repository, the expected performance on Argoverse 1 validation set is:
Models | minADE | minFDE | MR |
---|---|---|---|
Baseline | 0.73 | 1.12 | 11.07 |
Lane-Masking | 0.70 | 1.02 | 8.82 |
Distance to Intersection | 0.71 | 1.04 | 8.93 |
Maneuver Classification | 0.72 | 1.05 | 9.36 |
Success/Failure Classification | 0.70 | 1.01 | 8.59 |
We provide the pretrained checkpoints for the proposed above-mentioned models in checkpoints/.
To evaluate the prediction performance, run:
python test.py -m lanegcn --weight=lane_masking.ckpt --split=val
1. Clone this repository:
git clone https://github.com/AutoVision-cloud/SSL-Lanes
cd SSL-Lanes
2. Following is an example of create environment from scratch with anaconda, you can use pip as well:
conda create --name lanegcn python=3.7
conda activate lanegcn
conda install pytorch==1.5.1 torchvision cudatoolkit=10.2 -c pytorch # pytorch=1.5.1 when the code is release
# install argoverse api
pip install git+https://github.com/argoai/argoverse-api.git
# install others dependancy
pip install scikit-image IPython tqdm ipdb
3. Install Horovod and mpi4py
for distributed training. Horovod is more efficient than nn.DataParallel
for mulit-gpu training and easier to use than nn.DistributedDataParallel
. Before install horovod, make sure you have openmpi installed (sudo apt-get install -y openmpi-bin
).
pip install mpi4py
# install horovod with GPU support, this may take a while
HOROVOD_GPU_OPERATIONS=NCCL pip install horovod==0.19.4
# if you have only SINGLE GPU, install for code-compatibility
pip install horovod
if you have any issues regarding horovod, please refer to horovod github
4. Download Argoverse Motion Forecasting Dataset v1.1. After downloading and extracting the tar.gz files, the dataset directory should be organized as follows:
/path/to/dataset_root/
├── train/
| └── data/
| ├── 1.csv
| ├── 2.csv
| ├── ...
└── val/
└── data/
├── 1.csv
├── 2.csv
├── ...
5. Download the processed data.
bash get_data.sh
Copy the contents of the pretext task into ssl_pretext_tasks/LaneGCN
as follows:
cp -r ssl_pretext_tasks/lane_masking/final_task_masking/ ssl_pretext_tasks/LaneGCN/
# single node with 4 gpus
horovodrun -np 4 -H localhost:4 python /path/to/train.py -m lanegcn
If you find this project useful in your research, please consider starring the repository and citing:
@misc{bhattacharyya2022ssllanes,
title={SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving},
author={Prarthana Bhattacharyya, Chengjie Huang and Krzysztof Czarnecki},
year={2022},
archivePrefix={arXiv},
primaryClass={cs.CV}
}