Skip to content

Latest commit

 

History

History
143 lines (94 loc) · 5.45 KB

README.md

File metadata and controls

143 lines (94 loc) · 5.45 KB
Raw

ISSAFE & EDCNet

PWC
PWC
PWC
PWC

Introduction

This is the implementation of our papers below, including the code and dataset (DADA-seg).

ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data, IROS 2021, [paper].

Exploring Event-Driven Dynamic Context for Accident Scene Segmentation, T-ITS 2021, [paper].

Demo

issafe_demo

Updates

  • [04/12/2022] Generated event data release.
  • [12/27/2021] Initial release.

Installation

The requirements are listed in the requirement.txt file. To create your own environment, an example is:

conda create -n issafe python=3.7
conda activate issafe
cd /path/to/ISSAFE
pip install -r requirement.txt

Datasets

For the basic setting of this work, please prepare datasets of Cityscapes, and DADA-seg.

Our proposed DADA-seg dataset is a subset from DADA-2000.

The DADA-seg dataset and event data of Cityscapes are now available:

Dataset Image & label Event Label-only
DADA-seg (val) 46.8MB [Baidu Drive] 401.2MB [Baidu Drive] 2.7MB [Google Drive]
Cityscapes (train + val) - 7.5GB [Baidu Drive] -

Note: The event data is generated by EventGAN. The anchor and its previous frames are needed.

A structure of dataset should be:

dataset
├── Cityscapes
│   ├── event
│   │   ├── train
│   │   │   ├─aachen
│   │   │   │  ├─aachen_000000_000019_gtFine_event.npz	# event volume
│   │   └── val
│   ├── gtFine
│   │   ├── train
│   │   └── val
│   ├─leftImg8bit_prev # for event synthesic
│   │   ├─train
│   │   │  ├─aachen
│   │   │  │  ├─aachen_000000_000019_leftImg8bit_prev.png
│   │   └─val
│   ├── leftImg8bit
│   │   ├── train
│   └── └── val
└── DADA_seg
   ├── dof
   │   └── val
   ├── event
   │   └── val
   ├── gtFine
   │   └── val
   └── leftImg8bit
       ├── train
       └── val

(optional) other dataset sources used in EDCNet: BDD3K, KITTI-360, ApolloScape.

(optional) other modalities: dense optical flow.

Training

The model of EDCNet can be found at models/edcnet.py.

Before run the training script, please modify your own path configurations at mypath.py.

The training configurations can be adjusted at train.py.

An example of training is python train.py

Evaluation

The evaluation configurations can be adjusted at eval.py.

To achieve the evaluation result of EDCNet in D2S mode with 2 event time bins, the weights can be downloaded in Google Drive.

Put the weight at run/cityscapesevent/test_EDCNet_r18/model_best.pth.

An example of evaluation of the EDCNet at B=2 event time bins is python eval.py.

License

This repository is under the Apache-2.0 license. For commercial use, please contact with the authors.

Citation

If you are interested in this work, please cite the following work:

@INPROCEEDINGS{zhang2021issafe,
  author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
  booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
  title={ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data}, 
  year={2021},
  pages={1132-1139},
  doi={10.1109/IROS51168.2021.9636109}}
  
@ARTICLE{zhang2021edcnet,
  author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
  journal={IEEE Transactions on Intelligent Transportation Systems}, 
  title={Exploring Event-Driven Dynamic Context for Accident Scene Segmentation}, 
  year={2021},
  pages={1-17},
  doi={10.1109/TITS.2021.3134828}}