UnOVOST: Unsupervised Offline Video Object Segmentation and Tracking

Overview

Paper

Jonathon Luiten*, Idil Esen Zulfikar*, Bastian Leibe, "UnOVOST: Unsupervised Offline Video Object Segmentation and Tracking", WACV 2020

Requirements

This code is written in Python 3.6 and the following modules are required:

• cv2
• PIL
• pycocotools
• scipy
• yaml
• pytorch

Usage

Before running this code, prepare JSON files that contain mask, optical flow vector and ReID vector for each object proposal in a sequence.

• To generate masks for object proposals, we use this Mask R-CNN implementation.
• To extract Optical flow vector and ReID vector,we use ReID network and Optical-flow network in PremVOS.

An example json file can be downloaded from this link.

Afterwards, check your directory with JSON files to match this expected format:

proposals/
    val/
        bike-packing/
            00000.json
            ...
            00079.json

Finally, run the code:

python main.py --proposal_dir ../proposals/ --output_dir ../results/ --config  ../configs/unovost.yaml  

Citation

@inproceedings{luiten2020unovost,
  title={UnOVOST: Unsupervised Offline Video Object Segmentation and Tracking},
  author={Luiten, Jonathon and Zulfikar, Idil Esen and Leibe, Bastian},
  booktitle={Proceedings of the IEEE Winter Conference on Applications in Computer Vision},
  year={2020}
}

Contact

If you encounter any problems within the code or have any questions, please get in touch with Idil Esen Zulfikar (zuelfikar at vision dot rwth-aachen dot de) or Jonathon Luiten (luiten at vision dot rwth-aachen dot de).