Real-Time High-Resolution Background Matting. model requires capturing an additional background image and produces state-of-the-art matting results at 4K 30fps and HD 60fps .
- HD videos (by Sengupta et al.) (Our model is more robust on HD footage)
- 4K videos and images
We provide several scripts in this repo for you to experiment with our model. More detailed instructions are included in the files.
inference_images.py
: Perform matting on a directory of images.inference_video.py
: Perform matting on a video.inference_webcam.py
: An interactive matting demo using your webcam.
Additionally, you can try our notebooks in Google Colab for performing matting on images and videos.
You can run our model using PyTorch, TorchScript, TensorFlow, and ONNX. For detail about using our model, please check out the Usage / Documentation page.
Configure data_path.pth
to point to your dataset. The original paper uses train_base.pth
to train only the base model till convergence then use train_refine.pth
to train the entire network end-to-end. More details are specified in the paper.