Semantic Image Inpainting With Deep Generative Models and image restoration with GANS

[Inpainting CVPR2017] [Image Restore ICASSP2018]

Tensorflow implementation for semantic image inpainting:

Semantic Image Inpainting With Deep Generative Models

Raymond A. Yeh*, Teck Yian Lim*, Chen Chen, Alexander G. Schwing, Mark Hasegawa-Johnson, Minh N. Do

In CVPR 2017

* indicating equal contributions.

Overview

Implementation of proposed cost function and backpropogation to input.

In this code release, we load a pretrained DCGAN model, and apply our proposed objective function for the task of image completion.

Notes

To reproduce the CVPR2017 work, run the inpainting example.

Dependencies

• Tensorflow >= 1.0
• scipy + PIL/pillow (image io)
• pyamg (for Poisson blending)

Tested to work with both Python 2.7 and Python 3.5

Files

• src/model.py - Main implementation
• src/inpaint.py - command line application
• src/external - external code used. Citations in code
• graphs/dcgan-100.pb - frozen pretrained DCGAN with 100-dimension latent space

Weights

Git doesn't work nicely with large binary files. Please download our weights from here, trained on the CelebA dataset.

Alternatively, train your own GAN using your dataset. Conversion from checkpoint to Tensorflow ProtoBuf format can be done with this script

Running

Generate multiple candidates for completion:

python src/inpaint.py --model_file graphs/dcgan-100.pb \
    --maskType center --in_image testimages/face1.png \
    --nIter 1000 --blend

Generate completions for multiple input images:

python src/inpaint.py --model_file graphs/dcgan-100.pb \
    --maskType center --inDir testimages \
    --nIter 1000 --blend

Citation

@inproceedings{
    yeh2017semantic,
    title={Semantic Image Inpainting with Deep Generative Models},
    author={Yeh$^\ast$, Raymond A. and Chen$^\ast$, Chen and Lim, Teck Yian and Schwing Alexander G. and Hasegawa-Johnson, Mark and Do, Minh N.},
    booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
    year={2017},
    note = {$^\ast$ equal contribution},
}