Skip to content

Collaging on Internal Representations: An Intuitive Approach for Semantic Transfiguration

License

Notifications You must be signed in to change notification settings

quolc/neural-collage

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

39 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Neural Collage

Chainer implementation of our novel CNN-based image editing method that allows the user to change the semantic information of an image over a user-specified region:

Collaging on Internal Representations: An Intuitive Approach for Semantic Transfiguration

Ryohei Suzuki1,2*, Masanori Koyama2, Takeru Miyato2, Taizan Yonetsuji2
1The University of Tokyo, 2Preferred Networks, Inc.,
*This work was done when the author was at Preferred Networks, Inc.
arXiv:1811.10153

Colab examples

Collage-based image editing

Image editing pipeline

Spatial class-translation

Image editing results using spatial class-translation:

Spatially-modulated image generation results:

    

Semantic transplantation

Image editing results using semantic transplantation:

Spatial class + semantic transfiguration

Left: example image editing result with spatial class-translation + semantic transplantation.
Right: image generation with spatial class (horizontal) + semantic (vertical) morphing.

    

Setup

Prerequisites

  • Python >= 3.6

Install required python libraries:

pip install -r requirements.txt

Pre-trained models

If you want to use pretrained models for the image generation, please download the model from link and set the snapshot argument to the path to the downloaded pretrained model file (.npz).

Web-based demos

Spatial class-translation

# launch server on localhost:5000
python demo_spatial_translation.py \
--config ./configs/sn_projection_dog_and_cat_256_scbn.yml \
--gen_model ./sn_projection_dog_and_cat_256/ResNetGenerator_450000.npz \
--gpu 0

Semantic transplantation

# launch server on localhost:5000
python demo_feature_blending.py \
--config ./configs/sn_projection_dog_and_cat_256_scbn.yml \
--gen_model ./sn_projection_dog_and_cat_256/ResNetGenerator_450000.npz \
--gpu 0

Advanced Features and Training

Please go to this link.

About

Collaging on Internal Representations: An Intuitive Approach for Semantic Transfiguration

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published