Skip to content

Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture (AAAI-18)

Notifications You must be signed in to change notification settings

mil-tokyo/FTGAN

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Flow and Texture GAN (FTGAN)

Publication

Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture (arxiv) Katsunori Ohnishi*, Shohei Yamamoto*, Yoshitaka Ushiku, Tatsuya Harada.
In AAAI, 2018 * indicates equal contribution.

Pipeline

Requirements

Python 2.7

Chainer 3.1.0+

In addition, please pip install the following packages:

  • matplotlib
  • pillow
  • scipy

Demo

You can test FTGAN using pretrained model as follows:

cd src/joint_learning
python demo.py --gpu=0

If you failed to run because of model loading, please install git-lfs and run git lfs clone git@github.com:mil-tokyo/FTGAN.git, or please download models from Google Drive.

Train

Preprocess

  1. Download the dataset(Penn Action)and extract optical flow.

  2. Resize all frames (76*76) and convert to npy file.

  3. Setup dataset directory as follows.

    PennAction/
        npy_76/
            0001.npy
            0002.npy
            ...
            2326.npy
        npy_flow_76/
            0001.npy
            0002.npy
            ...
            2326.npy

Train FlowGAN

cd src/FlowGAN
python train.py --gpu=0 --root '/path/to/dataset/'

Train TextureGAN

cd src/TextureGAN
python train.py --gpu=0 --root '/path/to/dataset/'

Joint learning

cd src/joint_learning
python train.py --gpu=0 --root '/path/to/dataset/'

Example of Results

TextureGAN (from GT Flow and ) FTGAN (from and )

Citing FTGAN

If you find FTGAN useful in your research, please consider citing:

@inproceedings{ohnishi2018ftgan,
  title={Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture},
  author={Ohnishi, Katsunori and Yamamoto, Shohei and Ushiku, Yoshitaka and Harada, Tatsuya},
  booktitle={AAAI},
  year={2018}
}

About

Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture (AAAI-18)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages