iFusion Threestudio

Project Page | Paper

This code is forked from threestudio as an extension of iFusion for pose-free reconstruction.

iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse Views
Chin-Hsuan Wu, Yen-Chun Chen, Bolivar Solarte, Lu Yuan, Min Sun

Installation

git clone https://github.com/chinhsuanwu/ifusion-threestudio.git
cd ifusion-threestudio

Skip the following if you have already installed the environment for threestudio. Please refer to installation.md for detailed information.

pip install -r requirements.txt

Download or link Zero123-XL under load/zero123

cd load/zero123 && wget https://zero123.cs.columbia.edu/assets/zero123-xl.ckpt

⚠️ You must have an NVIDIA GPU with at least 20GB VRAM.

Usage

Run Zero123-SDS by specifing the path to transform.json obtained from iFusion

python launch.py --config configs/zero123-ifusion.yaml --train --gpu 0 data.transform_fp=path_to_transform.json

Run Magic123 with additional text prompt

python launch.py --config configs/magic123-ifusion-coarse-sd.yaml --train --gpu 0 data.transform_fp=path_to_transform.json system.prompt_processor.prompt="text"

Find out more examples at run.sh.

Citation

@article{wu2023ifusion,
  author = {Wu, Chin-Hsuan and Chen, Yen-Chun, Solarte, Bolivar and Yuan, Lu and Sun, Min},
  title = {iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse Views},
  journal = {arXiv preprint arXiv:2312.17250},
  year = {2023}
}

Acknowledgements

This code is built upon threestudio. We sincerely thank all the contributors for their efforts!