A paper list of 3D photography and cinemagraph.
This list is non-exhaustive. Feel free to pull requests or create issues to add papers.
Following this repo, I use some icons to (imprecisely) differentiate the 3D representations:
- 🍃 Layered Depth Image
- 💎 Mesh
✈️ Multiplane Images- 🚕 Nerf
- ☁️ Point Cloud
- 👾 Voxel
- ☃️ 3D Gaussian Splatting
Novel view synthesis with a single input image based on 3D geometry.
[arXiv 2024]GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warping [paper] [project page] ☃️[CVPR 2024]WonderJourney: Going from Anywhere to Everywhere [paper] [code] [project page] ☁️[CVPR 2024]MultiDiff: Consistent Novel View Synthesis from a Single Image [paper] [project page][arXiv 2023]Novel View Synthesis with View-Dependent Effects from a Single Image [paper] [project page][SA 2023]SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Images [paper] [code] 🍃[TOG 2023]ReShader: View-Dependent Highlights for Single Image View-Synthesis [paper] [code] [project page][NeurIPS 2023]Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision [paper] [code] [project page][ICCV 2023]Diffuse3D: Wide-Angle 3D Photography via Bilateral Diffusion [paper] [code] 🍃[ICCV 2023]LoLep: Single-View View Synthesis with Locally-Learned Planes and Self-Attention Occlusion Inference [paper]✈️ [arXiv 2023]Generative Novel View Synthesis with 3D-Aware Diffusion Models [paper] [project page][CVPR 2023]Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstruction [paper]✈️ [TIP 2023]Single-view Neural Radiance Fields with Depth Teacher [paper] 🚕[ECCV 2022]InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images [paper] [project page][SIGGRAPH 2022]Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images [paper] [code] [project page]✈️ [CVPR 2022]Efficient Geometry-aware 3D Generative Adversarial Networks [paper] [code] [project page][CVPRW 2022]Artistic Style Novel View Synthesis Based on A Single Image [paper] [code] [project page] ☁️[CVPR 2022]3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image [paper] [code] [project page] ☁️[ICCV 2021]Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image [paper] [code] [project page] 💎[ICCV 2021]MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis [paper] [code] [project page]✈️ 🚕[ICCV 2021]PixelSynth: Generating a 3D-Consistent Experience from a Single Image [paper] [code] [project page] ☁️[ICCV 2021]SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting [paper] [project page] 💎[ICCV 2021]Video Autoencoder: self-supervised disentanglement of static 3D structure and motion [paper] [code] [project page] 👾[ICCV 2021]Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image [paper] [code] [project page] 💎[CVPR 2021]Layout-Guided Novel View Synthesis from a Single Indoor Panorama [paper] [dataset][WACV 2021]Adaptive Multiplane Image Generation from a Single Internet Picture [paper]✈️ [CVPR 2020]Single-View View Synthesis with Multiplane Images [paper] [code] [project page]✈️ [CVPR 2020]SynSin: End-to-end View Synthesis from a Single Image [paper] [code] [project page] ☁️[CVPR 2020]3D Photography using Context-aware Layered Depth Inpainting [paper] [code] [project page] 🍃[Trans. Graph. 2020]One Shot 3D Photography [paper] [code] [project page] 🍃 💎[Trans. Graph. 2019]3D Ken Burns Effect from a Single Image [paper] [code] ☁️[ICCV 2019]Monocular Neural Image-based Rendering with Continuous View Control [paper] [code][ECCV 2018]Layer-structured 3D Scene Inference via View Synthesis [paper] [code] [project page] 🍃[SIGGRAPH Posters 2011]Layered Photo Pop-Up [poster] [abstract] [project page]
Binocular-input novel view synthesis. This list is not exhaustive.
[CVPR 2022]3D Moments from Near-Duplicate Photos [paper] [code] [project page] 🍃☁️[CVPR 2022]Stereo Magnification with Multi-Layer Images [paper] [code] [project page]✈️ 💎[ICCV 2019]Extreme View Synthesis [paper] [code][CVPR 2019]Pushing the Boundaries of View Extrapolation with Multiplane Images [paper]✈️ [SIGGRAPH 2018]Stereo Magnification: Learning View Synthesis using Multiplane Images [paper] [code] [project page]✈️
3D photography of other kinds, such as relying on multi-view input, using modalities other than natural images, using generative methods without explicit 3D modeling.
[SIGGRAPH 2023]ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields [paper] [code] [project page] 🚕[CVPR 2023]Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask [paper] [project page] 💎[ICLR 2023]Make-A-Video: Text-to-Video Generation without Text-Video Data [paper] [project page][ECCV 2022]SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image [paper] [code] [project page] 🚕[CVPR 2022]Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Image [paper] [code] [project page][ICCV 2021]Geometry-Free View Synthesis: Transformers and no 3D Priors [paper] [code] [project page][ICCV 2021]Learning to Stylize Novel Views [paper] [code] [project page] ☁️[ICCV 2021]Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis [paper] [code] [project page] 🚕[ICCV 2017]Personalized Cinemagraphs using Semantic Understanding and Collaborative Learning [paper]
3D Photography with dynamics. The works that use a single image as input:
[arXiv 2024]VividDream: Generating 3D Scene with Ambient Dynamics [paper] [project page] ☁️☃️[ACM MM 2023]Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image [paper] 🍃☁️[IJCAI 2023]Learning 3D Photography Videos via Self-supervised Diffusion on Single Images [paper][CVPR 2023]3D Cinemagraphy from a Single Image [paper] [code] [project page] 🍃☁️
The works with multi-view or video input:
[SIGGRAPH 2024]Modeling Ambient Scene Dynamics for Free-view Synthesis [paper] [project page] ☃️[arXiv 2024]LoopGaussian: Creating 3D Cinemagraph with Multi-view Images via Eulerian Motion Field [paper] [code] [project page] ☁️[CVPR 2023]3D Video Loops from Asynchronous Input [paper] [code] [project page]✈️
Landscape Animation: running water, moving clouds, etc.
[CVPR 2024]StyleCineGAN: Landscape Cinemagraph Generation using a Pre-trained StyleGAN [paper] [code] [project page][TOG 2023]Text-Guided Synthesis of Eulerian Cinemagraphs [paper] [code] [project page][SA 2022]Water Simulation and Rendering from a Still Photograph [paper] [project page][ICCV 2023]DiffDreamer: Consistent Single-view Perpetual View Generation with Conditional Diffusion Models [paper] [project page][ICLR 2023]Towards Smooth Video Composition [paper] [project page][ICCV 2023]Simulating Fluids in Real-World Still Images [paper] [code] [project page][CVPR 2022]Controllable Animation of Fluid Elements in Still Images [paper] [project page][CVPR 2022]StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2 [paper] [code] [project page][CVPR 2021]Animating Pictures with Eulerian Motion Fields [paper] [project page][MultiMedia 2021]Learning Fine-Grained Motion Embedding for Landscape Animation [paper][ECCV 2020]DeepLandscape: Adversarial Modeling of Landscape Videos [paper] [code] [project page][ECCV 2020]DTVNet: Dynamic Time-lapse Video Generation via Single Still Image [paper] [code][SIGGRAPH Asia 2019]Animating Landscape: Self-Supervised Learning of Decoupled Motion and Appearance for Single-Image Video Synthesis [paper] [code] [project page][CVPR 2018]Learning to Generate Time-lapse Videos Using Multi-stage Dynamic Generative Adversarial Networks [paper] [code] [project page]
Other papers.
[arXiv 2024]Explorative Inbetweening of Time and Space [paper] [project page][arXiv 2024]Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling [paper] [project page][arXiv 2023]AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidance [paper] [code] [project page][arXiv 2023]LivePhoto: Real Image Animation with Text-guided Motion Control [paper] [code] [project page][arXiv 2023]DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors [paper] [code][CVPR 2024]Generative Image Dynamics [paper] [project page][arXiv 2023]DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory [paper] [project page][arXiv 2023]Motion-Conditioned Diffusion Model for Controllable Video Synthesis [paper] [project page][CVPR 2023]Conditional Image-to-Video Generation with Latent Flow Diffusion Models [paper] [code][CVPR 2023]Blowing in the Wind: CycleNet for Human Cinemagraphs from Still Images [paper][ICCV 2021]iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesis [paper] [code] [project page][CVPR 2021]Stochastic Image-to-Video Synthesis Using cINNs [paper] [code] [project page][CVPR 2021]Understanding Object Dynamics for Interactive Image-to-Video Synthesis [paper] [code] [project page][SIGGRAPH 2021]Endless Loops: Detecting and Animating Periodic Patterns in Still Images [paper] [project page][ECCV 2018]Flow-Grounded Spatial-Temporal Video Prediction from Still Images [paper] [code][CVPR 2018]Controllable Video Generation with Sparse Trajectories [paper] [code] [project page][CVPR 2018]MoCoGAN: Decomposing Motion and Content for Video Generation [paper] [code]