README.md

Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror Reflections

Ankit Dhiman ^1,2* · Manan Shah ^1* · Rishubh Parihar ¹ · Yash Bhalgat ³ · Lokesh R Boregowda · R Venkatesh Babu ¹

^* Equal Contribution
¹ Vision and AI Lab, IISc Bangalore
² Samsung R & D Institute India - Bangalore
³ Visual Geometry Group, University of Oxford

Paper | Project Page

Summary

We tackle the challenge of generating realistic mirror reflections using diffusion-based generative models, formulated as an image inpainting task to enable user control over mirror placement. To support this, we introduce SynMirror, a dataset with $198K$ samples rendered from $66K$ 3D objects, including depth maps, normal maps, and segmentation masks to capture scene geometry.

We propose MirrorFusion, a novel depth-conditioned inpainting method that produces high-quality, photo-realistic reflections, given an input image and mirror mask. MirrorFusion outperforms state-of-the-art methods on SynMirror, offering new possibilities for image editing and augmented reality.