This repository contains an OpenGL pipeline, to render object RGBA-D layers from real 3D reconstructions (Stanford 2D-3D S). We provide C++ rendering code together with Python post-processing scripts. These layers can be merged based on per-pixel depth order to create Layered Depth Images (LDI). If you find this code useful, consider citing
@inproceedings{dhamo2019,
title={Object-Driven Multi-Layer Scene Decomposition From a Single Image},
author={Dhamo, Helisa and Navab, Nassir and Tombari, Federico},
booktitle={ICCV},
year={2019}
We used this rendered data to train the learned models from this paper.
- CMake 2.8 or above
- OpenCV 3.0 or above
- OpenGL 3.3
- Assimp
We tested the code with cuda=9.0, but other versions should be possible.
Download the Stanford 2D-3D S dataset from the official site .
Clone this repository and move to the root directory
git clone https://github.com/he-dhamo/OMLD-rendering.git
cd OMLD-rendering/
Prepare files to index pose and rgb files of the stanford data, for every area:
chmod +x prepare_filelists.sh
./prepare_filelists.sh <stanford_root_path>
Create output directories for every area
chmod +x make_dirs.sh
./make_dirs.sh
The default output directory is set to ./generated_images. If you change this, please update the config file with the same output dir.
Create build directory and move there: mkdir build && cd build
Set options and paths in include/config.h
Run CMake: cmake ..
Compile: make
Run executable: ./RenderLayers
The program will save output images under OUT_DIR, in a depth/, rgb/ and instance/ folder; e.g. depth/<frame_idx>.png for the whole image as well as depth/<frame_idx>_<layer_idx>.png for each instance layer. In addition, we save the camera intrinsics and original frame file name (for all generated frames) in metadata.json.
The C++ code is based on Learn OpenGL.
The post_pocessing/ directory contains python scripts that:
merge_layout.py merge structural components (wall, floor, window, ceiling) to a common layout layer. The script will remove the current layers and create new ones including a layout layer.
python merge_layout.py --area area_1 --global_dir ../generated_images/
create_splits.py create train and validation splits. The layers for training and testing are selected based on the level of amodal overlap with other layers. For example, to create the train splits:
python create_splits.py --split train --resume_json_state no --global_dir ../generated_images/