This repo is an extension to sunset1995 implementation of DirectVoxGO [1], and can be used to easily convert custom datasets from colmap to the format expected by DirectVoxGO.
Additionally, you may render random trajectories around the object as a video, or use an user-control script to move around the object live (depending on your hardware).
[1] Sun et al., Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction. arXiv:2111.11215 preprint, 2021.
DirectVoxGO expects image file names to begin by 0_ for train set, 1_ for validation set, and 2_ for test set. Use the script rename_images.py to decide the split and rename images.
Colmap is a software used to perform photogrammetry on a dataset of views of an objects, to recover intrinsic and extrinsic parameters. In our implementation, the camera model used in colmap is expected to be SIMPLE_PINHOLE and we expect a shared camera among all views. As an output, we typically have
model/
├── cameras.txt
├── images.txt
└── points3D.txt
where cameras.txt hold the intrinsic parameters of the camera and images.txt holds the extrinsic parameters for each view as a quaterion and a translation.
To format this data in the format expected by DirectVoxGO implementation, use
python3 colmap_to_directvoxgo.py --model path/to/model/ --images path/to/images/folder --output path/to/output/folder/
where model/ is the repository holding the cameras.txt and images.txt, and the image folder is e.g. the one used for the photogrammetry.
The center of the object needs to be known to perform rotations around the object which is not at the world origin.
After training a model, use
python3 run.py --config configs/path/to/config_file.py --export_coarse_only coarse_data.npz
to generate coarse density data used to find the center of the object.
Then, generate a random trajectory using a Bézier curve with
python3 generate_bezier_curve.py --output data/dataset/ --init path/to/initial_pose.txt --coordinates coarse_data.npz
where the initial pose is used at the beginning of the trajectory, and can for example be one of the pose from the test set. Here --output should be the folder where the training data is stored, so that a test_traj.txt file can be saved there with the poses to be used for the video.
The video can be generated with
python3 run.py --config logs/.../config.py --render_only --render_video
and is saved in the log directory from the config used.
Use
python3 interaction_3d.py --config logs/path/config.py --init path/initial_pose.txt --coordinates coarse_data.npz
The controls are:
- 8 <-> 5
- 4 <-> 6
- 1 <-> 2
Use q to exit the window gracefully.
This work was done by Jingjing Huang and Félix Marty as part of a one week long course at Mines Paris under the supervision of Jean-Emmanuel Deschaud.