v0.17 release

Open3D 0.17 Release Notes

We are happy to bring you the best Open3D yet! This is a "tick" release focused on resolving existing issues and eliminating bugs. We resolved over 150 issues for Open3D and Open3D-ML since the last release.

Here are the main highlights of this release:

• Open3D now has support for differentiable rendering with Mitsuba.

raven_demo2.mp4
Here we use Mitsuba to recover the albedo map from an input image for an object in Open3D. Example.

• New lighter wheel (~105MB) for x86_64 Linux, which is suitable for systems without CUDA. Try it out with:
  pip install open3d-cpu.
• The Open3D viewer app is now available for Windows. This version is unsigned, so expect Windows warnings.
• Apple Silicon wheels now come with 3DML support.
• Preview of distributed training with PyTorch for PointPillars on the Waymo dataset.
• PointCloud partitioning with PCA.

Visualization and GUI

[New] Integration with Mitsuba.

• Open3D geometries and materials seamlessly convert to Mitsuba's Principled BSDF:

dataset = o3d.data.MonkeyModel()
mesh = o3d.t.io.read_triangle_mesh(dataset.path)
mesh_center = mesh.get_axis_aligned_bounding_box().get_center()
mesh.material.set_default_properties()
mesh.material.material_name = 'defaultLit'
mesh.material.scalar_properties['metallic'] = 1.0
mesh.material.texture_maps['albedo'] = o3d.t.io.read_image(dataset.path_map['albedo'])
mesh.material.texture_maps['roughness'] = o3d.t.io.read_image(dataset.path_map['roughness'])
mesh.material.texture_maps['metallic'] = o3d.t.io.read_image(dataset.path_map['metallic'])

mi_mesh = mesh.to_mitsuba('monkey')
img = render_mesh(mi_mesh, mesh_center.numpy())
mi.Bitmap(img).write('test.exr')

• Also supports applying other Mitsuba BSDFs to Open3D geometries:

bsdf_rough_plastic = mi.load_dict({
    'type': 'roughplastic',
    'diffuse_reflectance': {
        'type': 'rgb',
        'value': [0.1, 0.1, 0.1]
    },
    'alpha': 0.1,
    'int_ior': 'bk7',
    'ext_ior': 'air',
})
mi_mesh = mesh.to_mitsuba('monkey', bsdf=bsdf_rough_plastic)
img = render_mesh(mi_mesh, mesh_center.numpy())
mi.Bitmap(img).write('test3.exr')

See examples/python/visualization/to_mitsuba.py for more details.

Visualization and GUI (other updates)

• [New] The Open3D viewer app is now available for Windows. This version is unsigned, so expect Windows warnings.

• [New] Add draw_web CLI for remote visualization. Try it out with
  (remotehost) $ open3d draw_web --bind_all /path/to/3D/file and then open the browser to http://remotehost:8888
• [New] Unprojection function in the ViewControl (contributed by @PieroV).
• [New] Added interaction methods for Python ImageWidget (contributed by @MartinEekGerhardsen).
• [Update] Updated customized_visualization tutorial (contributed by @ClarytyLLC).
• [Update] Convert material properties when converting from legacy to tensor TriangleMesh.
• [Fix] O3DVisualizer::RemoveGeometry() function vector subscript out of range (contributed by @UnadXiao).
• [Fix] Fast check for 3D tensorboard data. Open3D does not delay Tensorboard startup for large run folders now.
• [Fix] Fix scrolling direction bug for TextEdit (contributed by @cansik).
• [Fix] VisualizerWithVertexSelection: Call selection_changed callback, when deselecting all points (Ctrl + R) (contributed by @micsc12).

Geometry

• [New] robust plane detection (contributed by @plusk01).

• [New] Minimal bounding box computation for point clouds (contributed by @ligerlac).
• [New] Added RemoveStatisticalOutliers to tensor PointCloud. (contributed by @yuecideng).
• [New] Added ComputeTriangleNormals ComputeVertexNormals NormalizeNormals to tensor Trianglemesh. (contributed by @yuecideng).
• [New] Added normals oriented functions to tensor PointCloud. (contributed by @yuecideng).
• [New] Added tensor based OrientedboundingBox. (contributed by @yuecideng).
• [New] Added tutorial for tensor based PointCloud.
• [New] Enable python copy & deepcopy for tensor geometry. (contributed by @yuecideng).
• [New] Added partitioning of point clouds and triangle meshes using PCA.

pca_partition.mp4

• [New] Added a function to extract faces in a triangle mesh with a binary mask.

• [New] UV MAPS Tutorial (contributed by @samliozge).
• [Update] Added voting and ray jitter to RaycastingScene to improve robustness of signed distance queries.
• [Update] Improved speed of CreateFromTriangleMeshWithinBounds() by > 100x (contributed by @Hodong-Hwang).
• [Update] Added parallelization to UV atlas computation by partitioning the mesh.
• [Update] Function CreateFromVoxelGrid is made static (contributed by @mjaein).
• [Fix] Fixed wrong voxel center calculation in CreateFromTriangleMeshWithinBounds() (contributed by @plusk01).
• [Fix] Replace Vectors from internal embree headers with Eigen.
• [Fix] Use same beta value in SamplePointsPoissonDisk() as in the paper.
• [Fix] Fix Python Image calculations (pybind Image::To argument order).
• [Fix] Consistent face orientation for generated alpha shape.

Pipelines

• [New] Tensor reconstruction system upgrade: make fragments (contributed by @theNded).
• [Update] Added additional python bindings for RGBD odometry functionality (contributed by @jdavidberger).
• [Update] release python GIL for generalized icp (contributed by @yuecideng).
• [Fix] fix call to AdvancedMatching in FGR (contributed by @plusk01).
• [Fix] Update ICP fitness and rmse value at the end of the loop (contributed by @bialasjaroslaw).

I/O

• [Update] Add loading colors and uvs in tgeometry TriangleMesh IO.

Core

• [New] Support for arbitrary imports with Open3D submodules in Python. Now you can use any of the following and more:

from open3d.t.geometry import TriangleMesh
from open3d.ml.torch.ops import fixed_radius_search # pybind symbol
from open3d.ml.tf.models import KPFCNN             # Python code
from open3d.visualization import gui  # pybind symbol
from open3d.visualization import draw # Python code
from open3d.visualization.gui import Color
from open3d.visualization.rendering import Camera

No need for awkward shortcuts such as:

import open3d as o3d
Tensor = o3d.core.Tensor
Image = o3d.t.geometry.Image

• [New] unary minus operator (contributed by @yxlao).
• [Update] Replace license text with SPDX identifier.
• [Fix] Fixed error when use CUDA_CALL without open3d namespace. (contributed by @yuecideng).
• [Fix] Fixed multi-device CUDA switching bug. (contributed by @yxlao).
• [Fix] MiniVec's constructor shall not call Device's constructor/destructor (contributed by @yxlao).
• [Fix] Add neighbor search module to the documentation.

Build System

• [New] The Open3D viewer app is now available for Windows.
• [New] New lighter wheel (~100MB) for x86_64 Linux, which is suitable for systems without CUDA. Try it out with:
  pip install open3d-cpu.
• [Update] Update imgui to latest version (contributed by @roehling).
• [Update] Support CUDA 12.0 build. (contributed by @NobuoTsukamoto).
• [Update] Add options to use system libraries for new dependencies (contributed by @roehling).
• [Update] Hide un-needed DSO symbols in Linux and macOS.
• [Update] Dependency upgrades for security fixes: civetweb, curl, zlib, libjpeg-turbo, wheel, setuptools, werkzeug, nbformat, ipywidgets, oathlib, certifi .
• [Fix] Build with libomp 11.1 and package in macOS wheel for interoperability with PyTorch.
• [Fix] Fix Mac build issue with ASSIMP.
• [Fix] Wstringop-owerflow compilation error for GCC (contributed by @bialasjaroslaw).
• [Fix] Support msgpack versions without cmake (contributed by @cielavenir).
• [Fix] fix Python 3.8 cuda dll loading on Windows (contributed by @yxlao).
• [Fix] More portable ARM system detection.
• [Fix] Fix build compatibility with fmtlib 8 (contributed by @roehling).

Open3D-ML

• [New] PointPillars Waymo distributed training model weights and docs.
• [New] 3DML support with PyTorch and TensorFlow for Apple Silicon.
• [Update] Update PyTorch to v1.13.1.
• [Update] RandLaNet update_probs label smoothing logic (contributed by @mariusud).
• [Fix] Update broken links (contributed by @Birkenpapier and @friendship1).

Known Issues [WIP]

• [FIXED in v0.17.0-1fix6008] The Python wheel may crash when run on Apple Silicon systems, especially on M2. (#5951)
• The Windows app is unsigned, so expect warnings from the system.

Acknowledgment

We would like to thank all of our community contributors for their true labor of love for this release!

@bialasjaroslaw @Birkenpapier @cansik @cielavenir @ClarytyLLC @friendship1 @geppi @Hodong-Hwang @jdavidberger @johnthagen @ligerlac @mariusud @MartinEekGerhardsen @micsc12 @mjaein @NobuoTsukamoto @PieroV @plusk01 @roehling @samliozge @theNded @UnadXiao @yuecideng @yxlao

Also thanks to the many others who helped the Open3D community by reporting as well as resolving issues.