Google Summer of Code 2022
News (Sep 19, 2022): All Open3D GSoC projects have been succesfully completed! Congratulations to the GSoC contributors!
News (May 20, 2022): Here are the selected GSoC 2022 projects:
| Contributor | Project | Assigned Mentors |
|---|---|---|
| fanxiaolin | Open3D visualization on cloud notebooks | Rene |
| Narayanan E.R. | Open3D-ML 3D Object detection model (PV-RCNN++) | Sanskar Agrawal |
| Zhengyu | Geometry processing algorithms for tensor-based geometry | Yixing |
News: The application period has ended for GSoC 2022. Successful candidates will be notified by Google. You may apply for our Google Season of Docs 2022 project by May 06, 2022.
Open3D is an open-source library that supports rapid development of software that deals with 3D data. The Open3D frontend exposes a set of carefully selected data structures and algorithms in both C++ and Python. The backend is highly optimized and is set up for parallelization. Open3D was developed from a clean slate with a carefully considered set of dependencies. It can be set up on different platforms and compiled from source with minimal effort. The code is clean, consistently styled, and maintained via a clear code review mechanism. Open3D has been used in a number of published research projects and is actively deployed in the cloud. We welcome contributions from the open-source community.
This is our ideas page for Google Summer of Code 2022. See the GSoC website for information about participating in the program and Timeline for important dates.
If you are interested in contributing to Open3D, start with the videos and tutorials. Next, get familiar by downloading Open3D (C++ source as well as Python package) and check that you can build the library and run both C++ and Python examples. Visit the discussions and Open3D issues and Open3D-ML issues pages to see how people use Open3D. Before working on code changes, remember to read the contribution guide. If you have a question, you can ask here; If you see something that you know the answer to, go ahead and reply! We welcome participation from everyone.
Listed on this page are project ideas that fill critical gaps in Open3D's functionality. If a project description is unclear, or you have any questions, please ask here. A mentor or community member will get back to you.
When you decide on a project, email Open3D GSoC admins with your resume and the project that you are interested in. The mentor will get back to you and may proceed with a meeting and qualification task. After your meeting with the mentor, they will send you a qualification task to complete and return in a week. The qualification task is necessary to show that you are committed to the project and have the skills necessary for successfully completing it.
Apply definitely before the deadline on April 19th. Take a look at the GSoC timeline for additional information. Note, make sure you apply to Google before April 19th, even if you have not yet finished your qualification task. Please apply as soon as possible: Applications can be improved until the 19th of April, but not afterwards!
Contributors and their project proposals will be selected based on the meeting with the mentor, qualification task submitted and the GSoC application. This also depends on the number of GSoC slots allotted as a whole to Open3D. If you are selected, welcome! We are happy to work closely with you. If you are not selected, we hope you still provide your valuable contributions to Open3D and hope that you apply again next year.
If you are selected for a particular project, then you will be fully integrated into the core Open3D development team. Use the community bonding period (May 20 - June 12) to get used to working with Open3D and interacting with the core development team. Check in with your mentor and start by fixing some of the open issues to get used to the development process. The "work" period begins on June 13th and ends in September. You will attend regular status meetings, which are a great opportunity to ask questions and get feedback on your progress. You will also be able to follow along as everyone works collaboratively on the next Open3D release.
Make sure to keep your mentor in the loop with your progress, and reach out to them on discord or in a meeting if you have any questions. Structure your project in smaller pull requests that are reviewed and merged into Open3D gradually. You can find more software development advice in the GSoC contributor guide.
- Description: The goal of this project is to enhance the functionality of Open3D tensor-based 3D geometry processing algorithms. You’ll be contributing to the core geometry processing features of Open3D. Currently, Open3D supports comprehensive geometry processing with legacy Eigen-based geometry types, while tensor-based geometry support is limited.
-
Expected outcomes:
- Surface reconstruction for triangle mesh (C++ and Python wrappers).
- Geometry sampling for triangle mesh and point clouds (uniform sampling, decimation methods, clustering, etc.) (C++ and Python wrappers).
- Possible Mentors: Yixing Lao, Sameer Sheorey, Rishabh Singh
- Skills required: C++, Python, 3D Geometry
- Expected size: 350hr
- Difficulty level: Medium
- References:
- Description: The goal of this project is to enhance the functionality of Open3D tensor-based 3D geometry file I/O capabilities. Currently, Open3D supports comprehensive geometry file I/O and geometry creation to legacy Eigen based geometry types, while tensor based geometry support is limited.
-
Expected outcomes:
- I/O of geometry classes for tensor-based geometry (C++ and Python wrappers).
- Geometry creation API for tensor-based geometry (C++ and Python wrappers).
- Possible Mentors: Rishabh Singh, Rene Sepulveda, Yixing Lao
- Skills required: C++, Python
- Expected size: 350hr
- Difficulty level: Medium
-
References:
- Legacy file I/O support
-
Legacy geometry creation support (
create_*functions).
- Description: Currently, Open3D visualization does not work consistently on cloud notebooks such as Google Colab. This is due to a need to modernize the Open3D Jupyter widget as well as use CPU rendering when a GPU with graphics capability is not available in the cloud notebook. This will enable new users to quickly start working with Open3D visualization by directly opening tutorial notebooks in the cloud.
- Expected outcomes: Open3D visualization examples can be run from Google Colab.
- Possible Mentors: Yixing Lao, Rene Sepulveda
- Skills required: Python, C++, JavaScript.
- Expected size: 175hr
- Difficulty level: Medium
- References:
- Description: Open3D is rapidly adding new 3D rendering features while supporting cross-platform 3D rendering on GPU, CPU and remotely via WebRTC. Rapid code updates sometimes lead to unintended changes to rendering with different modes. We want to be able to detect such unintended changes before they are incorporated into Open3D through visualization regression CI tests. A library of reference rendering images can be created for pre-defined workflows and these compared with new rendering results to detect unintended rendering output.
- Expected outcomes: Open3D visualization examples should be verified to work correctly on GitHub actions (CPU rendering) and on Google cloud (Nvidia GPU rendering). Additional unit tests should be added for primary rendering functionality. An optional component, if time permits, is to test that various GUI interactions (such as mouse clicks) produce the correct visible result.
- Possible Mentors: Sameer Sheorey, Rene Sepulveda
- Skills required: Python, C++, image matching, test automation.
- Expected size: 350hr
- Difficulty level: Medium
- References:
- Description: Open3D-ML is an extension of Open3D for 3D machine learning tasks. This repo focuses on applications such as 3D semantic segmentation and 3D object detection and provides pre-trained models that reproduce state-of-the-art performance. The goal is to implement a new state-of-the-art 3D Object detection model (PV-RCNN++) in PyTorch and TensorFlow and train it on large scale datasets like Waymo to reproduce the original author's results. You will be provided with access to compute resources for this project.
-
Expected Outcomes:
- Implement PV-RCNN++ training and evaluation with PyTorch.
- Train and reproduce results on Waymo dataset.
- Port the PyTorch implementation to TensorFlow and ensure results can be reproduced.
- Possible Mentors: Sanskar Agarwal, Benjamin Ummenhoffer
- Skills Required: Python, PyTorch, TensorFlow.
- Expected size: 350 hrs.
- Difficulty level: Medium
- References:
- Yixing Lao (administrator AT ivcl DOT org)
- Sameer Sheorey (sameer DOT sheorey AT ivcl DOT org)
- Yixing Lao (@yxlao)
- Sameer Sheorey (@ssheorey)
- Rene Sepulveda (@errissa)
- Benjamin Ummenhofer (@benjaminum)
- Sanskar Agrawal (@sanskar107)
- Rishabh Singh (@reyanshsolis)
Adapted from FFMPEG GSoC ideas page