Skip to content

Commit

Permalink
Merge branch 'release/v0.2.0' into 'main'
Browse files Browse the repository at this point in the history
Release/v0.2.0

See merge request momentfactory/products/xagora/omniverse/omniverse-lidar-live-synthetic-data!8
  • Loading branch information
agrosjea committed Dec 22, 2023
2 parents a839461 + 36c6889 commit 1a6fcac
Show file tree
Hide file tree
Showing 61 changed files with 294 additions and 3,163 deletions.
7 changes: 7 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Copyright 2023 Moment Factory Studios Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, andor sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
64 changes: 44 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,63 @@
# MF Lidar live synthetic data [mf.ov.lidar_live_synth]

Adds an Action Graph Node ("Generic/Beam to Ouster UDP") to send Isaac beam data via the Ouster(tm) UDP procotol.
This will allow any third party software implementing supporting Ouster(tm) lidars to be connected to simulated sensors instead of physical sensors.
This allows any third party software implementing Ouster(tm) lidars to be connected to simulated sensors instead of physical sensors.

All the data inputs from the node should be plugged in from a "Isaac Read Lidar Beams Node".
Developped for kit 105.1 and currently working only in Isaac Sim.
This extensions provides pre-built binaries for Windows and Linux x86_64.
You may want to compile from the [source code](https://github.com/MomentFactory/Omniverse-Lidar-Live-Synthetic-Data)

## Supported Lidars
## Requirements
- kit 105 based
- Isaac Sim > 2023.1.0
- Linux or Windows platforms

### Supported Lidars

Currently, only Ouster™ sensors are supported.

The Lidar must have 16, 32, 64 or 128 rows to be supported by the procotol.

Lidar FOVs and resolutions are not transmitted in the protocol and therefore should match those of an actual Ouster(tm) model (22.5, 45 or 90 degrees FOV) for an accurate reconstruction by the receiving software.

JSON config files that describe the angles of the beams for an external application are included in the 'data' folder (example : [OusterJsonConfigOmniverse-OS0-16.json](exts/mf.ov.lidar_live_synth/data/OusterJsonConfigOmniverse-OS0-16.json)). These files can be used in Cirrus as the Ouster(tm) Json Config file to properly recronstruct the data with the correct beam angles. OS0 are 90 degrees FOV, OS1 are 45 and OS2 are 22.5.
JSON config files that describe the angles of the beams for an external application are included in the 'data' folder (example : [OusterJsonConfigOmniverse-OS0-16.json](source/extensions/mf.ov.lidar_live_synth/data/OusterJsonConfigOmniverse-OS0-16.json)). These files can be used in Cirrus as the Ouster(tm) Json Config file to properly recronstruct the data with the correct beam angles. OS0 are 90 degrees FOV, OS1 are 45 and OS2 are 22.5.

## Build

### Windows

- Run `./build.bat`

### Linux

- Install Docker
- Run `./build.sh`

## Using the extension

Requires Isaac Sim as well as a third party software that can receive and parse Ouster Lidar sensors frames.

## How to use
You can use the [isaac_lidar_sample_moving_cube.usd](source/extensions/mf.ov.lidar_live_synth/samples/isaac_lidar_sample_moving_cube.usd), or [isaac_lidar_ouster_sample.usd](source/extensions/mf.ov.lidar_live_synth/samples//isaac_lidar_ouster_sample.usd), or create your own following the instructions below.

Requires Isaac Sim as well as a third party software that can connect to Lidar sensors.
### Enable the extension

You can use the [isaac_lidar_sample_moving_cube.usd](./isaac_lidar_sample_moving_cube.usd), or [isaac_lidar_ouster_sample.usd](./isaac_lidar_ouster_sample.usd), or create your own following the instructions below.
In Isaac Sim :
- Windows > Extensions.
- Switch to THIRD PARY tab.
- Install and enable the extension.

### In Isaac Sim:
1. Activate the MF LIDAR LIVE SYNTHETIC DATA extension
2. Open or create a scene
1. Open or create a scene
- Meshes requires a Rigidbody to intercept Lidar raycast
- Right-click a mesh, then select `Add / Physics / Rigid Body`
3. Add a Lidar to the scene if not present
2. Add a Lidar to the scene if not present
- `Create / Isaac / Sensors / Lidar / Generic`
- Unfold Raw USD Properties
- Check `drawPoints` and/or `drawLines` if you want to see the point cloud
- Check the `enabled` property
- Use `horizontalFov`, `horizontalResolution`. `maxRange`, `minRange`, `verticalFov`, and `verticalResolution` to define the Lidar raycast zone
- set `rotationRate` to `0` if you want continuous raycast
4. Create an action graph
3. Create an action graph
- Right-click the Stage, then select `Create / Visual Scripting / Action Graph`
- Right-click the Action Graph then select "Open Graph"
- Add a `Event / On Playback Tick` node
Expand All @@ -48,18 +72,18 @@ You can use the [isaac_lidar_sample_moving_cube.usd](./isaac_lidar_sample_moving
- `Num Cols`
- `Num Rows`

6. Press the play icon (SPACE) to begin the simulation
4. Press the play icon (SPACE) to begin the simulation

### Beam to Ouster UDP
- `IP Address` (string): The IP address to send the data to
- `Port` (int): The port to send the data to (also used in Cirrus)
- `Broadcast` (bool): Check to property if the IP Address is a broadcast address
#### Beam to Ouster UDP fields
- `IP Address` (string): The IP address to send the data to.
- `Port` (int): The port to send the data to.
- `Broadcast` (bool): Check to indicate the IP Address is a broadcast address.

### Building the C++ extension
As the extension is written in C++ for performance reasons, developers need to build it before using it. Most of it works in the same way as the official Omniverse C++ examples (https://github.com/NVIDIA-Omniverse/kit-extension-template-cpp).
## Developer notes
As the extension is written in C++ for performance reasons, developers need to build it before using it. Most of it works in the same way as the official [Omniverse C++ examples](https://github.com/NVIDIA-Omniverse/kit-extension-template-cpp).

The first step is to run the build.bat file at the root of the repo. It will generate the actual extension files usable by Omniverse, as well as the Visual Studio files. It is recommended to work in Visual Studio (2019 and above) for C++, although VSCode should also work. The build.bat script generates the VS2019 .sln files in _compiler\vs2019\kit-extension-template-cpp.sln . It should work as-is. Do not upgrade the compiler and Windows SDK versions if asked to do so, and install the correct Windows SDK for the VS Installer if it is missing on your machine.
The first step is to run the `build.bat` file at the root of the repo. It will generate the actual extension files usable by Omniverse, as well as the Visual Studio files. It is recommended to work in Visual Studio (2019 and above) for C++, although VSCode should also work. The `build.bat` script generates the VS2019 `.sln` files in `_compiler\vs2019\kit-extension-template-cpp.sln` . It should work as-is. Do not upgrade the compiler and Windows SDK versions if asked to do so, and install the correct Windows SDK for the VS Installer if it is missing on your machine.

Unlike the samples, we do not recommend running the project by launching it via Visual Studio, since the extension is made specifically for Isaac Sim, and Visual Studio doesnt launch it within an Isaac Sim environment. It is recommended to run Isaac and attach the VS debugger to it by going to Debug -> Attach to Process and selecting the kit.exe coresponding to Isaac. Make sure to attach to Native Code. If you have the "Python - Profiling" extension, it might want to try to attach to Python code instead. One thing to note is that the symbols for the extension will only be loaded IF the extension is enabled after attaching. If the extension is already enabled, disabling then enabling it will also work. Also, to update the extension in Isaac after doing some changes and building, it needs to be disabled and enabled again (The extension willl probably fail to build if it is in use as the dll cannot be overwritten anyways).

To add the extension to Isaac, simply add the built plugin folder (c:/git/omniverse/omniverse-lidar-synthetic-data/_build/windows-x86_64/release/exts or c:/git/omniverse/omniverse-lidar-synthetic-data/_build/windows-x86_64/debug/exts for a debug build) to the extension manager paths
To add the extension to Isaac, simply add the built plugin folder (`c:/git/omniverse/omniverse-lidar-synthetic-data/_build/windows-x86_64/release/exts` or `c:/git/omniverse/omniverse-lidar-synthetic-data/_build/windows-x86_64/debug/exts` for a debug build) to the extension manager paths.
2 changes: 1 addition & 1 deletion VERSION.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
104.0
105.1
Empty file modified build.sh
100644 → 100755
Empty file.
17 changes: 9 additions & 8 deletions deps/ext-deps.packman.xml
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,19 @@
<import path="../_build/${platform}/${config}/kit/dev/all-deps.packman.xml">
<filter include="boost_preprocessor"/>
<filter include="imgui"/>
<filter include="nv_usd_py37_debug"/>
<filter include="nv_usd_py37_release"/>
<filter include="usd_ext_physics_debug"/>
<filter include="usd_ext_physics_release"/>
<filter include="nv_usd_py310_release"/>
</import>

<!-- Override the link paths to point to the correct locations. -->
<dependency name="boost_preprocessor" linkPath="../_build/target-deps/boost-preprocessor"/>
<dependency name="imgui" linkPath="../_build/target-deps/imgui"/>
<dependency name="nv_usd_py37_debug" linkPath="../_build/target-deps/nv_usd/debug"/>
<dependency name="nv_usd_py37_release" linkPath="../_build/target-deps/nv_usd/release"/>
<dependency name="usd_ext_physics_debug" linkPath="../_build/target-deps/usd_ext_physics/debug"/>
<dependency name="usd_ext_physics_release" linkPath="../_build/target-deps/usd_ext_physics/release"/>
<dependency name="nv_usd_py310_release" linkPath="../_build/target-deps/nv_usd/release"/>

<!-- Because we always use the release kit-sdk we have to explicitly refer to the debug usd package. -->
<dependency name="nv_usd_py310_debug" linkPath="../_build/target-deps/nv_usd/debug">
<package name="nv-usd" version="22.11.nv.0.2.1058.7d2f59ad-win64_py310_debug-dev_omniverse" platforms="windows-x86_64" checksum="02f7c3477830eb17699cc91774438edd8651f3ec0031582c67093ae3276f360b" />
<package name="nv-usd" version="22.11.nv.0.2.1058.7d2f59ad-linux64_py310-centos_debug-dev_omniverse" platforms="linux-x86_64" checksum="2ac18e0470d05b251a2f36691a1dc1b28da340da92b19175d890addb762adb0f"/>
<package name="nv-usd" version="22.11.nv.0.2.1058.7d2f59ad-linux-aarch64_py310_debug-dev_omniverse" platforms="linux-aarch64" checksum="904ede636008fb011b5f3d66c1a7c2969dfba291dcf1a227fa7503a714f1f18d" />
</dependency>

</project>
9 changes: 4 additions & 5 deletions deps/host-deps.packman.xml
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
<project toolsVersion="5.0">
<dependency name="premake" linkPath="../_build/host-deps/premake">
<package name="premake" version="5.0.0-alpha15.dev+pipeline3388156.1f299ea4-${platform}"/>
<package name="premake" version="5.0.0-alpha15.dev+pipeline3388156.1f299ea4-windows-x86_64" checksum="b1e5dcef9acf47b0c86a4630afa4fadc9485b878e25e4321ac5afbb826bbdf93" platforms="windows-x86_64" />
<package name="premake" version="5.0.0-alpha15.dev+pipeline3388156.1f299ea4-linux-x86_64" checksum="ae15e63cf6d53571fa3bdfa33ddcec8a3be90675cdd155590a26bcd75d04d73f" platforms="linux-x86_64" />
</dependency>

<dependency name="msvc" linkPath="../_build/host-deps/msvc">
<package name="msvc" version="2019-16.7.6-license" platforms="windows-x86_64"/>
<package name="msvc" version="2019-16.7.6-license" platforms="windows-x86_64" checksum="0e37c0f29899fe10dcbef6756bcd69c2c4422a3ca1101206df272dc3d295b92d" />
</dependency>

<dependency name="winsdk" linkPath="../_build/host-deps/winsdk">
<package name="winsdk" version="10.0.18362.0-license" platforms="windows-x86_64"/>
<package name="winsdk" version="10.0.18362.0-license" platforms="windows-x86_64" checksum="2db7aeb2278b79c6c9fbca8f5d72b16090b3554f52b1f3e5f1c8739c5132a3d6" />
</dependency>
</project>
2 changes: 2 additions & 0 deletions deps/kit-sdk-deps.packman.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@
<!-- Import dependencies from Kit SDK to ensure we're using the same versions. -->
<import path="../_build/${platform}/${config}/kit/dev/all-deps.packman.xml">
<filter include="carb_sdk_plugins"/>
<filter include="cuda"/>
<filter include="doctest"/>
<filter include="pybind11"/>
<filter include="python"/>
</import>

<!-- Override the link paths to point to the correct locations. -->
<dependency name="carb_sdk_plugins" linkPath="../_build/target-deps/carb_sdk_plugins"/>
<dependency name="cuda" linkPath="../_build/target-deps/cuda"/>
<dependency name="doctest" linkPath="../_build/target-deps/doctest"/>
<dependency name="pybind11" linkPath="../_build/target-deps/pybind11"/>
<dependency name="python" linkPath="../_build/target-deps/python"/>
Expand Down
4 changes: 3 additions & 1 deletion deps/kit-sdk.packman.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
<project toolsVersion="5.0">
<!-- We always depend on the release kit-sdk package, regardless of config -->
<dependency name="kit_sdk_${config}" linkPath="../_build/${platform}/${config}/kit" tags="${config} non-redist">
<package name="kit-sdk" version="104.2+release.295.529af2e4.tc.${platform}.release"/>
<package name="kit-sdk" version="105.1+release.127680.dd92291b.tc.windows-x86_64.release" platforms="windows-x86_64" checksum="78b6054c730a44b97e6551eae9e17f45384621f244d4babde5264a1d6df3038f" />
<package name="kit-sdk" version="105.1+release.127680.dd92291b.tc.linux-x86_64.release" platforms="linux-x86_64" checksum="2f8357eda2de9232c0b4cb345eb6c4d3c3aa8c4c9685ed45d4bfe749af57b0b8" />
</dependency>
</project>
23 changes: 16 additions & 7 deletions deps/repo-deps.packman.xml
Original file line number Diff line number Diff line change
@@ -1,23 +1,32 @@
<project toolsVersion="5.0">
<dependency name="repo_build" linkPath="../_repo/deps/repo_build">
<package name="repo_build" version="0.27.1"/>
<package name="repo_build" version="0.44.6" checksum="11858f3d45b15d83f0279fa96e2813232bfd65755d0cf45861f5fdd28a5a39b6" />
</dependency>
<dependency name="repo_changelog" linkPath="../_repo/deps/repo_changelog">
<package name="repo_changelog" version="0.3.2" checksum="fbe4bc4257d5aec1c964f2616257043095a9dfac8a10e027ac96aa89340f1423" />
</dependency>
<dependency name="repo_docs" linkPath="../_repo/deps/repo_docs">
<package name="repo_docs" version="0.10.4"/>
<package name="repo_docs" version="0.37.3" checksum="78bd6488c1cd7295ab6728d9cd0b79fac3684598bcaebefad710fc79e3a7b8ea" />
</dependency>
<dependency name="repo_kit_tools" linkPath="../_repo/deps/repo_kit_tools">
<package name="repo_kit_tools" version="0.8.16"/>
<package name="repo_kit_tools" version="0.11.8" checksum="8d6e1ade8b75b40f880505ba62308958d87a88e52db6a3b932be3da387a8a571" />
</dependency>
<dependency name="repo_licensing" linkPath="../_repo/deps/repo_licensing">
<package name="repo_licensing" version="1.12.0" checksum="2fa002302a776f1104896f39c8822a8c9516ef6c0ce251548b2b915979666b9d" />
</dependency>
<dependency name="repo_man" linkPath="../_repo/deps/repo_man">
<package name="repo_man" version="1.10.1"/>
<package name="repo_man" version="1.36.1" checksum="aba22f72ec46b7d2761c5fe2eee397bcb6958dda9b4a8aaca947eb69b97f6089" />
</dependency>
<dependency name="repo_package" linkPath="../_repo/deps/repo_package">
<package name="repo_package" version="5.6.10" />
<package name="repo_package" version="5.8.8" checksum="b8279d841f7201b44d9b232b934960d9a302367be59ee64e976345854b741fec" />
</dependency>
<dependency name="repo_format" linkPath="../_repo/deps/repo_format">
<package name="repo_format" version="2.7.0" checksum="8083eb423043de585dfdfd3cf7637d7e50ba2a297abb8bebcaef4307b80503bb" />
</dependency>
<dependency name="repo_source" linkPath="../_repo/deps/repo_source">
<package name="repo_source" version="0.3.0"/>
<package name="repo_source" version="0.4.2" checksum="05776a984978d84611cb8becd5ed9c26137434e0abff6e3076f36ab354313423" />
</dependency>
<dependency name="repo_test" linkPath="../_repo/deps/repo_test">
<package name="repo_test" version="1.7.1"/>
<package name="repo_test" version="2.9.3" checksum="1903a2a1c998ca4adc87bc20520e91a9af21bf18a6a48a8e05467fe29d674931" />
</dependency>
</project>

This file was deleted.

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
28 changes: 0 additions & 28 deletions exts/mf.ov.lidar_live_synth/config/extension.toml

This file was deleted.

Loading

0 comments on commit 1a6fcac

Please sign in to comment.