LIDAR Doesn't Detect Whole Mesh, But Only a Bounding Box

[flolu]

I have imported my own mesh from Blender into Unreal Engine. (export as .fbx from Blender and then import into Unreal Engine)

As you can see, there is a detailed mesh.

But unfortunately the LIDAR does not recognize the details of the mesh. It only detects a bounding box around the objects.

Is this caused by the LIDAR implementation of AirSim or do I have to modify some settings in Unreal Engine to make the LIDAR "see" the detailed mesh?

[WouterJansen]

Unreal uses two collision standards: simple and complex. By default simple will be used. You are looking into using the complex collision. More information on it here: https://docs.unrealengine.com/en-US/Engine/Physics/SimpleVsComplex/index.html#:~:text=Simple%20Collision%20are%20primitives%20like,will%20use%20the%20corresponding%20shape.

Two ways to achieve this:
1) Alter AirSim Plugin

AirSim/Unreal/Plugins/AirSim/Source/AirBlueprintLib.cpp

Line 593 in bd8ee6b

FCollisionQueryParams trace_params;

Edit the FCollisionQueryParams to trace the complex collision

FCollisionQueryParams trace_params;
trace_params.bTraceComplex = true;

2) Alter the mesh
Alter your meshes in Unreal to use the complex collision in all cases by choosing UseSimpleAsComplex.

[catproof]

Unreal uses two collision standards: simple and complex. By default simple will be used. You are looking into using the complex collision. More information on it here: https://docs.unrealengine.com/en-US/Engine/Physics/SimpleVsComplex/index.html#:~:text=Simple%20Collision%20are%20primitives%20like,will%20use%20the%20corresponding%20shape.

Two ways to achieve this: 1) Alter AirSim Plugin

AirSim/Unreal/Plugins/AirSim/Source/AirBlueprintLib.cpp

Line 593 in bd8ee6b

FCollisionQueryParams trace_params;

Edit the FCollisionQueryParams to trace the complex collision

FCollisionQueryParams trace_params;
trace_params.bTraceComplex = true;

2) Alter the mesh Alter your meshes in Unreal to use the complex collision in all cases by choosing UseSimpleAsComplex.

@WouterJansen @flolu if we go with option 1), do we have to rebuild AirSim? and also setup our environment again from scratch? https://microsoft.github.io/AirSim/build_windows/ "Run build.cmd from the command line. This will create ready to use plugin bits in the Unreal\Plugins folder that can be dropped into any Unreal project."

for option 2) I don't see the same settings that you see, for some reason. I clicked on a mesh in my environment and I get the following:

I think by default, AirSim should use the complex meshes... perhaps an update should be made?

[WouterJansen]

@WouterJansen @flolu if we go with option 1), do we have to rebuild AirSim? and also setup our environment again from scratch? https://microsoft.github.io/AirSim/build_windows/ "Run build.cmd from the command line. This will create ready to use plugin bits in the Unreal\Plugins folder that can be dropped into any Unreal project."

Depends on your workflow but yes, it requires to rebuild the Unreal plugin. The build.cmd will do this for you.

for option 2) I don't see the same settings that you see, for some reason. I clicked on a mesh in my environment and I get the following:
I think by default, AirSim should use the complex meshes... perhaps an update should be made?

You are looking at a StaticmeshComponent, not a staticMesh.

[catproof]

You are looking at a StaticmeshComponent, not a staticMesh.

@WouterJansen Ah I see. I figured out how to get to the staticMesh just now. Do you know if there is a way for me to select all the staticMeshs in the environment at the same time, and change this setting for all of them with one click?

To me, it seems like going with option 1 (Alter AirSim Plugin) might be more computationally efficient.

[catproof]

@WouterJansen after reading the unreal engine documents, you should actually set the static mesh to
Use Complex Collision As Simple
instead of
Use Simple Collision As Complex
as you suggested.

[catproof]

@WouterJansen i noticed in the cpp file you mentioned, trace_params appears more than once. Do we only change it in the GetObstacle method? or also change it in the HasObstacle and GetLastObstaclePosition methods?

[WouterJansen]

@WouterJansen after reading the unreal engine documents, you should actually set the static mesh to Use Complex Collision As Simple instead of Use Simple Collision As Complex as you suggested.

That is incorrect. If you use simple collision you will run into the issue as the first post said.

@WouterJansen i noticed in the cpp file you mentioned, trace_params appears more than once. Do we only change it in the GetObstacle method? or also change it in the HasObstacle and GetLastObstaclePosition methods?

The LiDAR simulation in AirSim relies on the HasObstacle() function so if you wish to make the LiDAR use complex collision by default then it needs to be done there. I don't know exactly what part of AirSim uses GetLastObstaclePosition() so cannot comment on that.

[catproof]

@WouterJansen "Use Complex Collision As Simple: This means that if a simple query is requested, the engine will query against complex shapes; basically ignoring the simple collision. This allows us to use the trimesh for the physics simulation collision..."

https://docs.unrealengine.com/4.27/en-US/InteractiveExperiences/Physics/SimpleVsComplex/

I modified the settings on all the trace_params in the file and rebuilt AirSim. No issues. I imagine the other trace_params are for the drone figuring out collisions with objects it may run into.

[WouterJansen]

@WouterJansen "Use Complex Collision As Simple: This means that if a simple query is requested, the engine will query against complex shapes; basically ignoring the simple collision. This allows us to use the trimesh for the physics simulation collision..."

https://docs.unrealengine.com/4.27/en-US/InteractiveExperiences/Physics/SimpleVsComplex/

Ah yes you are correct, the naming is confusing I find. Sorry.

[catproof]

@WouterJansen "Use Complex Collision As Simple: This means that if a simple query is requested, the engine will query against complex shapes; basically ignoring the simple collision. This allows us to use the trimesh for the physics simulation collision..."
https://docs.unrealengine.com/4.27/en-US/InteractiveExperiences/Physics/SimpleVsComplex/

Ah yes you are correct, the naming is confusing I find. Sorry.

lol ya, I agree. The naming is quite confusing. I had to read it the definition like 5 times just to make sure.

[QSJZDDY]

I can get the pointcloud data but when I start to simulate I can't see the lidar green scan points like your image. How to deal with it?

[WouterJansen]

I can get the pointcloud data but when I start to simulate I can't see the lidar green scan points like your image. How to deal with it?

The green points are a debug draw mode that can be enabled for the LiDAR sensor.
https://github.com/microsoft/AirSim/blob/master/docs/lidar.md#server-side-visualization-for-debugging

To enable, set DrawDebugPoints to true in your settings configuration file.

...
 "LidarSensor1": {
        ...
	"DrawDebugPoints": true,
        ...
},
...