Vulkan: Raster occlusion margin is too big on baked occluders

[TokisanGames]

Godot version

v4.0.alpha.custom_build.d29e95687

System information

Win 10/64 NVIDIA GeForce GTX 1060/PCIe/SSE2

Issue description

Simple scene w/ baked occluders

The background meshes do not occlude properly. Here you can see even at the start of the video the back ones should be hidden as soon as they are fully occluded, such as the left one as it begins. However, I have to move the camera until it is nearly inside the front object before the back ones turn off. But once you do go inside, the back objects turn on. So the camera is a few centimeters from the object before they turn off.

testocclusion.mp4

Here I made the front object 10x the size and pulled it back so the objects didn't overlap. Now the left back object is on, even though it's fully occluded. This is the border. If I move the camera just a hair right, the left object will turn off.

Steps to reproduce

Look at the MRP in overdraw mode.

Minimal reproduction project

test_occbake.zip

[Calinou]

Occludees use their AABB for frustum and occlusion culling (regardless if they have an OccluderInstance3D child), so sphere-like objects will have a lot of wasted space by design. This is especially the case when working with off-grid angles (45° rotation).

I suppose we could allow Extra Cull Margin to be set to negative values to counter this, but you could end up with false negatives if you push it too far.

[TokisanGames]

Hmm, so in the video and image above, if I mentally draw a cube around the bg rocks, there's still an excessive margin around the aabb.

[Calinou]

Hmm, so in the video and image above, if I mentally draw a cube around the bg rocks, there's still an excessive margin around the aabb.

You can visualize the AABB by selecting one of the rocks at a time in the editor. (Note that this doesn't take Extra Cull Margin into account.)

[TokisanGames]

Ok. Here I've selected the background rocks and you can see that even considering their rotated AABBs, the margin is still wayyyyy too big. There's no reason at all for that middle rock to not be culled, especially.

[Calinou]

This may be due to DirectionalLight3D shadow rendering causing the AABBs to be extended (to prevent occludees' shadows from disappearing when they shouldn't). To disable the preview sun in the editor, try adding a DirectionalLight3D node then hiding it.

[TokisanGames]

Good idea. I tried it, but the light made no difference to the effective culling margin. In the MRP there's no ground, so limited shadow use anyway.