Vulkan: SDFGI reflections appear to jitter when the camera rotates slowly

[WickedInsignia]

Godot version

4.0 Stable

System information

Windows 11, Nvidia RTX4070ti, AMD Ryzen7700x

Issue description

SDFGI reflections will visibly jitter with camera rotation, especially on materials with a metallic property of 1 and roughness property below 0.5. Will still occur on non-metallic materials.
This is visible at runtime as well as in the editor. It is not visible when panning/moving the camera laterally.

Steps to reproduce

Apply a reflective material to any object and observe the effect while the camera is being moved to "look around" with the mouse.
Alternatively, load MPR and observe reflective cubes while moving camera around.

Minimal reproduction project

SDFGI Reflection Jitter.zip

[Calinou]

This is due to limited precision from the algorithm used to calculate reflections (for performance reasons). I think the SDFGI rework will address this in some way, but I'm not sure how.

[WickedInsignia]

SDFGI is flickering like wild on this particular mesh, for whatever reason. The whole interior is jittering with camera movement. Thought it might be helpful to attach the most severe case I've seen.

LightmapLeakingV2.zip

[krazyjakee]

As I see it, this bug means reflections are unusable in a project where the player has control of the camera. I dug in and got some explanation from devs and ai.

Simplified Explanation:
The Godot engine uses something called a "normal buffer" as part of its rendering process. A buffer is basically a space in memory that stores data temporarily while it's being moved or used. In the context of 3D graphics, a normal buffer specifically holds information about the direction surfaces are facing in the 3D space. This information is crucial for accurately calculating how light should bounce off objects, which in turn affects things like reflections and lighting.

Now, the issue you're facing with the jittery reflections is likely due to the precision of this normal buffer. "Precision" refers to the detail level that the buffer can handle—think of it as the difference between drawing with a fine pen versus a chunky marker. When the precision is low, the engine can't accurately track the exact directions of surfaces, leading to errors in reflections, especially when the camera moves or rotates.

One proposed solution is to use "octahedron mapping" for the normals instead of the current method. Octahedron mapping is a way of storing and processing normal vectors that can reduce errors and improve the precision without needing more memory. It basically allows for a more detailed representation of how surfaces are oriented by wrapping them around an octahedron shape.

However, switching to octahedron mapping isn't straightforward because the normal buffer's data is directly available to users—who might be using this data for their own custom shaders or effects. Changing the format of this buffer would affect everyone who relies on the current system, so it's not a simple switch.

Couldn't user shaders simply recalculate the normal under octahedron mapping?

Yes, it is theoretically possible for a shader to recalculate the normal under octahedron mapping on the fly, although this would be an advanced technique and would involve additional computation that could impact performance, especially on less powerful hardware.

Here's how you could think about implementing this in a shader:

• Retrieve the Low-Precision Normal: The shader would first read the low-precision normal from the buffer.
• Decode to World Space: It would then decode this normal from its current encoding into a world-space normal vector. This means you would reverse whatever method Godot uses to pack the normals into the buffer.
• Re-Encode with Octahedron Mapping: The shader would next re-encode this world-space normal using octahedron normal mapping. This involves projecting the normal onto an octahedron, which is then unfolded into a plane. The result is a two-dimensional representation that can be stored with fewer bits without as much precision loss when it's converted back to a 3D vector.
• Use in Reflection Calculations: Finally, the shader uses this newly encoded high-precision normal for all subsequent calculations related to lighting and reflections.

While this process can potentially reduce the artifacts caused by low-precision normals, it requires that the shader do quite a bit of extra math. Shaders operate on the GPU and are run per-pixel for fragments or per-vertex for vertices, so this could add up to a significant amount of extra calculations, especially for complex scenes or high-resolution outputs.

[clayjohn]

Okay, time to end this conversation. It's going nowhere.

Also worth noting, calling Godot devs "goons" is a violation of our code of conduct and is far below the expectations we have for people engaging in the community. Please be kinder in the future.

[clayjohn]

Following up on this. I did a quick test using octahedral encoded normals to see if it improves the situation in the MRP and it does not. Accordingly, the source of the jitter is not low-precision normals, it is something else that needs to be investigated.

[WickedInsignia]

This issue and #84746 pose the biggest hurdles to VoxelGI's viability in production in my tests. Everything else is rather small but these are the show-stoppers for many use cases.
Thanks for investigating this! I'll make sure to test any developments.