Vulkan: soft shadows appear grainy, even on "Ultra" quality setting

[unfa]

Godot version

4.0-dev.20211004/

System information

Manjaro Linux KDE, Radeon RX480 (amdgpu driver)

Issue description

"Ultra":

"Low":

Here's how the shadow settings look in this project:

Steps to reproduce

1. Create a scene with soft shadows
2. Observe

Minimal reproduction project

No response

Bugsquad edit (keywords for easier searching): noisy, noise

[Calinou]

This is expected due to how soft shadows are rendered in the Vulkan renderer. Very soft shadows are difficult to render in real-time rendering engines, so I'd recommend using a lower blur value instead. This dithering pattern allows using a variable level of blur on a per-pixel basis, which allows for PCSS-like shadows and per-light shadow blur values.

Shadow rendering in 3.x uses PCF instead. This algorithm doesn't exhibit noise, but it's much more limited (on top of being slow if you want truly soft shadows).

Before #45326 was merged, it was even worse 🙂

[unfa]

Hmm. I wonder if some post-production step could make it more visually pleasing.

[Calinou]

Hmm. I wonder if some post-production step could make it more visually pleasing.

A while ago, I've tried to plug a real-time GLSL denoiser in the Vulkan renderer, but it's way too slow to be usable. It brought down rendering from 100+ FPS to about 25 FPS.

An alternative would be to use supersampling for 3D rendering, but again, this is very slow.

[clayjohn]

@unfa Do you have the blur setting turned up in the light itself? The images you have there look like there is an unusually high blur radius

[unfa]

@clayjohn Yes, that's at Blur=8 (maximum).

I guess I should just tone it down?

[clayjohn]

@unfa Yes, 8 is an extreme value.

Perhaps we should consider limiting the range to 0-2.

To be fair, the documentation does warn about this already Blurs the edges of the shadow. Can be used to hide pixel artifacts in low-resolution shadow maps. A high value can impact performance, make shadows appear grainy and can cause other unwanted artifacts. Try to keep as near default as possible.

[unfa]

Ok, I did not see that message, I tweaked the settings to achieve a pleasing (albeit grainy) result.

[Calinou]

Perhaps we should consider limiting the range to 0-2.

0-2 might be a tad too limiting until #50506 is merged (which would effectively make it 0-3 with the current scale). If we go forward with #50506, then clamping to 0-2 sounds good.

[Giwayume]

I actually really like the dither effect for games with detailed (realistic) textures, it's practically invisible and looks much better than other high-end shadow implementations I've seen (like Cyberpunk 2077's).

But obviously for games with simplistic flat textures which includes a lot of indie games it stands out like a sore thumb. Would be good to have an alternative that doesn't use a dithering pattern for more simplistic looking games. Those dither patterns are going to turn into a signature of "yep I made this game in Godot 4".

[Calinou]

Would be good to have an alternative that doesn't use a dithering pattern for more simplistic looking games.

You can turn off soft shadow filtering entirely in the project settings, which disables the dithering pattern. It won't look great, but it'll be fast.

An alternative is to do the opposite – if you push the shadow filtering quality all the way up, the dithering pattern should be less noticeable. When using the highest possible shadow filtering quality, you should be able to halve your light Shadow Blur values while getting the same amount of shadow blurriness.

If you really want to disable the dithering pattern but keep soft shadows, perform the following change in

godot/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl

Lines 292 to 298 in c5f4bf2

	// Interleaved Gradient Noise
	// https://www.iryoku.com/next-generation-post-processing-in-call-of-duty-advanced-warfare
	float quick_hash(vec2 pos) {
	const vec3 magic = vec3(0.06711056f, 0.00583715f, 52.9829189f);
	return fract(magic.z * fract(dot(pos, magic.xy)));
	}

diff --git a/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl b/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl
index 4d466342f8..51d26a6437 100644
--- a/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl
+++ b/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl
@@ -292,8 +292,7 @@ void light_compute(vec3 N, vec3 L, vec3 V, float A, vec3 light_color, float atte
 // Interleaved Gradient Noise
 // https://www.iryoku.com/next-generation-post-processing-in-call-of-duty-advanced-warfare
 float quick_hash(vec2 pos) {
-	const vec3 magic = vec3(0.06711056f, 0.00583715f, 52.9829189f);
-	return fract(magic.z * fract(dot(pos, magic.xy)));
+	return 0.5;
 }
 
 float sample_directional_pcf_shadow(texture2D shadow, vec2 shadow_pixel_size, vec4 coord) {

Then recompile Godot:

[catboy-catfish]

Would be good to have an alternative that doesn't use a dithering pattern for more simplistic looking games.

You can turn off soft shadow filtering entirely in the project settings, which disables the dithering pattern. It won't look great, but it'll be fast.

An alternative is to do the opposite – if you push the shadow filtering quality all the way up, the dithering pattern should be less noticeable. When using the highest possible shadow filtering quality, you should be able to halve your light Shadow Blur values while getting the same amount of shadow blurriness.

If you really want to disable the dithering pattern but keep soft shadows, perform the following change in

godot/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl

Lines 292 to 298 in c5f4bf2

	// Interleaved Gradient Noise
	// https://www.iryoku.com/next-generation-post-processing-in-call-of-duty-advanced-warfare
	float quick_hash(vec2 pos) {
	const vec3 magic = vec3(0.06711056f, 0.00583715f, 52.9829189f);
	return fract(magic.z * fract(dot(pos, magic.xy)));
	}

diff --git a/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl b/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl
index 4d466342f8..51d26a6437 100644
--- a/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl
+++ b/servers/rendering/renderer_rd/shaders/scene_forward_lights_inc.glsl
@@ -292,8 +292,7 @@ void light_compute(vec3 N, vec3 L, vec3 V, float A, vec3 light_color, float atte
 // Interleaved Gradient Noise
 // https://www.iryoku.com/next-generation-post-processing-in-call-of-duty-advanced-warfare
 float quick_hash(vec2 pos) {
-	const vec3 magic = vec3(0.06711056f, 0.00583715f, 52.9829189f);
-	return fract(magic.z * fract(dot(pos, magic.xy)));
+	return 0.5;
 }
 
 float sample_directional_pcf_shadow(texture2D shadow, vec2 shadow_pixel_size, vec4 coord) {

Then recompile Godot:

Are there any benefits to using quick_hash for the shadows instead of just giving them a value of 0.5 like this?

[Calinou]

Are there any benefits to using quick_hash for the shadows instead of just giving them a value of 0.5 like this?

Yes, dithering helps make the shadow aliasing (i.e. banding patterns in the shadowmap texture) less noticeable.

[catboy-catfish]

Yes, dithering helps make the shadow aliasing (i.e. banding patterns in the shadowmap texture) less noticeable.

Yeah, actually, I had to learn this the hard way before you commented. I thought it would be better because of no dithering, but without it it's just a bunch of hard shadows mixed together. There's also stair-stepping artifacts on the shadows (because they're not being blurred in any way).