Fix volumetric fog NaN values in textures from starting at a zero Vector2.

[viksl]

Reported issues: #78500

This attempts to fix these issues in the comments: #78500 (comment) and #78500 (comment) it might also fix the original issue there but I can't reproduce it on my side and I also couldn't find anyone else who could apart from the OP and Calinou.

Details:
At this line:

godot/servers/rendering/renderer_rd/shaders/environment/volumetric_fog_process.glsl

Line 308 in 6758a7f

view_pos.xy = (fog_unit_pos.xy * 2.0 - 1.0) * mix(params.fog_frustum_size_begin, params.fog_frustum_size_end, vec2(fog_unit_pos.z));

fog_unit_pos.z can be 0 which leads to the mix mix(params.fog_frustum_size_begin, params.fog_frustum_size_end, vec2(fog_unit_pos.z)) resolving into params.fog_frustum_size_begin which comes from this line:

godot/servers/rendering/renderer_rd/environment/fog.cpp

Line 546 in 6758a7f

fog_near_size = Vector2();

Here the fog_near_size = Vector2(); means it's zero which later on in every light calculation in the shader casues this: normalize(view_pos) to yield INF/-INF which is later used for calculations which are then stored in textures and result into NaN in textures. Due to temporal_reprojection's interpolation with previous frame these values are kept and spread more resulting in black artifacts like these in the video I captured here: https://youtu.be/Vz-fKhp5scQ

To reproduce this I needed to up the Volume Depth in Project Settings and increase Detail Spread in the volumetric fog otherwise it's difficult to reproduce since it can take longer time to happen, since all these result in similar visual glitches as I mentioned earlier this might also fix the original issue, I did manage to kind of reproduce it but had to do some jump arounds so I'm not 100% sure, hopefully @Calinou and @Jonne-G can test this and tell us if this helped the issue as well or if it only helps with the ones I mentioned, please?

As far as I can tell this is as far as the issue can be tracked, so far I've tested on Windows 10, GTX 1660TI and could not reproduce the issues after this fix while before I can reproduce it 100%.

I couldn't find a reason why Vector2() needs to be used for the start so if someone else knows better please feel free to let me know I'm not that far into udnerstanding the whole thing so maybe I'm missing some reasoning and this isn't a suitable fix?

The 0.001 is just a safety, currently frustum can't go below this but just in case it changes in the future for some reason.
(we could maybe also just set it as Vector2(0.01, 0.01); if the MAX(frustum_near_size) is an issue or do a proper calculation whatever that should be (no idea).

This is my first code PR, please have mercy if I'm making any mistakes here, thank you! ^.^

---

Side note:
Also I'm not sure about the use of detail_spread:

godot/servers/rendering/renderer_rd/shaders/environment/volumetric_fog_process.glsl

Line 305 in 6758a7f

fog_unit_pos.z = pow(fog_unit_pos.z, params.detail_spread);

this power pretty much always results in 0 depending on settings - Volume Depth and Detail Spread - for example spread 2.0 -> 9.53674E-07, spread 6.0 -> 0.0000 (you can get vallues with exponents of E-13 and such), it's not directly related to this but it was part of the chain which made me wonder if these values are ok in general?

[clayjohn]

The change seems low-risk to me and makes sense conceptually.

@Calinou Can you test this locally and confirm it fixes the bug? I can't reproduce the original bug on my hardware

[Calinou]

@Calinou Can you test this locally and confirm it fixes the bug? I can't reproduce the original bug on my hardware

This PR doesn't appear to resolve the issue on my end (Linux + NVIDIA):

Edit: This is fixed by #82546.

[viksl]

@Calinou I have no idea how to reproduce that. I even tried different machines with different systems (win 10, win 11, Linux) with different nvidia GPUs. Would you perhaps have an idea about something which could serve as a lead, please?

Could you just for a quick test change the lines in this PR to this please: fog_near_size = Vector2(1.0, 1.0); instead at both places just to see larger value?

[Calinou]

Could you just for a quick test change the lines in this PR to this please: fog_near_size = Vector2(1.0, 1.0); instead at both places just to see larger value?

I get the same issue as before with the fog_near_size changed as you said.

[viksl]

Thank you, though it's unfortunate. I don't know how to reproduce the original issue, the issues thix fixes just result in similar artifacts. I wish I knew what was the difference for the reproduction but the original poster was on windows 10 (just like me) and you are on Linux. We all have nvidia GPUs, I tried with other people on win and linux and nvidia and amd GPUs and no one but you two could reproduce this so far. :/

Anyone has any ideas perhaps?

EDIT: Also just to be clear, it only happens with spotlights and only at these extreme settings correct?

[clayjohn]

Looks good. This should be a low risk merge

[akien-mga]

Thanks!

[YuriSizov]

Cherry-picked for 4.1.3.