[Merged by Bors] - Use prepass shaders for shadows

[geieredgar]

Objective

• Fixes Transparent base_color_texture with AlphaMode::Mask should make shadow casters transparent #4372.

Solution

• Use the prepass shaders for the shadow passes.
• Move DEPTH_CLAMP_ORTHO from ShadowPipelineKey to MeshPipelineKey and the associated clamp operation from depth.wgsl to prepass.wgsl.
• Remove depth.wgsl .
• Replace ShadowPipeline with ShadowSamplers.

Instead of running the custom ShadowPipeline we run the PrepassPipeline with the DEPTH_PREPASS flag and additionally the DEPTH_CLAMP_ORTHO flag for directional lights as well as the ALPHA_MASK flag for materials that use AlphaMode::Mask(_).

[superdump]

I think you need this too:

diff --git a/crates/bevy_pbr/src/prepass/mod.rs b/crates/bevy_pbr/src/prepass/mod.rs
index f4f75f7a0..87b471721 100644
--- a/crates/bevy_pbr/src/prepass/mod.rs
+++ b/crates/bevy_pbr/src/prepass/mod.rs
@@ -214,6 +214,9 @@ where
             "MAX_CASCADES_PER_LIGHT".to_string(),
             MAX_CASCADES_PER_LIGHT as i32,
         ));
+        if key.mesh_key.contains(MeshPipelineKey::DEPTH_CLAMP_ORTHO) {
+            shader_defs.push("DEPTH_CLAMP_ORTHO".into());
+        }
 
         if layout.contains(Mesh::ATTRIBUTE_UV_0) {
             shader_defs.push("VERTEX_UVS".into());

Without it, if you run the shadow_biases example and look down and to the left, the centres of the shadows will disappear as the spheres move out of view. The depth clamping is meant to work around that problem. The above hunk adds it to the prepass specialisation.

[geieredgar]

Thank you for catching that! Fixed it.

[IceSentry]

Does this work correctly in webgl?

Also, does it handle the case where someone hasn't enabled the DepthPrepass on their camera? I didn't see any code checking for this or enabling it by default.

Oh sorry, I misunderstood, you reused the prepass pipeline/shaders, not the textures generated by the prepass

[pkupper]

Until we get support for transparent shadow maps, would it be reasonable to treat Materials with AlphaMode::Blend as if they had AlphaMode::Mask(0.5) in the context of shadow mapping? I have noticed, that many third party 3d assets generally tend to use alpha blending over alpha masking for things like foliage, even if they don't really make good use of it (maybe because alpha blending can anti-alias the edges). It's cumbersome having to modify these assets, so assuming AlphaMode::Mask(0.5) for shadows would be a decent approximation.

[geieredgar]

@pkupper Based on the observation that for AlphaMode::Add or AlphaMode::Blend and alpha = 0.0 the blending operation will not change the currently stored color/alpha value in the fragment buffer and the fragment is therefore invisible, I decided to discard fragments with those AlphaModes, if the alpha value is below 0.05. I have chosen that value because that was the threshold below which I couldn't see the cube in the transparency_3d example anymore.

[robtfm]

the code looks good.

i wonder how a custom material would specify that it wants material bindings for the shadow fragment stage? it looks like currently it uses the mesh key alpha mode to decide; maybe that's fair but maybe we should have a custom key bit so that users can set it specifically.

[coreh]

Tried it out and it works pretty well! Left some comments, but looks good!

[coreh]

Not for this PR, but now that we read the material here, perhaps we could also have a per-material flag to disable casting shadows?

[IceSentry]

Oh, that's a pretty neat idea, I guess it would be less ECS friendly though, since now you'd need to query all material instead of just querying for NotShadowCaster.

[IceSentry]

I'm not super familiar with the shadow rendering code, but code looks good to me.

[superdump]

This looks good to me and I’d like to merge it if @robtfm is satisfied too, but you’ve committed changes recently so I’d like to check that you think the PR is ready to merge or if you’re still making changes.

[geieredgar]

Yes, I think the PR is ready to merge.

[superdump]

bors r+

[bors]

Pull request successfully merged into main.

Build succeeded:

• build-and-install-on-iOS
• build-android
• build (macos-latest)
• build (ubuntu-latest)
• build-wasm
• build (windows-latest)
• build-without-default-features (bevy)
• build-without-default-features (bevy_ecs)
• build-without-default-features (bevy_reflect)
• check-compiles
• check-doc
• check-missing-examples-in-docs
• check-missing-features-in-docs
• ci
• markdownlint
• msrv
• run-examples
• run-examples-on-wasm
• run-examples-on-windows-dx12