Camera Output Modes, MSAA Writeback, and BlitPipeline (bevyengine#7671)

# Objective

Alternative to bevyengine#7490. I wrote all of the code in this PR, but I have added @robtfm as co-author on commits that build on ideas from bevyengine#7490. I would not have been able to solve these problems on my own without much more time investment and I'm largely just rephrasing the ideas from that PR.

Fixes bevyengine#7435
Fixes bevyengine#7361
Fixes bevyengine#5721

## Solution

This implements the solution I [outlined here](bevyengine#7490 (comment)).


 * Adds "msaa writeback" as an explicit "msaa camera feature" and default to msaa_writeback: true for each camera. If this is true, a camera has MSAA enabled, and it isn't the first camera for the target, add a writeback before the main pass for that camera.
 * Adds a CameraOutputMode, which can be used to configure if (and how) the results of a camera's rendering will be written to the final RenderTarget output texture (via the upscaling node). The `blend_state` and `color_attachment_load_op` are now configurable, giving much more control over how a camera will write to the output texture.
 * Made cameras with the same target share the same main_texture tracker by using `Arc<AtomicUsize>`, which ensures continuity across cameras. This was previously broken / could produce weird results in some cases. `ViewTarget::main_texture()` is now correct in every context.
 * Added a new generic / specializable BlitPipeline, which the new MsaaWritebackNode uses internally. The UpscalingPipelineNode now uses BlitPipeline instead of its own pipeline. We might ultimately need to fork this back out if we choose to add more configurability to the upscaling, but for now this will save on binary size by not embedding the same shader twice.
 * Moved the "camera sorting" logic from the camera driver node to its own system. The results are now stored in the `SortedCameras` resource, which can be used anywhere in the renderer. MSAA writeback makes use of this.

---

## Changelog

- Added `Camera::msaa_writeback` which can enable and disable msaa writeback.
- Added specializable `BlitPipeline` and ported the upscaling node to use this.
- Added SortedCameras, exposing information that was previously internal to the camera driver node.
- Made cameras with the same target share the same main_texture tracker, which ensures continuity across cameras.

crates/bevy_core_pipeline/src/upscaling/upscaling.wgsl → crates/bevy_core_pipeline/src/blit/blit.wgsl

@@ -1,13 +1,11 @@
 #import bevy_core_pipeline::fullscreen_vertex_shader
 
 @group(0) @binding(0)
-var hdr_texture: texture_2d<f32>;
+var in_texture: texture_2d<f32>;
 @group(0) @binding(1)
-var hdr_sampler: sampler;
+var in_sampler: sampler;
 
 @fragment
 fn fs_main(in: FullscreenVertexOutput) -> @location(0) vec4<f32> {
-    let hdr_color = textureSample(hdr_texture, hdr_sampler, in.uv);
-
-    return hdr_color;
+    return textureSample(in_texture, in_sampler, in.uv);
 }

crates/bevy_core_pipeline/src/blit/mod.rs

@@ -0,0 +1,104 @@
+use bevy_app::{App, Plugin};
+use bevy_asset::{load_internal_asset, HandleUntyped};
+use bevy_ecs::prelude::*;
+use bevy_reflect::TypeUuid;
+use bevy_render::{render_resource::*, renderer::RenderDevice, RenderApp};
+
+use crate::fullscreen_vertex_shader::fullscreen_shader_vertex_state;
+
+pub const BLIT_SHADER_HANDLE: HandleUntyped =
+    HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2312396983770133547);
+
+/// Adds support for specialized "blit pipelines", which can be used to write one texture to another.
+pub struct BlitPlugin;
+
+impl Plugin for BlitPlugin {
+    fn build(&self, app: &mut App) {
+        load_internal_asset!(app, BLIT_SHADER_HANDLE, "blit.wgsl", Shader::from_wgsl);
+        let Ok(render_app) = app.get_sub_app_mut(RenderApp) else {
+            return
+        };
+
+        render_app
+            .init_resource::<BlitPipeline>()
+            .init_resource::<SpecializedRenderPipelines<BlitPipeline>>();
+    }
+}
+
+#[derive(Resource)]
+pub struct BlitPipeline {
+    pub texture_bind_group: BindGroupLayout,
+    pub sampler: Sampler,
+}
+
+impl FromWorld for BlitPipeline {
+    fn from_world(render_world: &mut World) -> Self {
+        let render_device = render_world.resource::<RenderDevice>();
+
+        let texture_bind_group =
+            render_device.create_bind_group_layout(&BindGroupLayoutDescriptor {
+                label: Some("blit_bind_group_layout"),
+                entries: &[
+                    BindGroupLayoutEntry {
+                        binding: 0,
+                        visibility: ShaderStages::FRAGMENT,
+                        ty: BindingType::Texture {
+                            sample_type: TextureSampleType::Float { filterable: false },
+                            view_dimension: TextureViewDimension::D2,
+                            multisampled: false,
+                        },
+                        count: None,
+                    },
+                    BindGroupLayoutEntry {
+                        binding: 1,
+                        visibility: ShaderStages::FRAGMENT,
+                        ty: BindingType::Sampler(SamplerBindingType::NonFiltering),
+                        count: None,
+                    },
+                ],
+            });
+
+        let sampler = render_device.create_sampler(&SamplerDescriptor::default());
+
+        BlitPipeline {
+            texture_bind_group,
+            sampler,
+        }
+    }
+}
+
+#[derive(PartialEq, Eq, Hash, Clone, Copy)]
+pub struct BlitPipelineKey {
+    pub texture_format: TextureFormat,
+    pub blend_state: Option<BlendState>,
+    pub samples: u32,
+}
+
+impl SpecializedRenderPipeline for BlitPipeline {
+    type Key = BlitPipelineKey;
+
+    fn specialize(&self, key: Self::Key) -> RenderPipelineDescriptor {
+        RenderPipelineDescriptor {
+            label: Some("blit pipeline".into()),
+            layout: vec![self.texture_bind_group.clone()],
+            vertex: fullscreen_shader_vertex_state(),
+            fragment: Some(FragmentState {
+                shader: BLIT_SHADER_HANDLE.typed(),
+                shader_defs: vec![],
+                entry_point: "fs_main".into(),
+                targets: vec![Some(ColorTargetState {
+                    format: key.texture_format,
+                    blend: key.blend_state,
+                    write_mask: ColorWrites::ALL,
+                })],
+            }),
+            primitive: PrimitiveState::default(),
+            depth_stencil: None,
+            multisample: MultisampleState {
+                count: key.samples,
+                ..Default::default()
+            },
+            push_constant_ranges: Vec::new(),
+        }
+    }
+}

crates/bevy_core_pipeline/src/bloom/mod.rs

@@ -513,7 +513,11 @@ impl FromWorld for BloomPipelines {
                                 dst_factor: BlendFactor::One,
                                 operation: BlendOperation::Add,
                             },
-                            alpha: BlendComponent::REPLACE,
+                            alpha: BlendComponent {
+                                src_factor: BlendFactor::One,
+                                dst_factor: BlendFactor::One,
+                                operation: BlendOperation::Max,
+                            },
                         }),
                         write_mask: ColorWrites::ALL,
                     })],

crates/bevy_core_pipeline/src/core_2d/mod.rs

@@ -7,6 +7,7 @@ pub mod graph {
         pub const VIEW_ENTITY: &str = "view_entity";
     }
     pub mod node {
+        pub const MSAA_WRITEBACK: &str = "msaa_writeback";
         pub const MAIN_PASS: &str = "main_pass";
         pub const BLOOM: &str = "bloom";
         pub const TONEMAPPING: &str = "tonemapping";

crates/bevy_core_pipeline/src/core_3d/mod.rs

@@ -7,6 +7,7 @@ pub mod graph {
         pub const VIEW_ENTITY: &str = "view_entity";
     }
     pub mod node {
+        pub const MSAA_WRITEBACK: &str = "msaa_writeback";
         pub const PREPASS: &str = "prepass";
         pub const MAIN_PASS: &str = "main_pass";
         pub const BLOOM: &str = "bloom";

crates/bevy_core_pipeline/src/lib.rs

@@ -1,9 +1,11 @@
+pub mod blit;
 pub mod bloom;
 pub mod clear_color;
 pub mod core_2d;
 pub mod core_3d;
 pub mod fullscreen_vertex_shader;
 pub mod fxaa;
+pub mod msaa_writeback;
 pub mod prepass;
 pub mod tonemapping;
 pub mod upscaling;
@@ -18,12 +20,14 @@ pub mod prelude {
 }
 
 use crate::{
+    blit::BlitPlugin,
     bloom::BloomPlugin,
     clear_color::{ClearColor, ClearColorConfig},
     core_2d::Core2dPlugin,
     core_3d::Core3dPlugin,
     fullscreen_vertex_shader::FULLSCREEN_SHADER_HANDLE,
     fxaa::FxaaPlugin,
+    msaa_writeback::MsaaWritebackPlugin,
     prepass::{DepthPrepass, NormalPrepass},
     tonemapping::TonemappingPlugin,
     upscaling::UpscalingPlugin,
@@ -52,6 +56,8 @@ impl Plugin for CorePipelinePlugin {
             .add_plugin(ExtractResourcePlugin::<ClearColor>::default())
             .add_plugin(Core2dPlugin)
             .add_plugin(Core3dPlugin)
+            .add_plugin(BlitPlugin)
+            .add_plugin(MsaaWritebackPlugin)
             .add_plugin(TonemappingPlugin)
             .add_plugin(UpscalingPlugin)
             .add_plugin(BloomPlugin)

crates/bevy_core_pipeline/src/msaa_writeback.rs

@@ -0,0 +1,181 @@
+use crate::blit::{BlitPipeline, BlitPipelineKey};
+use bevy_app::{App, Plugin};
+use bevy_ecs::prelude::*;
+use bevy_render::{
+    camera::ExtractedCamera,
+    render_graph::{Node, NodeRunError, RenderGraph, RenderGraphContext, SlotInfo, SlotType},
+    renderer::RenderContext,
+    view::{Msaa, ViewTarget},
+    RenderSet,
+};
+use bevy_render::{render_resource::*, RenderApp};
+
+/// This enables "msaa writeback" support for the `core_2d` and `core_3d` pipelines, which can be enabled on cameras
+/// using [`bevy_render::camera::Camera::msaa_writeback`]. See the docs on that field for more information.
+pub struct MsaaWritebackPlugin;
+
+impl Plugin for MsaaWritebackPlugin {
+    fn build(&self, app: &mut App) {
+        let Ok(render_app) = app.get_sub_app_mut(RenderApp) else {
+            return
+        };
+
+        render_app.add_system(queue_msaa_writeback_pipelines.in_set(RenderSet::Queue));
+        let msaa_writeback_2d = MsaaWritebackNode::new(&mut render_app.world);
+        let msaa_writeback_3d = MsaaWritebackNode::new(&mut render_app.world);
+        let mut graph = render_app.world.resource_mut::<RenderGraph>();
+        if let Some(core_2d) = graph.get_sub_graph_mut(crate::core_2d::graph::NAME) {
+            let input_node = core_2d.input_node().id;
+            core_2d.add_node(
+                crate::core_2d::graph::node::MSAA_WRITEBACK,
+                msaa_writeback_2d,
+            );
+            core_2d.add_node_edge(
+                crate::core_2d::graph::node::MSAA_WRITEBACK,
+                crate::core_2d::graph::node::MAIN_PASS,
+            );
+            core_2d.add_slot_edge(
+                input_node,
+                crate::core_2d::graph::input::VIEW_ENTITY,
+                crate::core_2d::graph::node::MSAA_WRITEBACK,
+                MsaaWritebackNode::IN_VIEW,
+            );
+        }
+
+        if let Some(core_3d) = graph.get_sub_graph_mut(crate::core_3d::graph::NAME) {
+            let input_node = core_3d.input_node().id;
+            core_3d.add_node(
+                crate::core_3d::graph::node::MSAA_WRITEBACK,
+                msaa_writeback_3d,
+            );
+            core_3d.add_node_edge(
+                crate::core_3d::graph::node::MSAA_WRITEBACK,
+                crate::core_3d::graph::node::MAIN_PASS,
+            );
+            core_3d.add_slot_edge(
+                input_node,
+                crate::core_3d::graph::input::VIEW_ENTITY,
+                crate::core_3d::graph::node::MSAA_WRITEBACK,
+                MsaaWritebackNode::IN_VIEW,
+            );
+        }
+    }
+}
+
+pub struct MsaaWritebackNode {
+    cameras: QueryState<(&'static ViewTarget, &'static MsaaWritebackBlitPipeline)>,
+}
+
+impl MsaaWritebackNode {
+    pub const IN_VIEW: &'static str = "view";
+
+    pub fn new(world: &mut World) -> Self {
+        Self {
+            cameras: world.query(),
+        }
+    }
+}
+
+impl Node for MsaaWritebackNode {
+    fn input(&self) -> Vec<SlotInfo> {
+        vec![SlotInfo::new(Self::IN_VIEW, SlotType::Entity)]
+    }
+    fn update(&mut self, world: &mut World) {
+        self.cameras.update_archetypes(world);
+    }
+    fn run(
+        &self,
+        graph: &mut RenderGraphContext,
+        render_context: &mut RenderContext,
+        world: &World,
+    ) -> Result<(), NodeRunError> {
+        let view_entity = graph.get_input_entity(Self::IN_VIEW)?;
+        if let Ok((target, blit_pipeline_id)) = self.cameras.get_manual(world, view_entity) {
+            let blit_pipeline = world.resource::<BlitPipeline>();
+            let pipeline_cache = world.resource::<PipelineCache>();
+            let pipeline = pipeline_cache
+                .get_render_pipeline(blit_pipeline_id.0)
+                .unwrap();
+
+            // The current "main texture" needs to be bound as an input resource, and we need the "other"
+            // unused target to be the "resolve target" for the MSAA write. Therefore this is the same
+            // as a post process write!
+            let post_process = target.post_process_write();
+
+            let pass_descriptor = RenderPassDescriptor {
+                label: Some("msaa_writeback"),
+                // The target's "resolve target" is the "destination" in post_process
+                // We will indirectly write the results to the "destination" using
+                // the MSAA resolve step.
+                color_attachments: &[Some(target.get_color_attachment(Operations {
+                    load: LoadOp::Clear(Default::default()),
+                    store: true,
+                }))],
+                depth_stencil_attachment: None,
+            };
+
+            let bind_group =
+                render_context
+                    .render_device()
+                    .create_bind_group(&BindGroupDescriptor {
+                        label: None,
+                        layout: &blit_pipeline.texture_bind_group,
+                        entries: &[
+                            BindGroupEntry {
+                                binding: 0,
+                                resource: BindingResource::TextureView(post_process.source),
+                            },
+                            BindGroupEntry {
+                                binding: 1,
+                                resource: BindingResource::Sampler(&blit_pipeline.sampler),
+                            },
+                        ],
+                    });
+
+            let mut render_pass = render_context
+                .command_encoder()
+                .begin_render_pass(&pass_descriptor);
+
+            render_pass.set_pipeline(pipeline);
+            render_pass.set_bind_group(0, &bind_group, &[]);
+            render_pass.draw(0..3, 0..1);
+        }
+        Ok(())
+    }
+}
+
+#[derive(Component)]
+pub struct MsaaWritebackBlitPipeline(CachedRenderPipelineId);
+
+fn queue_msaa_writeback_pipelines(
+    mut commands: Commands,
+    pipeline_cache: Res<PipelineCache>,
+    mut pipelines: ResMut<SpecializedRenderPipelines<BlitPipeline>>,
+    blit_pipeline: Res<BlitPipeline>,
+    view_targets: Query<(Entity, &ViewTarget, &ExtractedCamera)>,
+    msaa: Res<Msaa>,
+) {
+    for (entity, view_target, camera) in view_targets.iter() {
+        // only do writeback if writeback is enabled for the camera and this isn't the first camera in the target,
+        // as there is nothing to write back for the first camera.
+        if msaa.samples() > 1 && camera.msaa_writeback && camera.sorted_camera_index_for_target > 0
+        {
+            let key = BlitPipelineKey {
+                texture_format: view_target.main_texture_format(),
+                samples: msaa.samples(),
+                blend_state: None,
+            };
+
+            let pipeline = pipelines.specialize(&pipeline_cache, &blit_pipeline, key);
+            commands
+                .entity(entity)
+                .insert(MsaaWritebackBlitPipeline(pipeline));
+        } else {
+            // This isn't strictly necessary now, but if we move to retained render entity state I don't
+            // want this to silently break
+            commands
+                .entity(entity)
+                .remove::<MsaaWritebackBlitPipeline>();
+        }
+    }
+}