Frustum culling (#2861)

# Objective

Implement frustum culling for much better performance on more complex scenes. With the Amazon Lumberyard Bistro scene, I was getting roughly 15fps without frustum culling and 60+fps with frustum culling on a MacBook Pro 16 with i9 9980HK 8c/16t CPU and Radeon Pro 5500M.

macOS does weird things with vsync so even though vsync was off, it really looked like sometimes other applications or the desktop window compositor were interfering, but the difference could be even more as I even saw up to 90+fps sometimes.

## Solution

- Until the bevyengine/rfcs#12 RFC is completed, I wanted to implement at least some of the bounding volume functionality we needed to be able to unblock a bunch of rendering features and optimisations such as frustum culling, fitting the directional light orthographic projection to the relevant meshes in the view, clustered forward rendering, etc.
- I have added `Aabb`, `Frustum`, and `Sphere` types with only the necessary intersection tests for the algorithms used. I also added `CubemapFrusta` which contains a `[Frustum; 6]` and can be used by cube maps such as environment maps, and point light shadow maps.
  - I did do a bit of benchmarking and optimisation on the intersection tests. I compared the [rafx parallel-comparison bitmask approach](https://github.com/aclysma/rafx/blob/c91bd5fcfdfa3f4d1b43507c32d84b94ffdf1b2e/rafx-visibility/src/geometry/frustum.rs#L64-L92) with a naïve loop that has an early-out in case of a bounding volume being outside of any one of the `Frustum` planes and found them to be very similar, so I chose the simpler and more readable option. I also compared using Vec3 and Vec3A and it turned out that promoting Vec3s to Vec3A improved performance of the culling significantly due to Vec3A operations using SIMD optimisations where Vec3 uses plain scalar operations.
- When loading glTF models, the vertex attribute accessors generally store the minimum and maximum values, which allows for adding AABBs to meshes loaded from glTF for free.
- For meshes without an AABB (`PbrBundle` deliberately does not have an AABB by default), a system is executed that scans over the vertex positions to find the minimum and maximum values along each axis. This is used to construct the AABB.
- The `Frustum::intersects_obb` and `Sphere::insersects_obb` algorithm is from Foundations of Game Engine Development 2: Rendering by Eric Lengyel. There is no OBB type, yet, rather an AABB and the model matrix are passed in as arguments. This calculates a 'relative radius' of the AABB with respect to the plane normal (the plane normal in the Sphere case being something I came up with as the direction pointing from the centre of the sphere to the centre of the AABB) such that it can then do a sphere-sphere intersection test in practice.
- `RenderLayers` were copied over from the current renderer.
- `VisibleEntities` was copied over from the current renderer and a `CubemapVisibleEntities` was added to support `PointLight`s for now. `VisibleEntities` are added to views (cameras and lights) and contain a `Vec<Entity>` that is populated by culling/visibility systems that run in PostUpdate of the app world, and are iterated over in the render world for, for example, queuing up meshes to be drawn by lights for shadow maps and the main pass for cameras.
- `Visibility` and `ComputedVisibility` components were added. The `Visibility` component is user-facing so that, for example, the entity can be marked as not visible in an editor. `ComputedVisibility` on the other hand is the result of the culling/visibility systems and takes `Visibility` into account. So if an entity is marked as not being visible in its `Visibility` component, that will skip culling/visibility intersection tests and just mark the `ComputedVisibility` as false.
- The `ComputedVisibility` is used to decide which meshes to extract.
- I had to add a way to get the far plane from the `CameraProjection` in order to define an explicit far frustum plane for culling. This should perhaps be optional as it is not always desired and in that case, testing 5 planes instead of 6 is a performance win.

I think that's about all. I discussed some of the design with @cart on Discord already so hopefully it's not too far from being mergeable. It works well at least. 😄

pipelined/bevy_gltf2/src/loader.rs

@@ -5,14 +5,15 @@ use bevy_asset::{
 use bevy_core::Name;
 use bevy_ecs::world::World;
 use bevy_log::warn;
-use bevy_math::Mat4;
+use bevy_math::{Mat4, Vec3};
 use bevy_pbr2::{PbrBundle, StandardMaterial};
 use bevy_render2::{
     camera::{
         Camera, CameraPlugin, CameraProjection, OrthographicProjection, PerspectiveProjection,
     },
     color::Color,
     mesh::{Indices, Mesh, VertexAttributeValues},
+    primitives::Aabb,
     texture::{Image, ImageType, TextureError},
 };
 use bevy_scene::Scene;
@@ -528,11 +529,17 @@ fn load_node(
                 let material_asset_path =
                     AssetPath::new_ref(load_context.path(), Some(&material_label));
 
-                parent.spawn_bundle(PbrBundle {
-                    mesh: load_context.get_handle(mesh_asset_path),
-                    material: load_context.get_handle(material_asset_path),
-                    ..Default::default()
-                });
+                let bounds = primitive.bounding_box();
+                parent
+                    .spawn_bundle(PbrBundle {
+                        mesh: load_context.get_handle(mesh_asset_path),
+                        material: load_context.get_handle(material_asset_path),
+                        ..Default::default()
+                    })
+                    .insert(Aabb::from_min_max(
+                        Vec3::from_slice(&bounds.min),
+                        Vec3::from_slice(&bounds.max),
+                    ));
             }
         }
 

pipelined/bevy_pbr2/src/bundle.rs

@@ -1,7 +1,11 @@
 use crate::{DirectionalLight, PointLight, StandardMaterial};
 use bevy_asset::Handle;
 use bevy_ecs::bundle::Bundle;
-use bevy_render2::mesh::Mesh;
+use bevy_render2::{
+    mesh::Mesh,
+    primitives::{CubemapFrusta, Frustum},
+    view::{ComputedVisibility, Visibility, VisibleEntities},
+};
 use bevy_transform::components::{GlobalTransform, Transform};
 
 #[derive(Bundle, Clone)]
@@ -10,6 +14,10 @@ pub struct PbrBundle {
     pub material: Handle<StandardMaterial>,
     pub transform: Transform,
     pub global_transform: GlobalTransform,
+    /// User indication of whether an entity is visible
+    pub visibility: Visibility,
+    /// Algorithmically-computed indication of whether an entity is visible and should be extracted for rendering
+    pub computed_visibility: ComputedVisibility,
 }
 
 impl Default for PbrBundle {
@@ -19,14 +27,41 @@ impl Default for PbrBundle {
             material: Default::default(),
             transform: Default::default(),
             global_transform: Default::default(),
+            visibility: Default::default(),
+            computed_visibility: Default::default(),
         }
     }
 }
 
+#[derive(Clone, Debug, Default)]
+pub struct CubemapVisibleEntities {
+    data: [VisibleEntities; 6],
+}
+
+impl CubemapVisibleEntities {
+    pub fn get(&self, i: usize) -> &VisibleEntities {
+        &self.data[i]
+    }
+
+    pub fn get_mut(&mut self, i: usize) -> &mut VisibleEntities {
+        &mut self.data[i]
+    }
+
+    pub fn iter(&self) -> impl DoubleEndedIterator<Item = &VisibleEntities> {
+        self.data.iter()
+    }
+
+    pub fn iter_mut(&mut self) -> impl DoubleEndedIterator<Item = &mut VisibleEntities> {
+        self.data.iter_mut()
+    }
+}
+
 /// A component bundle for "point light" entities
 #[derive(Debug, Bundle, Default)]
 pub struct PointLightBundle {
     pub point_light: PointLight,
+    pub cubemap_visible_entities: CubemapVisibleEntities,
+    pub cubemap_frusta: CubemapFrusta,
     pub transform: Transform,
     pub global_transform: GlobalTransform,
 }
@@ -35,6 +70,8 @@ pub struct PointLightBundle {
 #[derive(Debug, Bundle, Default)]
 pub struct DirectionalLightBundle {
     pub directional_light: DirectionalLight,
+    pub frustum: Frustum,
+    pub visible_entities: VisibleEntities,
     pub transform: Transform,
     pub global_transform: GlobalTransform,
 }

pipelined/bevy_pbr2/src/lib.rs

@@ -18,8 +18,10 @@ use bevy_render2::{
     render_graph::RenderGraph,
     render_phase::{sort_phase_system, AddRenderCommand, DrawFunctions},
     render_resource::{Shader, SpecializedPipelines},
+    view::VisibilitySystems,
     RenderApp, RenderStage,
 };
+use bevy_transform::TransformSystem;
 
 pub mod draw_3d_graph {
     pub mod node {
@@ -49,20 +51,53 @@ impl Plugin for PbrPlugin {
             .init_resource::<AmbientLight>()
             .init_resource::<DirectionalLightShadowMap>()
             .init_resource::<PointLightShadowMap>()
-            .init_resource::<AmbientLight>();
+            .init_resource::<AmbientLight>()
+            .add_system_to_stage(
+                CoreStage::PostUpdate,
+                update_directional_light_frusta
+                    .label(SimulationLightSystems::UpdateDirectionalLightFrusta)
+                    .after(TransformSystem::TransformPropagate),
+            )
+            .add_system_to_stage(
+                CoreStage::PostUpdate,
+                update_point_light_frusta
+                    .label(SimulationLightSystems::UpdatePointLightFrusta)
+                    .after(TransformSystem::TransformPropagate),
+            )
+            .add_system_to_stage(
+                CoreStage::PostUpdate,
+                check_light_visibility
+                    .label(SimulationLightSystems::CheckLightVisibility)
+                    .after(TransformSystem::TransformPropagate)
+                    .after(VisibilitySystems::CalculateBounds)
+                    .after(SimulationLightSystems::UpdateDirectionalLightFrusta)
+                    .after(SimulationLightSystems::UpdatePointLightFrusta)
+                    // NOTE: This MUST be scheduled AFTER the core renderer visibility check
+                    // because that resets entity ComputedVisibility for the first view
+                    // which would override any results from this otherwise
+                    .after(VisibilitySystems::CheckVisibility),
+            );
 
         let render_app = app.sub_app(RenderApp);
         render_app
             .add_system_to_stage(RenderStage::Extract, render::extract_meshes)
-            .add_system_to_stage(RenderStage::Extract, render::extract_lights)
+            .add_system_to_stage(
+                RenderStage::Extract,
+                render::extract_lights.label(RenderLightSystems::ExtractLights),
+            )
             .add_system_to_stage(
                 RenderStage::Prepare,
                 // this is added as an exclusive system because it contributes new views. it must run (and have Commands applied)
                 // _before_ the `prepare_views()` system is run. ideally this becomes a normal system when "stageless" features come out
-                render::prepare_lights.exclusive_system(),
+                render::prepare_lights
+                    .exclusive_system()
+                    .label(RenderLightSystems::PrepareLights),
             )
             .add_system_to_stage(RenderStage::Queue, render::queue_meshes)
-            .add_system_to_stage(RenderStage::Queue, render::queue_shadows)
+            .add_system_to_stage(
+                RenderStage::Queue,
+                render::queue_shadows.label(RenderLightSystems::QueueShadows),
+            )
             .add_system_to_stage(RenderStage::Queue, render::queue_shadow_view_bind_group)
             .add_system_to_stage(RenderStage::Queue, render::queue_transform_bind_group)
             .add_system_to_stage(RenderStage::PhaseSort, sort_phase_system::<Shadow>)

pipelined/bevy_pbr2/src/light.rs

@@ -1,4 +1,14 @@
-use bevy_render2::{camera::OrthographicProjection, color::Color};
+use bevy_ecs::prelude::*;
+use bevy_math::Mat4;
+use bevy_render2::{
+    camera::{CameraProjection, OrthographicProjection},
+    color::Color,
+    primitives::{Aabb, CubemapFrusta, Frustum, Sphere},
+    view::{ComputedVisibility, RenderLayers, Visibility, VisibleEntities, VisibleEntity},
+};
+use bevy_transform::components::GlobalTransform;
+
+use crate::{CubeMapFace, CubemapVisibleEntities, CUBE_MAP_FACES};
 
 /// A light that emits light in all directions from a central point.
 ///
@@ -156,3 +166,177 @@ impl Default for AmbientLight {
 pub struct NotShadowCaster;
 /// Add this component to make a `Mesh` not receive shadows
 pub struct NotShadowReceiver;
+
+#[derive(Debug, Hash, PartialEq, Eq, Clone, SystemLabel)]
+pub enum SimulationLightSystems {
+    UpdateDirectionalLightFrusta,
+    UpdatePointLightFrusta,
+    CheckLightVisibility,
+}
+
+pub fn update_directional_light_frusta(
+    mut views: Query<(&GlobalTransform, &DirectionalLight, &mut Frustum)>,
+) {
+    for (transform, directional_light, mut frustum) in views.iter_mut() {
+        let view_projection = directional_light.shadow_projection.get_projection_matrix()
+            * transform.compute_matrix().inverse();
+        *frustum = Frustum::from_view_projection(
+            &view_projection,
+            &transform.translation,
+            &transform.back(),
+            directional_light.shadow_projection.far(),
+        );
+    }
+}
+
+pub fn update_point_light_frusta(
+    mut views: Query<(&GlobalTransform, &PointLight, &mut CubemapFrusta)>,
+) {
+    let projection = Mat4::perspective_infinite_reverse_rh(std::f32::consts::FRAC_PI_2, 1.0, 0.1);
+    let view_rotations = CUBE_MAP_FACES
+        .iter()
+        .map(|CubeMapFace { target, up }| GlobalTransform::identity().looking_at(*target, *up))
+        .collect::<Vec<_>>();
+
+    for (transform, point_light, mut cubemap_frusta) in views.iter_mut() {
+        // ignore scale because we don't want to effectively scale light radius and range
+        // by applying those as a view transform to shadow map rendering of objects
+        // and ignore rotation because we want the shadow map projections to align with the axes
+        let view_translation = GlobalTransform::from_translation(transform.translation);
+        let view_backward = transform.back();
+
+        for (view_rotation, frustum) in view_rotations.iter().zip(cubemap_frusta.iter_mut()) {
+            let view = view_translation * *view_rotation;
+            let view_projection = projection * view.compute_matrix().inverse();
+
+            *frustum = Frustum::from_view_projection(
+                &view_projection,
+                &transform.translation,
+                &view_backward,
+                point_light.range,
+            );
+        }
+    }
+}
+
+pub fn check_light_visibility(
+    mut point_lights: Query<(
+        &PointLight,
+        &GlobalTransform,
+        &CubemapFrusta,
+        &mut CubemapVisibleEntities,
+        Option<&RenderLayers>,
+    )>,
+    mut directional_lights: Query<
+        (&Frustum, &mut VisibleEntities, Option<&RenderLayers>),
+        With<DirectionalLight>,
+    >,
+    mut visible_entity_query: Query<
+        (
+            Entity,
+            &Visibility,
+            &mut ComputedVisibility,
+            Option<&RenderLayers>,
+            Option<&Aabb>,
+            Option<&GlobalTransform>,
+        ),
+        Without<NotShadowCaster>,
+    >,
+) {
+    // Directonal lights
+    for (frustum, mut visible_entities, maybe_view_mask) in directional_lights.iter_mut() {
+        visible_entities.entities.clear();
+        let view_mask = maybe_view_mask.copied().unwrap_or_default();
+
+        for (
+            entity,
+            visibility,
+            mut computed_visibility,
+            maybe_entity_mask,
+            maybe_aabb,
+            maybe_transform,
+        ) in visible_entity_query.iter_mut()
+        {
+            if !visibility.is_visible {
+                continue;
+            }
+
+            let entity_mask = maybe_entity_mask.copied().unwrap_or_default();
+            if !view_mask.intersects(&entity_mask) {
+                continue;
+            }
+
+            // If we have an aabb and transform, do frustum culling
+            if let (Some(aabb), Some(transform)) = (maybe_aabb, maybe_transform) {
+                if !frustum.intersects_obb(aabb, &transform.compute_matrix()) {
+                    continue;
+                }
+            }
+
+            computed_visibility.is_visible = true;
+            visible_entities.entities.push(VisibleEntity { entity });
+        }
+
+        // TODO: check for big changes in visible entities len() vs capacity() (ex: 2x) and resize
+        // to prevent holding unneeded memory
+    }
+
+    // Point lights
+    for (point_light, transform, cubemap_frusta, mut cubemap_visible_entities, maybe_view_mask) in
+        point_lights.iter_mut()
+    {
+        for visible_entities in cubemap_visible_entities.iter_mut() {
+            visible_entities.entities.clear();
+        }
+        let view_mask = maybe_view_mask.copied().unwrap_or_default();
+        let light_sphere = Sphere {
+            center: transform.translation,
+            radius: point_light.range,
+        };
+
+        for (
+            entity,
+            visibility,
+            mut computed_visibility,
+            maybe_entity_mask,
+            maybe_aabb,
+            maybe_transform,
+        ) in visible_entity_query.iter_mut()
+        {
+            if !visibility.is_visible {
+                continue;
+            }
+
+            let entity_mask = maybe_entity_mask.copied().unwrap_or_default();
+            if !view_mask.intersects(&entity_mask) {
+                continue;
+            }
+
+            // If we have an aabb and transform, do frustum culling
+            if let (Some(aabb), Some(transform)) = (maybe_aabb, maybe_transform) {
+                let model_to_world = transform.compute_matrix();
+                // Do a cheap sphere vs obb test to prune out most meshes outside the sphere of the light
+                if !light_sphere.intersects_obb(aabb, &model_to_world) {
+                    continue;
+                }
+                for (frustum, visible_entities) in cubemap_frusta
+                    .iter()
+                    .zip(cubemap_visible_entities.iter_mut())
+                {
+                    if frustum.intersects_obb(aabb, &model_to_world) {
+                        computed_visibility.is_visible = true;
+                        visible_entities.entities.push(VisibleEntity { entity });
+                    }
+                }
+            } else {
+                computed_visibility.is_visible = true;
+                for visible_entities in cubemap_visible_entities.iter_mut() {
+                    visible_entities.entities.push(VisibleEntity { entity })
+                }
+            }
+        }
+
+        // TODO: check for big changes in visible entities len() vs capacity() (ex: 2x) and resize
+        // to prevent holding unneeded memory
+    }
+}