Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Render filters on display objects #11702

Merged
merged 6 commits into from
Jun 25, 2023
Merged

Render filters on display objects #11702

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
912149a
wgpu: Don't perform copy inside Surface::apply_filter
Dinnerbone Jun 23, 2023
1feb061
wgpu: Take in a &wgpu::Texture not our own texture as we don't need o…
Dinnerbone Jun 23, 2023
e4a13ad
wgpu: Move run_copy_pipeline to utils
Dinnerbone Jun 23, 2023
fa0d9a9
render: Add filters to BitmapCacheEntry
Dinnerbone Jun 23, 2023
10dde22
tests: Add ColorMatrixFilter visual test
Dinnerbone Jun 23, 2023
f05fe98
core: Add dumb filter list inside debug ui
Dinnerbone Jun 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions core/src/debug_ui/display_object.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -391,6 +391,17 @@ impl DisplayObjectWindow {
ui.label(summary_color_transform(color_transform));
ui.end_row();
});

let filters = object.filters();
if !filters.is_empty() {
CollapsingHeader::new(format!("Filters ({})", filters.len()))
.id_source(ui.id().with("filters"))
.show(ui, |ui| {
for filter in filters {
ui.label(format!("{:?}", filter));
}
});
}
}

pub fn show_position<'gc>(
Expand Down
1 change: 1 addition & 0 deletions core/src/display_object.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -768,6 +768,7 @@ pub fn render_base<'gc>(this: DisplayObject<'gc>, context: &mut RenderContext<'_
handle: handle.clone(),
commands: offscreen_context.commands,
clear: this.opaque_background().unwrap_or_default(),
filters: this.filters(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to be sure: do you think us copying the filters vec (the call does a deep .clone() inside) is okay if dome each time here and in debug code?

Copy link
Contributor Author

@Dinnerbone Dinnerbone Jun 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug code: I don't mind perf losses there so much, but I'd like to change filters to return a ref but that felt out of scope (already is vec today) which would be fine for debug

For render: It needs a clone ideally, I don't like the idea of refs leaking out to render, or things that can be interior mutable. I'd like to end up with a state that rendering can be done entirely on another thread on desktop (maybe web in a few years...) and trying to keep that design in mind, it submits work and moves on

Edit: The size of a filter shouldn't be too big, either!

});
}

Expand Down
1 change: 1 addition & 0 deletions render/src/backend.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ pub struct BitmapCacheEntry {
pub handle: BitmapHandle,
pub commands: CommandList,
pub clear: Color,
pub filters: Vec<Filter>,
}

pub trait RenderBackend: Downcast {
Expand Down
106 changes: 85 additions & 21 deletions render/wgpu/src/backend.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use crate::surface::{LayerRef, Surface};
use crate::target::{MaybeOwnedBuffer, TextureTarget};
use crate::target::{RenderTargetFrame, TextureBufferInfo};
use crate::uniform_buffer::{BufferStorage, UniformBuffer};
use crate::utils::BufferDimensions;
use crate::utils::{run_copy_pipeline, BufferDimensions};
use crate::{
as_texture, format_list, get_backend_names, ColorAdjustments, Descriptors, Error,
QueueSyncHandle, RenderTarget, SwapChainTarget, Texture, Transforms,
Expand Down Expand Up @@ -471,26 +471,69 @@ impl<T: RenderTarget + 'static> RenderBackend for WgpuRenderBackend<T> {
texture.height,
wgpu::TextureFormat::Rgba8Unorm,
);
surface.draw_commands(
RenderTargetMode::ExistingWithColor(
texture.texture.clone(),
wgpu::Color {
if entry.filters.is_empty() {
surface.draw_commands(
RenderTargetMode::ExistingWithColor(
texture.texture.clone(),
wgpu::Color {
r: f64::from(entry.clear.r) / 255.0,
g: f64::from(entry.clear.g) / 255.0,
b: f64::from(entry.clear.b) / 255.0,
a: f64::from(entry.clear.a) / 255.0,
},
),
&self.descriptors,
&self.meshes,
entry.commands,
&mut uniform_buffer,
&mut color_buffer,
&mut uniform_encoder,
&mut draw_encoder,
LayerRef::None,
&mut self.offscreen_texture_pool,
);
} else {
let mut target = surface.draw_commands(
RenderTargetMode::FreshWithColor(wgpu::Color {
r: f64::from(entry.clear.r) / 255.0,
g: f64::from(entry.clear.g) / 255.0,
b: f64::from(entry.clear.b) / 255.0,
a: f64::from(entry.clear.a) / 255.0,
},
),
&self.descriptors,
&self.meshes,
entry.commands,
&mut uniform_buffer,
&mut color_buffer,
&mut uniform_encoder,
&mut draw_encoder,
LayerRef::None,
&mut self.offscreen_texture_pool,
);
}),
&self.descriptors,
&self.meshes,
entry.commands,
&mut uniform_buffer,
&mut color_buffer,
&mut uniform_encoder,
&mut draw_encoder,
LayerRef::None,
&mut self.offscreen_texture_pool,
);
for filter in entry.filters {
target = surface.apply_filter(
&self.descriptors,
&mut draw_encoder,
&mut self.offscreen_texture_pool,
target.color_texture(),
(0, 0),
(target.width(), target.height()),
filter,
);
}
run_copy_pipeline(
&self.descriptors,
target.color_texture().format(),
texture.texture.format(),
target.color_texture().size(),
&texture.texture.create_view(&Default::default()),
target.color_view(),
target.whole_frame_bind_group(&self.descriptors),
target.globals(),
target.color_texture().sample_count(),
&mut draw_encoder,
);
}
}

self.surface.draw_commands_and_copy_to(
Expand Down Expand Up @@ -802,17 +845,38 @@ impl<T: RenderTarget + 'static> RenderBackend for WgpuRenderBackend<T> {
.create_command_encoder(&wgpu::CommandEncoderDescriptor {
label: label.as_deref(),
});
surface.apply_filter(
let applied_filter = surface.apply_filter(
&self.descriptors,
&mut draw_encoder,
&mut self.offscreen_texture_pool,
source_texture,
&source_texture.texture,
source_point,
source_size,
dest_texture,
dest_point,
filter,
);
draw_encoder.copy_texture_to_texture(
wgpu::ImageCopyTexture {
texture: applied_filter.color_texture(),
mip_level: 0,
origin: wgpu::Origin3d { x: 0, y: 0, z: 0 },
aspect: Default::default(),
},
wgpu::ImageCopyTexture {
texture: &dest_texture.texture,
mip_level: 0,
origin: wgpu::Origin3d {
x: dest_point.0,
y: dest_point.1,
z: 0,
},
aspect: Default::default(),
},
wgpu::Extent3d {
width: (applied_filter.width()).min(dest_texture.width - dest_point.0),
height: (applied_filter.height()).min(dest_texture.height - dest_point.1),
depth_or_array_layers: 1,
},
);
let index = target.submit(
&self.descriptors.device,
&self.descriptors.queue,
Expand Down
57 changes: 16 additions & 41 deletions render/wgpu/src/surface.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ use crate::mesh::Mesh;
use crate::surface::commands::{chunk_blends, Chunk, CommandRenderer};
use crate::utils::{remove_srgb, supported_sample_count};
use crate::{
ColorAdjustments, Descriptors, MaskState, Pipelines, PushConstants, Texture, TextureTransforms,
ColorAdjustments, Descriptors, MaskState, Pipelines, PushConstants, TextureTransforms,
Transforms, UniformBuffer, DEFAULT_COLOR_ADJUSTMENTS,
};
use ruffle_render::commands::CommandList;
Expand All @@ -20,7 +20,7 @@ use target::CommandTarget;
use tracing::instrument;
use wgpu::util::DeviceExt;

use self::commands::run_copy_pipeline;
use crate::utils::run_copy_pipeline;

pub use crate::surface::commands::LayerRef;

Expand Down Expand Up @@ -346,13 +346,11 @@ impl Surface {
descriptors: &Descriptors,
draw_encoder: &mut wgpu::CommandEncoder,
texture_pool: &mut TexturePool,
source_texture: &Texture,
source_texture: &wgpu::Texture,
source_point: (u32, u32),
source_size: (u32, u32),
dest_texture: &Texture,
dest_point: (u32, u32),
filter: Filter,
) {
) -> CommandTarget {
let target = match filter {
Filter::ColorMatrixFilter(filter) => self.apply_color_matrix(
descriptors,
Expand Down Expand Up @@ -391,30 +389,7 @@ impl Surface {
// a clear (in case no other draw commands were issued, we still need
// the background clear color applied)
target.ensure_cleared(draw_encoder);

draw_encoder.copy_texture_to_texture(
wgpu::ImageCopyTexture {
texture: target.color_texture(),
mip_level: 0,
origin: wgpu::Origin3d { x: 0, y: 0, z: 0 },
aspect: Default::default(),
},
wgpu::ImageCopyTexture {
texture: &dest_texture.texture,
mip_level: 0,
origin: wgpu::Origin3d {
x: dest_point.0,
y: dest_point.1,
z: 0,
},
aspect: Default::default(),
},
wgpu::Extent3d {
width: (target.width()).min(dest_texture.width - dest_point.0),
height: (target.height()).min(dest_texture.height - dest_point.1),
depth_or_array_layers: 1,
},
)
target
}

#[allow(clippy::too_many_arguments)]
Expand All @@ -423,7 +398,7 @@ impl Surface {
descriptors: &Descriptors,
texture_pool: &mut TexturePool,
draw_encoder: &mut wgpu::CommandEncoder,
source_texture: &Texture,
source_texture: &wgpu::Texture,
source_point: (u32, u32),
source_size: (u32, u32),
filter: &ColorMatrixFilter,
Expand All @@ -443,7 +418,7 @@ impl Surface {
);
let texture_transform =
make_texture_transform(descriptors, source_size, source_point, source_texture);
let source_view = source_texture.texture.create_view(&Default::default());
let source_view = source_texture.create_view(&Default::default());
let bitmap_group = descriptors
.device
.create_bind_group(&wgpu::BindGroupDescriptor {
Expand Down Expand Up @@ -532,7 +507,7 @@ impl Surface {
descriptors: &Descriptors,
texture_pool: &mut TexturePool,
draw_encoder: &mut wgpu::CommandEncoder,
source_texture: &Texture,
source_texture: &wgpu::Texture,
source_point: (u32, u32),
source_size: (u32, u32),
filter: &BlurFilter,
Expand Down Expand Up @@ -569,7 +544,7 @@ impl Surface {

let texture_transform =
make_texture_transform(descriptors, source_size, source_point, source_texture);
let source_view = source_texture.texture.create_view(&Default::default());
let source_view = source_texture.create_view(&Default::default());
for i in 0..2 {
let blur_x = (filter.blur_x.to_f32() - 1.0).max(0.0);
let blur_y = (filter.blur_y.to_f32() - 1.0).max(0.0);
Expand All @@ -578,8 +553,8 @@ impl Surface {
(
&source_view,
texture_transform.as_entire_binding(),
source_texture.width as f32,
source_texture.height as f32,
source_texture.width() as f32,
source_texture.height() as f32,
)
} else {
let previous = &targets[(i - 1) % 2];
Expand Down Expand Up @@ -686,7 +661,7 @@ fn make_texture_transform(
descriptors: &Descriptors,
source_size: (u32, u32),
source_point: (u32, u32),
source_texture: &Texture,
source_texture: &wgpu::Texture,
) -> wgpu::Buffer {
descriptors
.device
Expand All @@ -700,23 +675,23 @@ fn make_texture_transform(
// the full source texture, then the scale factor will be less than 1.
// This will produce U-V coordinates that do not extend to the full [0, 1]
// range, which makes us sample just the source region.
source_size.0 as f32 / source_texture.width as f32,
source_size.0 as f32 / source_texture.width() as f32,
0.0,
0.0,
0.0,
],
[
0.0,
source_size.1 as f32 / source_texture.height as f32,
source_size.1 as f32 / source_texture.height() as f32,
0.0,
0.0,
],
[0.0, 0.0, 1.0, 0.0],
// Offset to 'source_point'. Note that we divide by the full texture size,
// since that's what the UV coordinates are sampling from.
[
source_point.0 as f32 / source_texture.width as f32,
source_point.1 as f32 / source_texture.height as f32,
source_point.0 as f32 / source_texture.width() as f32,
source_point.1 as f32 / source_texture.height() as f32,
0.0,
0.0,
],
Expand Down
Loading