Window alpha transparency support

[valpackett]

Is your feature request related to a problem? Please describe.
Currently, windows are hardcoded to be opaque:

config.composite_alpha_mode = hal::window::CompositeAlphaMode::OPAQUE;

so the a in wgpu::RenderPassColorAttachmentDescriptor clear_color doesn't do anything.

Describe the solution you'd like
There should be support for the transparent modes. In particular, premultiplied mode should be handled, as it's the only one supported at least on my machine (using Mesa RADV driver for Radeon).

Additional context
Hardcoding ::PREMULTIPLIED makes transparency work, but I actually have to multiply the colors by the alpha:

[kvark]

I would be fine with exposing the composite alpha mode

[valpackett]

As in the whole CompositeAlphaMode choice? ah good old "let the user handle it" :)

Swapchain creation is public API so this affects webgpu-headers…

[kvark]

Is there an alternative to telling users what modes are available, and letting them select one?
I'd be happy to consider one!

As for webgpu-headers - you are right, we'll need to expose it there as well, just like we expose the present modes. cc @Kangz

[valpackett]

I was thinking "pick the most common mode and handle conversions" but yeah, exposing everything to the user would be much easier.

[Kangz]

Can we have a brief investigation of what the compositing modes are and how they translate to the various APIs? I worry for example that on macOS the compositing mode might be a part of the CALayer so we'd be mutating that.

Also are there any guarantees that we can support all compositing modes on all platforms, or does for example Vulkan give little guarantees so we need to have queries in webgpu.h for it?

[kvark]

Can we have a brief investigation of what the compositing modes are and how they translate to the various APIs?

Agreed, would be great to have it!

I worry for example that on macOS the compositing mode might be a part of the CALayer so we'd be mutating that.

Pretty sure we are already mutating CALayer properties when you are re-configuring the swapchain, since we set the size and the vsync, etc

Also are there any guarantees that we can support all compositing modes on all platforms

No

[atsuzaki]

Any updates on this?

[grovesNL]

@atsuzaki I don't think anyone has been looking into this yet (unless @myfreeweb has a branch somewhere?), but please feel free to start looking into it if you'd like.

An investigation into how to handle this on each backend and/or prototyping the changes with one or more backends would be great.

[valpackett]

I just have a hardcoded premultiplied mode, nothing smarter, no API or anything

[nyxtom]

I've made a similar change with the PREMULTIPLIED mode but it doesn't seem to work in macOS. Not entirely sure what's missing. Presumably everything needed would be in place (such as color states and having a clear color with an alpha (plus the change in `swap_chain.rs`)

    let window = WindowBuilder::new()
        .with_title("Rust by Example: WGPU!")
        .with_decorations(false)
        .with_transparent(true)
        .build(&event_loop)
        .unwrap();

            color_states: &[
                wgpu::ColorStateDescriptor {
                    format: sc_desc.format,
                    color_blend: wgpu::BlendDescriptor {
                        src_factor: wgpu::BlendFactor::SrcAlpha,
                        dst_factor: wgpu::BlendFactor::OneMinusSrcAlpha,
                        operation: wgpu::BlendOperation::Add
                    },
                    alpha_blend: wgpu::BlendDescriptor {
                        src_factor: wgpu::BlendFactor::One,
                        dst_factor: wgpu::BlendFactor::OneMinusSrcAlpha,
                        operation: wgpu::BlendOperation::Add
                    },
                    //color_blend: wgpu::BlendDescriptor::REPLACE,
                    //alpha_blend: wgpu::BlendDescriptor::REPLACE,
                    write_mask: wgpu::ColorWrite::ALL
                }
            ]    

            let mut render_pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
                color_attachments: &[
                    wgpu::RenderPassColorAttachmentDescriptor {
                        attachment: &frame.view,
                        resolve_target: None,
                        ops: wgpu::Operations {
                            load: wgpu::LoadOp::Clear(wgpu::Color {
                                r: 0.005,
                                g: 0.005,
                                b: 0.01,
                                a: 0.5
                            }),
                            store: true
                        }
                    }
                ],
                depth_stencil_attachment: None
            });

    config.composite_alpha_mode = hal::window::CompositeAlphaMode::PREMULTIPLIED;

[kvark]

gfx-rs Metal backend needs to handle this better, I guess.

[kvark]

@nyxtom Metal transparency support is coming in gfx-rs/gfx#3561

[ryanw]

Transparent windows don't work for me either. Using Linux and Xorg. I ran the Vulkan validation layers and see the error below. My GPU is an RTX 2080 Ti, and transparent windows work with OpenGL things. Seems a little weird that the only supported mode on my GPU is VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR.

vulkaninfo says it only supports opaque, so perhaps it's a driver issue of some kind. Maybe it'd work with XCB instead of Xlib?

supportedCompositeAlpha: count = 1
	COMPOSITE_ALPHA_OPAQUE_BIT_KHR

VUID-VkSwapchainCreateInfoKHR-compositeAlpha-01280(ERROR / SPEC): msgNum: -1341842926 - Validation Error: [ VUID-VkSwapchainCreateInfoKHR-compositeAlpha-01280 ] Object 0: handle = 0x55f648593ab8, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xb0051a12 | vkCreateSwapchainKHR() called with a non-supported pCreateInfo->compositeAlpha (i.e. VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR). Supported values are:
VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR
. The Vulkan spec states: compositeAlpha must be one of the bits present in the supportedCompositeAlpha member of the VkSurfaceCapabilitiesKHR structure returned by vkGetPhysicalDeviceSurfaceCapabilitiesKHR for the surface (https://vulkan.lunarg.com/doc/view/1.2.162.0/linux/1.2-extensions/vkspec.html#VUID-VkSwapchainCreateInfoKHR-compositeAlpha-01280)
Objects: 1
[0] 0x55f648593ab8, type: 3, name: NULL

[LoganDark]

Still interested in a solution to this

[coderedart]

just wanted to express interest too.

just like ryanw, transparency doesn't work for me either on linux/X11 with 1070Ti.
Glutin's transparency example works flawlessly too. if i run vulkaninfo i'm getting
supportedCompositeAlpha: count = 1 COMPOSITE_ALPHA_OPAQUE_BIT_KHR
.

[coderedart]

just checking, but is this repo still the main one? i heard the repos were moved around. i will go make a new issue for transparency, if there's a new repo somewhere.

[coderedart]

just bumping here as its been a few months. but transparency in windows 10 is working. tested on

Edition	Windows 10 Pro
Version	21H1
OS build	19043.1348
GPU: 1070 ti

and wgpu master branch by just changing clear color to [0.0, 0.0, 0.0, 0.0] in the Load operation, and creating winit window with transparency enabled and making sure to have blend state to ALPHA_BLENDING.

bunny mark example

[coderedart]

I might have realized this too late. but is there actually a point in exposing CompositeAlphaMode ?
because transparency works right now. and the mode is still hardcoded as Opaque in wgpu (using master branch). atleast on windows or linux, It seems as long as i write the Alpha channel to framebuffer, and create the window with transparency enabled (glfw / winit tested), it works normally.

[valpackett]

@coderedart is this Vulkan or D3D12? Have you only tested on nvidia GPUs?

If this is Vulkan, something in your stack definitely violates the spec:

VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR: The alpha component, if it exists, of the images is ignored in the compositing process. Instead, the image is treated as if it has a constant alpha of 1.0.

this clearly says that in opaque mode, there MUST NOT be any transparency.

It works correctly on Mesa — [0.0, 0.0, 0.0, 0.0] is treated as [0.0, 0.0, 0.0, 1.0] in opaque mode so I get a black background.

[coderedart]

I am making an egui app and used transparency on both windows and linux (1070ti) without any issues. I just cloned and ran the hello-triangle example from wgpu repo by just changing the clear color and enabling the transparency option for window, and it works perfectly.

just wanted to express interest too.

just like ryanw, transparency doesn't work for me either on linux/X11 with 1070Ti. Glutin's transparency example works flawlessly too. if i run vulkaninfo i'm getting supportedCompositeAlpha: count = 1 COMPOSITE_ALPHA_OPAQUE_BIT_KHR .

as mentioned previously, i only have opaque bit mentioned in driver, so i thought vulkan drivers don't really care about niche things like this. and wgpu source definitely sets the opaque bit.

I will check on windows with dx backend and report back.
EDIT: windows 11 + 1070ti latest drivers
vulkan configurator shows vulkan info with only opaque bit.
wgpu hello-triangle works with transparency on vulkan backend. but dx12 just has black instead of transparency. dx11/gl backends just crash without starting.

SECOND EDIT:
I had a laptop nearby with Amd A12-9720p cpu and arch installed on it. hello-triangle vulkan transparency works.
RADV Carrizo is the driver name. vulkan api 1.2.195 version.
the special thing to note is that it had both Opaque and Inherit bits available in the vulkaninfo.

in all these cases, i just changed the window builder to use transparency and renderpass clear color to transparent contant. didn't touch any other code, and just ran cargo run --example hello-triangle. on windows though, the whole screen was going blank for a second, so i had to separate out hello-triangle example into a normal cargo new project and it worked fine then. also, i expected dx would be the backend on windows 11, but it seems wgpu chooses vulkan

[valpackett]

dx12 just has black instead of transparency

Yep, so D3D12 seems to be working correctly. (@coderedart could you please confirm that you get transparency on DX12 if you change the composite_alpha_mode?)

---

Looking at open source WSI implementations:

• Mesa Wayland:
  • always reports VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR | VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR capabilities
  • whether you have set VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR determines whether you get a surface format with alpha
• Mesa X11:
  • reports VK_COMPOSITE_ALPHA_INHERIT_BIT_KHR | VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR or VK_COMPOSITE_ALPHA_INHERIT_BIT_KHR | VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR capabilities depending on whether the visual has bits left for alpha
  • does not seem to use it for anything
    • the argument is passed to wsi_x11_get_dri3_modifiers but not used there :D
• Mesa Win32 (yeah that's a thing):
  • always reports VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR | VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR capabilities
  • does not seem to use it for anything
• MoltenVK:
  • reports VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR | VK_COMPOSITE_ALPHA_POST_MULTIPLIED_BIT_KHR | VK_COMPOSITE_ALPHA_INHERIT_BIT_KHR capabilites
    • what in the hell is post-multiplied?! Think different I guess >_<
  • sets an opaque flag on the Metal layer if VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR is set

---

Conclusion: the opaque bit actually works correctly on Wayland and macOS (MoltenVK), and the similar bit seems to work on D3D12, but many other Vulkan implementations ignore it and INCORRECTLY let transparency work with VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR. I have raised a Mesa issue about X11.

[coderedart]

@unrelentingtech

none of the pre/post composite modes work on windows dx12 backend. and all of them work with transparency on vulkan backend (but i do get validation errors saying only opaque bit is supported, and i'm using unsupported bit with pre or post multiplied).

the dx12 backend just crashes

[2022-02-09T01:26:28Z ERROR wgpu_hal::dx12] SwapChain creation error: 0x887A0001
[2022-02-09T01:26:28Z ERROR wgpu_core::device] surface configuration failed: swap chain creation
thread 'main' panicked at 'Error in Surface::configure: invalid surface', C:\Users\red\source\repos\wgpu\wgpu\src\backend\direct.rs:231:9
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c\/library\std\src\panicking.rs:517
   1: std::panicking::begin_panic_fmt
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c\/library\std\src\panicking.rs:460
   2: wgpu::backend::direct::Context::handle_error_fatal<enum$<wgpu_core::present::ConfigureSurfaceError> >
             at C:\Users\coder\source\repos\wgpu\wgpu\src\backend\direct.rs:231
   3: wgpu::backend::direct::impl$3::surface_configure
             at C:\Users\coder\source\repos\wgpu\wgpu\src\backend\direct.rs:938
   4: wgpu::Surface::configure
             at C:\Users\coder\source\repos\wgpu\wgpu\src\lib.rs:3181
   5: hello_triangle::run::generator$0
             at .\main.rs:78
   6: core::future::from_generator::impl$1::poll<hello_triangle::run::generator$0>
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c\library\core\src\future\mod.rs:80
   7: pollster::block_on<core::future::from_generator::GenFuture<hello_triangle::run::generator$0> >
             at C:\Users\coder\.cargo\registry\src\github.com-1ecc6299db9ec823\pollster-0.2.5\src\lib.rs:125
   8: hello_triangle::main
             at .\main.rs:145
   9: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c\library\core\src\ops\function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: process didn't exit successfully: `target\debug\hello-triangle.exe` (exit code: 101)

OS:

Edition	Windows 11 Pro
Version	21H2
Installed on	‎27-‎01-‎2022
OS build	22000.493
Experience	Windows Feature Experience Pack 1000.22000.493.0

nvidia driver version: 511.23

I have no idea if there's a dx12 equivalent of vulkaninfo which i can use to check supported CompositeModes.

I hope mesa don't fix the bug xD this issue has been here for a long time, and if they fix it (especially as nvidia doesn't expose any other composite modes), there's no way to get transparency for vulkan now. I gotta go back to opengl 😭

[valpackett]

there's no way to get transparency for vulkan now

No, the temporary way to get transparency for now is to just patch wgpu like I've been doing, and the real way is to make wgpu expose composite_alpha_mode to the user.

[kvark]

Thank you so much for investigating this and filing an upstream issue, @unrelentingtech ! Looking forward to hear from Mesa folks on it.

[valpackett]

@kvark in any case the "driver bug" is only that some drivers allow transparency when they shouldn't, it's not that big of a deal, just something to understand when people are saying "transparency already works".

The original issue here is still that wgpu should give us access to this setting.

[kvark]

Yes, I think we are aligned here. Just need a PR.

[ElhamAryanpur]

hey folks! How's the progress on this topic?

[coderedart]

I think this can be closed. we now have CompositeAlphaMode for wgpu Surface.

[teoxoy]

Fixed by #2836

[dhardy]

Two things I don't understand about the solution #2836:

1. Portable applications which want transparency are supposed to check which of PreMultiplied, PostMultiplied and Inherit are supported, pick their preference, then output either pre-multiplied or un-multiplied textures accordingly?
2. Portable applications need platform-specific code to know what Inherit means?

The status quo seems like a half-solution:

• Ideally, an application could choose between Opaque, PreMultiplied and PostMultiplied, and Winit will convert if this preference is not available
• Ideally, Inherit would not be needed; otherwise at least there should be better documentation of how to determine/configure transparency
• Otherwise, it would be good to have documentation of which CompositeAlphaModes are supported where.

[teoxoy]

@dhardy could you open a new issue with your thoughts to improve the status quo?
I'm not too familiar with this area at the moment.
Thank you!