Integration of own DirectX 11 renderer

[llfab]

We are considering to use Avalonia within our .NET 5 based framework. We have our own 3D renderer which is based on DirectX 11. This renderer is currently integrated into WPF. Because WPF is DirectX 9 based, the only option to make our renderer work 100% correct within WPF (unfortunately), was to draw the rendered texture into a Windows Forms surface and use a WindowsFormsHost within WPF tree.
I wonder whether this could be avoided using Avalonia. It would be beneficial to be able to integrate the rendered DirectX 11 texture directly into an Avalonia UI scene graph node. In WPF this would work theoretically (passing the DirectX 11 texture into the DirectX 9 WPF scene graph. However, this works not 100% especially for multiple screens which is why we chose the the stable way with WindowsFormsHost.
The draw back of the WindowsFormsHost workaround ist, that there cannot be any WPF controls overlayed onto the rendered 3D scene (because Windows Forms surfaces are rendered ontop of everything). Hence, a true integration into the Avalonio UI scene graph may bring the benefit of correct overlays and also smooth dynamic resizing.
Of course, this is only targeting Windows target platform.

[BreyerW]

You could lookup OpenGLControlBase and see if you can fit it to work with DX11. If not check if OpenGLBitmap can be retrofitted - bitmap way should allow rendering on separate thread which would lessen overhead of avalonia on your own rendering.

Last possibility would be embedding native window which is pretty much what you are already doing in wpf with downsides you mentioned except you dont have to bring whole win form, all you need is native handle of window (can be obtained via interop or via windowing lib) - and if you really need overlaying controls over your rendering you can simulate it via floating windows with owner window set to make sure they wont go below main window

[llfab]

Understand. Thanks. Do I understand correctly, Avalonia UI scene graph is always rendered using OpenGL?

[BreyerW]

I believe it is opengl es via skia lib actually to make porting to mobile easier although im not 100% certain since im not maintanainer

[grokys]

Skia is our main rendering backend but we have a Direct2D1 backend also: https://github.com/AvaloniaUI/Avalonia/tree/master/src/Windows/Avalonia.Direct2D1

It's not as well maintained as the Skia backend so you may encounter problems but it should work. If this is useful to you, we're looking for a maintainer for it ;)

To enable it, add the Avalonia.Direct2D1 package and call UseDirect2D1() in your AppBuilder setup.

[kekekeks]

I'm planning to add DxgiBitmap class soon-ish. For now the process of integrating a custom DirectX renderer would involve quite a bit of tinkering with ANGLE, Skia and our ICustomDrawingOperation.

[BreyerW]

@kekekeks would be nice if you could abstract away these bitmaps behind something like RenderingBitmap in future since there might be more such bitmaps in future like vulkan, metal and maybe specialized dx12 (this bitmap could use avalonia configuration set on startup to decide which backend to use if not set explicitly during creation) making switching unpleasant and potentially error prone

[amerkoleci]

I think we need to solve in common way the hosting part, game engines uses d3d12, d3d11, vulkan, opengl or metal nowadays, there is need to have common ways of hosting game engine view into Avalonia, I know gritgene is using Avalonia for editor and I plan to do the same, would love to help with PR as well.

[kekekeks]

GritGene is using a custom ISkiaGpu implementation and ICustomDrawingOperation to integrate with our rendering pipeline directly. While that being viable, I'd still prefer the old good render to texture and use as a bitmap approach. For that we need to support platform primitives for surface sharing.

• Windows - DXGI Shared Handle. That allows interop with pretty much any GPU-backed surface since DirectX, OpenGL and Vulkan all have support for DXGI handles.
• macOS - IOSurface. That's the GPU surface sharing API used for cross-process communication. Chrome is using this to transfer frames from renderer processes to the UI process. We are using IOSurface to present our rendering output in CALayer. Both OpenGL and Metal support rendering into those.
• X11 - I guess we could use GLX context sharing like we do for OpenGlControl now. And then use VK_KHR_external_memory to render into OpenGLBitmap. The question is how that would work once we add support for running Avalonia on top of Vulkan directly.

[llfab]

@kekekeks Very helpful. Thank you. Can I find a sample implementation of the ISkiaGpu/ICustomDrawingOperation sample somewhere?

[kekekeks]

@llfab I'm currently working on OpenTK integration and coincidentally that requires DXGI support from our rendering system. So I'm planning to add something like

class DxgiBitmap : IBitmap
{
    IDisposable Lock();
    void SetSharedHandle(IntPtr handle, bool isNtHandle);
}

next month.

[llfab]

@kekekeks That sounds good. Let me know when there is something that we can try.

[BAndysc]

@kekekeks sorry to ping, but I was wondering if there is any update on OpenTK integration? I've been trying using OpenGLControlBase, but it looks like it has problems with multiple textures.

[kekekeks]

Sorry, I'm suffering from post-covid complications, will continue OpenTK once I get better.

[BAndysc]

Sorry to hear that, all the best to you! You guys are doing a great job with Avalonia 👍

[SuRGeoNix]

This feature would be great, if we could have a rendering surface of our own while it is treated as a normal Avalonia control so you can overlap it with other content. MS didn't do that so many years and I don't think that they will ever do it (even in MAUI/Win UI 3). I'm mainly interested to use it for video playback with my custom d3d11 renderer.

[llfab]

As of now, we have PoC'ed many of this kind of integrations.
Vulkan 3D renderer into WPF (DX9) via DX11 shared texture
Vulkan 3D renderer into Avalona (OpenGL) via shared texture
Vulkan 3D renderer into Avalonia (on Vulkan) via shared texture AND shared instance/device

All of this can be done while just the one with no technology breakages results in high fps.

Most of it still requires above changes which are unclear whether / when to be integrated into master.

[Borgleader]

Is there an example of this somewhere? How involved is it to do something like this?

[llfab]

I currently do not have a sample that I can publish. It is pretty tricky with lots of OpenGL/DX commands. Also, you do not get great FPS and resize and these sort of things are also not good. Furthermore, such implementations rely on some of the extension methods from OpenGL and will therefor not run on all hardware environment. If you want to render your own DX you should set Avalonia onto DX. However, this seems not really worth the effort. Better to port your rendering to Vulkan and wait for Avalonia to support Vulkan natively...

[Borgleader]

Woops I missed the notification. Thank you for your answer. I guess I'll wait for Avalonia to support Vulkan then :) Thanks again.

[llfab]

This may be interesting with repect to DX: #9572