Migeran LibGodot Feature #90510

kisg · 2024-04-11T02:34:04Z

Features

Compile Godot Engine as either static or shared library.
Expose Godot Engine controls (startup, iteration, shutdown) over the GDExtension API to a host process.
On Apple Platforms, Godot Engine windows (both the main window, and other windows created after startup) may be rendered into surfaces provided by the host process.
- This support can be extended to other platforms, read on for details.
Sample applications (separate repository)
- C++ Sample (tested on MacOS and Linux)
- SwiftUI Sample (tested on iOS)

Why LibGodot?

LibGodot has a number of different use cases.

Control Godot Engine from a host application
- E.g. Start Godot from a .NET process to use standard .NET tooling
Use it for automation of development tasks (e.g. building an asset pipeline using a Python GDExtension API)
Embed Godot into another application while displaying Godot Windows as part of the host application's UI.

How to try it out?

We provide a Github repository with preconfigured build environment, sample apps and more information: https://github.com/migeran/libgodot_project

Migeran LibGodot Design

The core idea of the LibGodot feature is to build upon Godot's strong extension capabilities: its modularity, powerful type system and the GDExtension API.

The main advantage of this approach is, that this way LibGodot may be used from any language that has a GDExtension binding (e.g. C++, Swift, Rust, Python, ... etc.), with only minimal changes to the binding.

Godot Instance Lifecycle Management

The main class added by the LibGodot design is the GodotInstance:

class GodotInstance : public Object {
	GDCLASS(GodotInstance, Object);

	static void _bind_methods();

	bool started = false;

public:
	GodotInstance();
	~GodotInstance();

	bool start();
	bool is_started();
	bool iteration();
	void shutdown();
};

This class is made accessible over the GDExtension API. This class can be used to control a Godot instance.

To actually create a Godot instance a new symbol is added to the GDExtension API:

GDExtensionObjectPtr gdextension_create_godot_instance(int p_argc, char *p_argv[], GDExtensionInitializationFunction p_init_func);

This function can be used to create a new Godot instance and return a GodotInstance object reference to control it. Both samples show how easy it is to bind this function and then use the generated GDExtension API bindings with the returned GodotInstance object.

To properly destroy a Godot instance the GDExtension API is extended by another symbol:

typedef void (*GDExtensionInterfaceDestroyGodotInstance)(GDExtensionObjectPtr p_godot_instance);

This function is made available through the GDExtension API's getProcAddress mechanism.

Note: Due to Godot's internal architecture (the use of global data structures and singletons) only one Godot instance may be created in a process.

Embedding Godot UI

Note: UI embedding is currently implemented for Apple platforms. Please read the Next Steps section for more information on the status of other platforms.

To allow for embedding the Godot UI into a host process the host process needs to be able to pass the necessary data about a native surface where
Godot may render its contents. The form of this native surface is entirely platform and rendering backend specific.

The Migeran LibGodot design adds the following types to Godot to allow this:

A new DisplayServerEmbedded implementation which uses externally provided native surfaces as its rendering targets.
A RenderingNativeSurface class and its associated platform specific subclasses, e.g. RenderingNativeSurfaceApple.
The Window class is extended by a set_native_surface(Ref<RenderingNativeSurface>) method which allows specifying the rendering target of a Window in a typesafe, platform independent manner. It is even possible to change the rendering target of a Window dynamically during its lifecycle.

These classes are all exposed to the GDExtension API, so it is easy to use them from the host process.

The DisplayServerEmbedded class also exposes methods that allow the injection of input events from the host process into the Godot instance.

The RenderingNativeSurface class hierarchy has an additional function: provides a mechanism using the Factory Method design pattern to create compatible RenderingContextDrivers for a native surface.

This allowed us to make the DisplayServerEmbedded class platform agnostic.

We also refactored the other DisplayServer implementations to use RenderingNativeSurface during initialization.

Since all these classes are exposed over the GDExtension API, these can be seamlessly used from any supported language with a GDExtension binding.

Rationale for RenderingNativeSurface Design

For those who are familiar with Godot Engine internals: there was already a way to pass in platform specific native surface data (called WindowPlatformData) during initialization.

Why is this refactoring necessary to a full fledged reference counted object?

We chose reference counting because it makes it easier to use on the client side, no need to manage the lifecycle manually. Earlier versions of this PR used simple Godot Objects, but they required manual memory management which was error prone.
While on Apple platforms it is very easy to pass in a CAMetalLayer reference, and Metal (and thus MoltenVk) will happily render into it, other platforms impose way more restrictions.
- For example: On Windows and Linux a separate Vulkan instance and the use of external textures is required to render Godot on a separate thread from the host application's main renderer thread.
- This is not just a theoretical option: We already implemented a prototype on Linux and Windows based on Godot 4.2, where the host process and Godot are using Vulkan External Textures to pass the rendered frames from Godot to the host process. We intend to upgrade and refactor this patch to the current version of the LibGodot patch.
- To use External Textures a thread-safe queue has to be implemented between between the host process and Godot, and a reference-counted RenderingNativeSurface subclass would be an ideal place to implement it, e.g. RenderingNativeSurfaceVulkanExternalTextureWin32.

Next Steps

Open for Discussion

We are happy to discuss any part of the design, naming conventions, additional features ... etc.

Merging for Godot 4.3 as an Experimental Feature

We propose to merge the current LibGodot patch as an experimental feature in Godot 4.3, because it is already very usable in its current form for many use cases.

Godot 4.4 Developments

During the Godot 4.4 development cycle additional features may be added, for Example:

Support for additional platforms for UI embedding: Android, Windows, Linux (both X11 and Wayland).
Support for OpenGL (Native / ANGLE) UI embedding on all platforms.
Add a Configuration API that can be used on startup instead of supplying command line parameters.

Sponsors & Acknowledgements

Initial development sponsored by Smirk Software
Rebasing to Godot 4.3 and further development sponsored by Xibbon Inc.

* The GDExtension registration of the host process & build system changes were based on @Faolan-Rad's LibGodot PR: #72883

About the Authors

This feature was implemented by Migeran - Godot Engine & Robotics Experts.

Production edit: This closes Add a build-time option to compile Godot as a shared library (for use in other languages) godot-proposals#6267.

fire · 2024-04-11T07:31:53Z

You can fix some of the github ci actions tests with pre-commit run -a to format the text.

Here are the rest of the errors. https://github.com/V-Sekai/godot/actions/runs/8642913827

dsnopek

Thanks! I'm so excited to finally see this PR :-)

I looked only at the GDExtension-related changes so far. There's a bunch of rendering and display server related changes that I haven't looked at.

At high-level, this seems like a pretty good approach. Also, I really appreciate that the code changes to GDExtension are fairly minimal. :-)

On PR #72883, we discussed the possibility of introducing a new concept, perhaps called GDExtensionLoader, that would encapsulate the loading behavior, so that we could have GDExtensionLibraryLoader that used OS::open_dynamic_library() (so the current way) and then maybe a GDExtensionEmbeddedLoader that would just take an initialization function.

However, given how simple the changes are here, I think we could perhaps save something like that for a future refactor? It could make things a little cleaner by not having to constantly check if GDExtension::library is null or not, though.

core/extension/gdextension_interface.h

core/register_core_types.cpp

dsnopek · 2024-04-11T14:05:29Z

core/extension/gdextension_manager.cpp

+		Error err = libgodot->initialize_extension_function(p_init_func, "lib_godot");
+		if (err != OK) {
+			ERR_PRINT("Error initializing extension function");
+		} else {
+			libgodot->set_path("res://__LibGodot");
+			GDExtensionManager::get_singleton()->load_extension("res://__LibGodot");


This PR introduces a bunch of new names, and, while I don't necessarily want to start bikeshedding, we probably should discuss if these are the names we want to use, and, in particular, are they internally consistent, and consistent with names already used in other parts of Godot.

So, here we have:

"lib_godot" as the fake entry point name

"res://__LibGodot" as the fake library path. (Also, this needs to be unique among all extensions - with this PR, are developers able to register only one faux GDExtension from their application or multiple? In @Faolan-Rad's PR, if I'm remembering correctly, more than one faux GDExtension could be registered. If multiple are allowed, we'll need a unique path for each.)

And elsewhere we have:

The LIBRARY_ENABLED define

The "Embedded" feature tag

The general term "embedded" to describe a faux GDExtension registered through libgodot

These seem a little bit inconsistent, using the terms "libgodot", "embedded" and "library" in different places.

I agree, and we are open to naming suggestions. I agree that it is important to have consistent naming. Should we just try to use LIBGODOT everywhere as the key for this? Or should we move to the more generic "LIBRARY" name?

Coming back to the naming question because it seems there's still some mixed names (although, it's much improved from the last time I went through this PR).

Personally, I don't like the super generic "library".

I'd be for either "LibGodot" or "embedded Godot", since they are more descriptive, with maybe a slight preference for "LibGodot" because that sounds more like a product/feature name, making it easier to refer to.

If we did go for "LibGodot", then we'd end up with a LIBGODOT_ENABLED define, libgodot:// path, libgodot feature tag, and "LibGodot" in the docs and other human-readable places.

I'm surprised there hasn't been more bike-shedding about the name :-)

core/extension/gdextension.cpp

core/extension/gdextension_interface.cpp

core/extension/gdextension_manager.cpp

fire · 2024-04-12T08:16:33Z

I've tried to get the Godot Engine Github actions continuous integration to pass. Most tests pass except for Windows. https://github.com/V-Sekai/godot/tree/vsk-libgodot-migeran-4.3

Edited: Time is limited for me to work on this, so I hope this can help your efforts.

kisg · 2024-04-12T22:41:45Z

Thanks! I'm so excited to finally see this PR :-)

Thank you, and thank you for taking the time to review it!

I looked only at the GDExtension-related changes so far. There's a bunch of rendering and display server related changes that I haven't looked at.

At high-level, this seems like a pretty good approach. Also, I really appreciate that the code changes to GDExtension are fairly minimal. :-)

On PR #72883, we discussed the possibility of introducing a new concept, perhaps called GDExtensionLoader, that would encapsulate the loading behavior, so that we could have GDExtensionLibraryLoader that used OS::open_dynamic_library() (so the current way) and then maybe a GDExtensionEmbeddedLoader that would just take an initialization function.

However, given how simple the changes are here, I think we could perhaps save something like that for a future refactor? It could make things a little cleaner by not having to constantly check if GDExtension::library is null or not, though.

I think introducing the concept of library loaders would benefit not just this use case, but also other language support that are implemented as GDExtensions. If you recall, I proposed a feature like that in my .NET integration proposal: https://docs.google.com/document/d/1QwZpo3oIKHyita1mDOIB_xIxdwJ5rtAmpY4_vP9S02Y/edit?usp=sharing

However, I am also not sure if we should try to push this whole loader concept into this PR, we could do that in a follow-up PR. We can discuss this more in the GDExtension meeting.

kisg · 2024-04-12T23:40:46Z

I've tried to get the Godot Engine Github actions continuous integration to pass. Most tests pass except for Windows. https://github.com/V-Sekai/godot/tree/vsk-libgodot-migeran-4.3

Edited: Time is limited for me to work on this, so I hope this can help your efforts.

Thank you @fire ! Would it be ok if I squash your changes into a single commit, and add it to our branch?

fire · 2024-04-13T05:18:08Z

Do what what works for you.

Edited:

As the initiator of the libgodot-style patch, I've successfully persuaded several individuals, including @Faolan-Rad to contribute to the libgodot project. I am excited about the new places we can use Godot Engine.

fire · 2024-05-03T18:00:52Z

@kisg Can you review the changes to libgodot from here V-Sekai#35?

Here's the modified libgodot_project project working on Windows. https://github.com/V-Sekai/libgodot_project. You may need to force mingw.

kisg · 2024-06-11T00:04:10Z

With this latest update most review comments should be fixed. New implementation uses GDExtensionLoader PR as the base: #91166

Also updated to a recent master.

Still on the TODO list:

Windows build
Fix CI builds
Create a .NET/C# example using the new GDExtension-based .NET implementation.

- Implements the concept of GDExtension loaders that can be used to customize how GDExtensions are loaded and initialized. - Moves the parsing of `.gdextension` config files to the new `GDExtensionLibraryLoader`. - `GDExtensionManager` is now meant to be the main way to load/unload extensions and can optionally take a `GDExtensionLoader`. - `EditorFileSystem` avoids unloading extensions if the file still exists, this should prevent unloading extensions that are outside the user project.

@Faolan-Rad

* Based on top of the GDExtensionLoader PR: godotengine#91166 * Add a new GodotInstance GDCLASS that provides startup and iteration commands to control a Godot instance. * Adds a libgodot_create_godot_instance entry point that creates a new Godot instance and returns a GodotInstance object. * Adds a libgodot_destroy_godot_instance entry point that destroys the Godot instance. * Allow specifying an external native rendering surface handle to render Godot into the UI of a host application. * It is also possible to embed multiple Godot windows into the UI of a host application. * Currently supported on MacOS and iOS Sample Apps: https://github.com/migeran/libgodot_project Developed by [Migeran](https://migeran.com) Sponsors & Acknowledgements: * Initial development sponsored by [Smirk Software](https://www.smirk.gg/) * Rebasing to Godot 4.3 and further development sponsored by [Xibbon Inc.](https://xibbon.com) * The GDExtension registration of the host process & build system changes were based on @Faolan-Rad's LibGodot PR: godotengine#72883 Co-Authored-By: Gabor Koncz <gabor.koncz@migeran.com>

GeorgeS2019 · 2024-07-12T20:49:46Z

Create a .NET/C# example using the new GDExtension-based .NET implementation.

See #72883 (comment)

YakoYakoYokuYoku

Overall, this is a pretty good job, there's some suggestions I've made to improve this PR.

AThousandShips · 2024-07-24T14:37:37Z

@YakoYakoYokuYoku Where did you leave suggestions? I can't see any code suggestions

GeorgeS2019 · 2024-07-24T14:50:39Z

@AThousandShips
@kisg
The windows build PR is ready for merge
I think this PR is ready to merge too. Please feedback

[Update] 24.07.2024 => now that we have

go
c++
swift
Next is c#

to create sample app
JiepengTan/libgodot_llgo_demo#1

akien-mga · 2024-07-24T14:55:03Z

@GeorgeS2019 We're in feature freeze for 4.3, and this PR is slated for 4.4. Please be patient.

YakoYakoYokuYoku · 2024-07-24T14:18:50Z

SConstruct

@@ -162,7 +162,6 @@ env.__class__.use_windows_spawn_fix = methods.use_windows_spawn_fix

 env.__class__.add_shared_library = methods.add_shared_library
 env.__class__.add_library = methods.add_library
-env.__class__.add_program = methods.add_program


This line shouldn't be removed as it can be used later on with the addition of an option to build a program wrapper around the library.

YakoYakoYokuYoku · 2024-07-24T14:19:36Z

SConstruct

+else:
+    env.__class__.add_program = methods.add_program


Per above, these two lines should be removed.

YakoYakoYokuYoku · 2024-07-24T14:20:50Z

core/extension/gdextension_library_loader.cpp

+		// Iterate the libraries, finding the best matching tags.
+		String best_library_path;
+		Vector<String> best_library_tags;
+		for (const String &E : libraries) {


Nit, change this E into lib or something else.

E is the standard godot C++ style.

YakoYakoYokuYoku · 2024-07-24T14:22:47Z

drivers/SCsub

@@ -25,6 +25,7 @@ SConscript("coremidi/SCsub")
 SConscript("winmidi/SCsub")

 # Graphics drivers
+SConscript("apple/SCsub")


This results in build failures in platforms other than Apple, please use an if statement.

YakoYakoYokuYoku · 2024-07-24T14:24:01Z

core/extension/godot_instance.h

I'd not recommend doing a git move here, please undo it and create a new file instead.

Do you know how to force git do this operation? This is a automatic action.

YakoYakoYokuYoku · 2024-07-24T15:07:02Z