Merge .NET 6 branch with master #64089

We will be progressively moving most code to C#. The plan is to only use Mono's embedding APIs to set things at launch. This will make it much easier to later support CoreCLR too which doesn't have rich embedding APIs. Additionally the code in C# is more maintainable and makes it easier to implement new features, e.g.: runtime codegen which we could use to avoid using reflection for marshaling everytime a field, property or method is accessed. SOME NOTES ON INTEROP We make the same assumptions as GDNative about the size of the Godot structures we use. We take it a bit further by also assuming the layout of fields in some cases, which is riskier but let's us squeeze out some performance by avoiding unnecessary managed to native calls. Code that deals with native structs is less safe than before as there's no RAII and copy constructors in C#. It's like using the GDNative C API directly. One has to take special care to free values they own. Perhaps we could use roslyn analyzers to check this, but I don't know any that uses attributes to determine what's owned or borrowed. As to why we maily use pointers for native structs instead of ref/out: - AFAIK (and confirmed with a benchmark) ref/out are pinned during P/Invoke calls and that has a cost. - Native struct fields can't be ref/out in the first place. - A `using` local can't be passed as ref/out, only `in`. Calling a method or property on an `in` value makes a silent copy, so we want to avoid `in`. REGARDING THE BUILD SYSTEM There's no longer a `mono_glue=yes/no` SCons options. We no longer need to build with `mono_glue=no`, generate the glue and then build again with `mono_glue=yes`. We build only once and generate the glue (which is in C# now). However, SCons no longer builds the C# projects for us. Instead one must run `build_assemblies.py`, e.g.: ```sh %godot_src_root%/modules/mono/build_scripts/build_assemblies.py \ --godot-output-dir=%godot_src_root%/bin \ --godot-target=release_debug` ``` We could turn this into a custom build target, but I don't know how to do that with SCons (it's possible with Meson). OTHER NOTES Most of the moved code doesn't follow the C# naming convention and still has the word Mono in the names despite no longer dealing with Mono's embedding APIs. This is just temporary while transitioning, to make it easier to understand what was moved where.

We are moving in the direction of no dynamic code generation, so this is no longer desired. The feature can still be easily implemented by any project that still want it.

The main focus here was to remove the majority of code that relied on Mono's embedding APIs, specially the reflection APIs. The embedding APIs we still use are the bare minimum we need for things to work. A lot of code was moved to C#. We no longer deal with any managed objects (`MonoObject*`, and such) in native code, and all marshaling is done in C#. The reason for restructuring the code and move away from embedding APIs is that once we move to .NET Core, we will be limited by the much more minimal .NET hosting. PERFORMANCE REGRESSIONS ----------------------- Some parts of the code were written with little to no concern about performance. This includes code that calls into script methods and accesses script fields, properties and events. The reason for this is that all of that will be moved to source generators, so any work prior to that would be a waste of time. DISABLED FEATURES ----------------- Some code was removed as it no longer makes sense (or won't make sense in the future). Other parts were commented out with `#if 0`s and TODO warnings because it doesn't make much sense to work on them yet as those parts will change heavily when we switch to .NET Core but also when we start introducing source generators. As such, the following features were disabled temporarily: - Assembly-reloading (will be done with ALCs in .NET Core). - Properties/fields exports and script method listing (will be handled by source generators in the future). - Exception logging in the editor and stack info for errors. - Exporting games. - Building of C# projects. We no longer copy the Godot API assemblies to the project directory, so MSBuild won't be able to find them. The idea is to turn them into NuGet packages in the future, which could also be obtained from local NuGet sources during development.

We're targeting .NET 5 for now to make development easier while .NET 6 is not yet released. TEMPORARY REGRESSIONS --------------------- Assembly unloading is not implemented yet. As such, many Godot resources are leaked at exit. This will be re-implemented later together with assembly hot-reloading.

Some Linux distros use their distro name as the RID for directory names. If the .NET Host directory cannot be found with the generic RID, try to get the rid from `dotnet --info`. The generic RID should still be the first choice. Some platforms like Windows 10 define the RID as `win10-x64` but still use the generic `win-x64` for directory names. Co-authored-by: Lewis James <lewiji+github@gmail.com>

This replaces the way we invoke methods and set/get properties. This first iteration rids us of runtime type checking in those cases, as it's now done at compile time. Later it will also stop needing the use of reflection. After that, we will only depend on reflection for generic Godot Array and Dictionary. We're stuck with reflection in generic collections for now as C# doesn't support generic/template specialization. This is only the initial implementation. Further iterations are coming, specially once we switch to the native extension system which completely changes the way members are accessed/invoked. For example, with the native extension system we will likely need to create `UnmanagedCallersOnly` invoke wrapper methods and return function pointers to the engine. Other kind of members, like event signals will be receiving the same treatment in the future.

Finalizers are longer guaranteed to be called on exit now that we switched to .NET Core. This results in native instances leaking. The only solution I can think of so far is to keep a list of all instances alive to dispose when the AssemblyLoadContext.Unloading event is raised.

This base implementation is still very barebones but it defines the path for how exporting will work (at least when embedding the .NET runtime). Many manual steps are still needed, which should be automatized in the future. For example, in addition to the API assemblies, now you also need to copy the GodotPlugins assembly to each game project.

The editor no longer needs to create temporary instances to get the default values. The initializer values of the exported properties are still evaluated at runtime. For example, in the following example, `GetInitialValue()` will be called when first looks for default values: ``` [Export] int MyValue = GetInitialValue(); ``` Exporting fields with a non-supported type now results in a compiler error rather than a runtime error when the script is used.

Previously, for each scripts class instance that was created from code rather than by the engine, we were constructing, configuring and assigning a new CSharpScript. This has changed now and we make sure there's only one CSharpScript associated to each type.

These two had been disabled while moving to .NET 5, as the previous implementation relied on Mono embedding APIs.

Previously, we added source generators for invoking/accessing methods, properties and fields in scripts. This freed us from the overhead of reflection. However, the generated code still used our dynamic marshaling functions, which do runtime type checking and box value types. This commit changes the bindings and source generators to include 'static' marshaling. Based on the types known at compile time, now we generate the appropriate marshaling call for each type.

This commit adds initial support for games exported as NativeAOT shared libraries. At this moment, the NativeAOT runtime is experimental. Additionally, Godot is not trim-safe as it still makes some use of reflection. For the time being, a rd.xml file is needed to prevent code triming: ``` <Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata"> <Application> <Assembly Name="GodotSharp" Dynamic="Required All" /> <Assembly Name="GAME_ASSEMBLY" Dynamic="Required All" /> </Application> </Directives> ``` These are the csproj changes for publishing: ``` <PropertyGroup> <NativeLib>Shared</NativeLib> </PropertyGroup> <ItemGroup> <RdXmlFile Include="rd.xml" /> <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="7.0.0-*" /> </ItemGroup> ``` More info: - https://github.com/dotnet/runtimelab/blob/feature/NativeAOT/docs/using-nativeaot/compiling.md - https://github.com/dotnet/runtimelab/tree/feature/NativeAOT/samples/NativeLibrary - https://github.com/dotnet/runtimelab/blob/feature/NativeAOT/docs/using-nativeaot/rd-xml-format.md

In the past, the Godot editor distributed the API assemblies and copied them to project directories for projects to reference them. This changed with the move to .NET 5/6. Godot no longer copies the assemblies to project directories. However, the project Sdk still tried to reference them from the same location. From now on, the GodotSharp API is distributed as a NuGet package, which the Sdk can reference. Added an option to `build_assemblies.py` to copy all Godot NuGet packages to an existing local NuGet source. This will be needed during development, while packages are not published to a remote NuGet repository. This option also makes sure to remove packages of the same version installed (~/.nuget/packages). Very useful during development, when packages change, to make sure the package being used by a project is the same we just built and not one from a previous build. A local NuGet source can be created like this: ``` mkdir ~/MyLocalNuGetSource && \ dotnet nuget add source ~/MyLocalNuGetSource/ -n MyLocalNuGetSource ```

Changed the signal declaration signal to: ``` // The following generates a MySignal event [Signal] public delegate void MySignalEventHandler(int param); ```

- Array and Dictionary now store `Variant` instead of `System.Object`. - Removed generic Array and Dictionary. They cause too much issues, heavily relying on reflection and very limited by the lack of a generic specialization. - Removed support for non-Godot collections. Support for them also relied heavily on reflection for marshaling. Support for them will likely be re-introduced in the future, but it will have to rely on source generators instead of reflection. - Reduced our use of reflection. The remaining usages will be moved to source generators soon. The only usage that I'm not sure yet how to replace is dynamic invocation of delegates.

This new version does not support the following type arguments: - Generic types - Array of Godot Object (Godot.Object[]) or derived types The new implementation uses delegate pointers to call the Variant conversion methods. We do type checking only once in the static constructor to get the conversion delegates. Now, we no longer need to do type checking every time, and we no longer have to box value types. This is the best implementation I could come up with, as C# generics don't support anything similar to C++ template specializations.

The setting is initially assigned the name of the Godot project, but it's kept freezed to prevent issues when renaming the Godot project. The user can always rename the C# project and solution manually and change the setting to the new name.

We were using it to workaround a limitation of `Unsafe.AsPointer` and `ref struct`s. However, we can get the same result with some tricks, since we have control over the declaration of these structs.

- Moves interop functions to UnmanagedCallbacks struct that contains the function pointers and is passed to C#. - Implements UnmanagedCallbacksGenerator, a C# source generator that generates the UnmanagedCallbacks struct in C# and the body for the NativeFuncs methods (their implementation just calls the function pointer in the UnmanagedCallbacks). The generated methods are needed because .NET pins byref parameters of native calls, even if they are 'ref struct's, which don't need pinning. The generated methods use `Unsafe.AsPointer` so that we can benefit from byref parameters without suffering overhead of pinning. Co-authored-by: Raul Santos <raulsntos@gmail.com>

This should clarify development workflow with the NuGet packages.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge .NET 6 branch with master #64089

Merge .NET 6 branch with master #64089

Commits on Aug 22, 2022