Generate compatibility layer

[Ughuuu]

Generate compatibility layer between godot and godot_cpp at GDExtension level.

• Fixes Add an easier way to support GDExtensions for both addons and modules, specifically for includes godot-proposals#9212

[Ughuuu]

Actually no need to change the ci since we dont export the headers just the lib atm.

[Ughuuu]

Should the output_header_mapping_godot.json file be in the repo at root(or should there be other ways of generating this file every time)? If so it would need to be manually generated each time based on similar/matching godot version (eg. godot-cpp 4.3, godot 4.3)
As for generating it every time, a param could be given to the scons run command (probl as env var) as to where the godot repo is and run the command there and generate the bindings then?
Eg. if you set GODOT_REPO_PATH then the godot_compat bindings are generated, if not no. I'll give it a try and see if it works well will do it like that.

Edit:

Made it work without needing the file commited, just needs to have the godot_repo set, eg.:

scons godot_repo="../godot"

[dsnopek]

Given that this is mapping godot-cpp-style includes to Godot includes, could this be implemented by changing our normal .hpp headers to instead include the Godot header when built as a module?

So, for example, could it have godot_cpp/classes/xr_server.hpp like:

#ifdef GODOT_MODULE
#include "servers/xr_server.h"
#else

// All the rest of the code that's normally in xr_server.hpp...

#endif

Then a GDExtension doesn't actually need to change its includes to build as a module.

[Ughuuu]

Very good idea, must say, has some downsides, but overall a lot of upsides.
This might work for the generated stuff, but won't work for the non generated stuff (eg files in include folder, such as variant/variant.hpp, etc. These need to be manually changed). If the manual ones are also updated, then it will work and won't need any mapping file.
However, this will have the downside of tying godot-cpp to godot version. Which for eg. if you work with godot-cpp 4.3 and godot 4.4, and godot changed location of some files, then it won't work.

Another thing I forgot, godot uses no namespacing, while godot-cpp uses using namespace godot;. Right now I added in the godot_compat incldues the using namespace godot; to polute the global namespace to have symetry.

[dsnopek]

If the manual ones are also updated, then it will work and won't need any mapping file.

We could have the hand-written files include a generated file that would have the Godot includes (or nothing, if the user didn't pass godot_repo="...").

However, this will have the downside of tying godot-cpp to godot version. Which for eg. if you work with godot-cpp 4.3 and godot 4.4, and godot changed location of some files, then it won't work.

Well, if this code is only added if the developer specified godot_repo="...", then godot-cpp isn't bound to any Godot version unless requested.

[Ughuuu]

Sounds good to me, so then I start to implement that change where everything just works magically inside the godot_cpp includes?

1. the generated files will have (if there is a godot_repo is passed) both addon and module.
2. the manual files will get also overridden (at generation time? if godot_repo is set and basically add the ifdef for addon and module as well)

Also, this will only happen if godot_repo is sent, so no extra file anywhere. Also, I update both SConstruct and CMake with this extra godot_repo param.

EDIT:

And about the using namespace godot; that is only in godot-cpp (As addon) how will that be solved? Right now I do (eg.):

#ifdef GODOT_MODULE_COMPAT
#include <core/math/a_star_grid_2d.h>
#else
#include <godot_cpp/classes/a_star_grid2d.hpp>
using namespace godot;
#endif

[dsnopek]

2. the manual files will get also overridden (at generation time? if godot_repo is set and basically add the ifdef for addon and module as well)

Just to be clear, I don't think the manual files need to get "overridden", they just need to have code like this added:

#ifdef GODOT_MODULE
// The next included file is generated: if we had `godot_repo="..."` it'll contain the Godot includes, if not, it'll just be an empty file.
#include <godot_cpp/compat/HEADER.hpp>
#else

// ... the code that's there now ...

#endif

So, they'd still be hand-written, but they'd include a generated file. We definitely shouldn't have the generation touch the content of the hand-written files.

[Ughuuu]

Making it draft while I implement new changes.

[dsnopek]

Thanks!

I think the overall idea of the new direction this PR is taking seems good. However, there's some things in the implementation that I think could be improved (more details below), and there still appears to be a bunch of stuff left over from the previous direction.

[dsnopek]

So, this is modifying the generated files afterwards? Why not just make this part of generating the files in the first place?

[Ughuuu]

There are a lot of functions that create the file. Also this is how it was previously as I created the file in another folder. I can update if you want, but that will polute the smaller functions (that generate file in the first place)

[dsnopek]

Personally, I think we should update the functions that generate the files in the first place

[Ughuuu]

Ok, well, got it working now all around. If you want this change, I can do that instead, but first let me bring some counter arguments I guess. If in future we don't need this tight integration anymore, it's easier to remove. All the code for compat is in one function. If there are bugs/issues with it, its one function to debug, and not multiple functions.
Downsides to having it separate is that people might not test it as often and it might break, though since in other functions it will be under if's anyway, won't be much tighter integrated anyway.

So yeah, if you think it's better to integrate them in functions, will update again the code. Just double checking.

[dsnopek]

I really do think we should integrate this into the functions generating the files in the first place. But I guess that's just my opinion? If you can get other GDExtension team members on-board, I'd be fine to go with the consensus, but I personally think it's silly to modify files after the fact, when it's the same script that's generating them in the first place.

[Ughuuu]

Who are other gdextension members? Should i request a review on this pr at this stage?

[dsnopek]

See https://godotengine.org/teams/#scripting