Improve the performance of the GDScript VM #6031
Comments
|
If we can't find a single solution that works, it may be possible we can just do an AOT compiler for platforms that support codegen, and have alternative write to C/GDExtension for platforms that do not. |
|
Disclaimer: My knowledge on compiler infrastructure and build systems is limited. For me personally, the concern is less that GDScript is slow, and more that the alternatives are miserable to work with. My language of choice for many years was C++, and so GDExtension looks nice enough, but having to download external tools and build systems and thread everything together gets in the way of what I want to do, which is write performant code to solve my problems. I don't feel like I gain any meaningful control from choosing a compiler infrastructure or refining the build system. I also don't really care about editor integration as much as I care about ease of use. If I could write a single trivial .cpp file and drag and drop it over some random executable that did all the boilerplate garbage and spat out a DLL for me to plug into my project, I wouldn't give a damn about editor integration. So perhaps a distinction needs to be drawn. Are we trying to: |
|
@vpellen The goal is to improve performance for the average user, so definitely this is not for you. |
|
I think I'd be in favor of having a good GDScript centric optimizer, working against the codegen format of GDScript, with the option of exporting in executable format via some method. It's a big undertaking but I feel it ensures that GDScript behaves well in all cases (including dynamically created scripts), and makes prototyping easier (not having to export each time to get the optimization but only missing the boost from executable format) |
|
If we go with LLVM, I want to recommend using MLIR. It's somewhat of a more powerful, more generic evolution of LLVM IR that could allow some other things in the future, like compiling GDScript to compute shaders directly. There's a lot of research at the moment and it's pretty powerful. (It's used by TensorFlow, for instance). There are already abstractions for most of the common programming language features like structured control flow, functions, etc. And it already has a lot of stuff implemented, like the propagation of debug symbols. So line-by-line debugging would just be a matter of plugging into LLDB. |
|
@jabcross The problem with LLVM IMO is that its enormous and that you are on your own for a lot of stuff to generate IR code. Given GDScript demands a ton of C ABI interoperabilty, C should work better and be simpler than an IR, but of course things are more complicated from there. This is why QBE seems more interesting than LLVM in the fact that its designed around C ABI interop. I also don't think GDScript has a ton of optimizer potential that LLVM can take advantage of, so another possibility is to do simpler optimization manually and then run on something like SLJIT. |
|
(Not a core developer, just a big fan and casual/average user of Godot, mostly through GDScript, dabbled with Mono) I'm asking because GDScript being a simple included VM is to me one of the biggest advantages of Godot for people to get started. Not requiring a compile step makes iterating on code changes very quick, and having everything included in the editor without any external install makes it the most pleasant Game Engine I've used. I think Godot's GDScript VM is what gives Godot such great ergonomics, at least for average users (like myself :D ). If this is an alternative to the VM that lives side by side, then I'm a bit less concerned about the ergonomics being impacted. But I'd still be concerned if that increases the workload on the team. It seems it'd be effectively another whole back end (or set of back ends) that is officially supported, and may make future development much more complicated as it requires supporting many back ends. If the team has the bandwidth, then that concern is also addressed. Do average users run into performance issues frequently enough with GDScript that it merits addressing with this large a shift in implementation strategy? Is there any usage pattern that can be identified that would be better addressed providing some other tooling (for example, nodes that abstract away the concept of resource pooling if a frequent performance concern is creating too many entities)? |
It's x86 and Linux + Windows only. Cross-building with any external compiler will be a huge pain. Any version of clang should be able to generate code for any target, but linking the final shared library and any use of C/C++ standard libraries won't work without access to the platform SDKs for each target. But I guess the same will be true for the integrated LLVM / QBE as well. |
|
@reduz MLIR addresses a lot of these issues, but you're right that, at least for development of the engine itself, LLVM is quite big. But the compiler could dynamically link to the LLVM library once it's done, so the end users wouldn't be exposed to that. The GDScript optimizer could be made as a separate module that most engine devs wouldn't have to compile. MLIR provides a number of ways to deal with C ABI interop. (I had to learn them for my MsC project). There's also a C/C++ MLIR dialect being worked on that will greatly simplify connecting directly to C code at any level of abstraction, so we won't be limited to the ABI forever. |
Hey, I'll have you know I'm remarkably average! Jokes aside, the reason I bring it up is because robust maintainable large-scale integrated performance improvements are probably going to be exceedingly difficult to implement. The average user is not going to be well versed in build systems and compiler infrastructure, which means if you want a good solution it'll have to be integrated with the existing editor in a pretty seamless way. From what little knowledge I have on the subject, LLVM would probably be the most robust and future proof, but the overhead concerns are real, and I feel like Godot has a history of rejecting existing external codebases in favor of home-grown ones (Bullet is the most recent example of this I can think of). I have a sneaking suspicion that the larger codebases will be deemed excessive, while the smaller ones won't quite fit the needs, which will lead to the development of a custom solution that will take three years to develop and be miserable to maintain. I'm hoping my concerns are unwarranted. |
|
Something not mentioned yet but I think is worth seriously considering here is using WASM itself as the GDScript intermediary format. There are plenty of fast and actively maintained FOSS WASM runtimes, (AOT, JIT and interpreters). Wasmer (AOT & MIT), WAVM (JIT & BSD) and wasm3 (interpreter & MIT) stick out to me as being the most performant in their respective categories. They are all close to native (WAVM even claims to exceed native performance in some cases). Using WASM for this would also enable a better extension experience. No more hassle with making sure that the correct binaries are built and included for all possible platforms ahead of time. Just drop in your one WASM file and use any language that supports a WASM target to write Godot extensions, the WASM runtime will take care of executing the extension efficiently on any of the export targets. |
|
@Splizard WASM is overkill IMO and the memory isolation for something like this really gets in the way. It also does not solve the problems mentioned above (export to platforms that need native AOT). |
|
As someone who actually build a commercial AOT compiler from JVM bytecode to x86 and ARM machine code using LLVM, I think compiling to C is the best option for various reasons:
The one downside is that it's harder to emit debugging information, if it's a goal that the AOT compiled code should also be debuggable via the GDScript debugger. I don't think that's a requirement for a V1. The other downside you mentioned is requiring a C compiler. I don't think this is a problem though. Godot could bundle or on-demand download Clang on all platforms. That's not really much more in download size than bundling the LLVM binaries. If needed, a customized Clang build could be maintained that strip away anything not needed. |
|
Seconding @badlogic in that LLVM is not only big, but also changes A LOT. |
|
@badlogic I think the AOT route via C is useful, but its still more hassle than what Godot users are accostumed to. IMO this is something that we should have to support anyway at some point (else Wasm, iOS or console optimization will not happen) but, as you say, for development this is a hassle. To me C AOT (basically compiling GDScript to a C code that uses the GDExtension API) is something that is very simple to do (likely few thousands loc) and maintain. But for development, my feeling is that we should have something that works out of the box, which is why I still think something like SLJIT makes more sense. The only thing you miss is the optimizer pass, but given the nature of GDScript I doubt this will be much a problem and the performance gain will still be very significant. |
|
For reference, either as LLVM, C or SLJIT, all that needs to be done is provide a VM back-end: Either spitting C or building an SLJIT execution buffer based on this is not a lot of work (order of thousands of lines of code). The reason is that most of those instructions, if you look closely, already call C functions to do most of the heavy loading (as an example, all built-in types, operators, constructors, and all the entire engine API are available via C function pointers). Eventually, significant optimization can be achieved by providing specialized versions of the instructions when the types are known (the codegen.h already lets you know this, so this work is already done). Even adding an SSA optimizer with most common optimizations would likely not be very hard. In contrast, writing an WASM, LLVM or QBE back-end would probably be orders of magnitude more complex, because you need to replace all this with inline code in IR format. Of course, this has more optimization potential than the previous methods, but the amount of work required and the maintenance requirements are enormous. |
|
@reduz I don't think a C generator + compiler requirement is a hassle. Users already have to download export templates for all platforms. Providing the user a single click download inside the editor to fetch (a trimmed down) Clang seems to be pretty OK to me in terms of hassle. As you said, you will need an AOT solution for iOS and WASM anyways, where SLJIT won't work. It's also questionable if an SLJIT generator can actually achieve any performance improvement compared to the current bytecode interpreter. |
WASM runtimes totally support native AOT.
(https://medium.com/wasmer/wasmer-1-0-3f86ca18c043)
Reference types in WASM are a thing (also supported by wasmer), I'm not sure about the performance profile of say keeping all packed arrays and strings as reference types but it's certainly a solution. As @vpellen indicates, I don't think you can beat the user experience of dropping a single WASM file into your project and being able to export this seamlessly to mobile devices, consoles, etc. Let the WASM runtimes focus on performance! |
|
@badlogic It is a hassle if this is a requirement to even use the engine. Godot is a tiny download that just works and a large amount of the Godot community largely appreciates this. The export templates are only downloaded at the time of exporting the game, but you don't need them to start working quickly. I agree with you that this is likely the path we will have to take at some point, but it should only be an optional step for games that only really need this extra performance. The current bytecode interpreter still has a good amount of room for optimization (addressing needs to be further simplified and typed instructions need to be added), but as the intepreter is a gigantic jump table, it still has a big toll on the L1 caches and branch predictor, which should be significantly improved with something like SLJIT, and that works out of the box with little work. |
|
@Splizard Its good to know you can handle AOT but, as I said, I still think the isolation sucks and creates a lot of glue friction that I would be happy to avoid. Additionally, you still need some sort of compiler so, in the end, the advantage provided is minimal vs the drawbacks. |
|
For the reference, there is an existing proposal for a WASM runtime with a decent amount of support, but not much discussion: #3370 |
|
I know it is an unpopular opinion, what about dropping GDScript and using Python instead? I would like to clarify my point. As a matter of fact, writing a programming language is the work of a full-time team. I find it ambitious for Godot Engine to create a game engine with a whole new approach with a new programming language. Perhaps Godot Engine should do with GDScript the same thing that happened with VisualScript. At the same time switching to another language with a syntax close to GDScript, such as python. It should close a lot of tickets directly related to GDScript and allow us to focus on the core of Godot Engine, i.e. being a game engine. I'm aware that this is only my opinion, and that I'm not the person who maintains GDScript, nor the one who will be in charge of switching to Python, and even less the one who maintains Godot Engine in a broader sense. |
|
@xsellier We like GDScript because it allows us to tightly integrate into the engine, to remove a lot of overhead both conceptually and in terms of the interop. Maintenance of our own tool does come at a cost, but so does maintenance of a blackbox solution that we must embed into our project. Take our approach to physics for example, you see the same thing. We opted to abandon Bullet as the main backend because it is very hard to keep that bridge between the two parties sound. Same applies here. Aye, it will close some bugs in our codebase. But it will also open new bugs which may not be within our reach to fix. So we'll have to bother other projects with our issues, projects which may not share our goals or don't see our problems as worthwhile. But our responsibilities before our own users won't go anywhere, so we'd have to either take the blame or ship some hacks and ad-hoc fixes, if we can. That's not great at all. |
|
@xsellier Python is a terribe option. Letting aside that there is not a good VM that we can use and that all the nice integration to the editor would be lost, the VM is not designed to work standalone. Even if you make it work standalone, you lose access to the rest of the ecosystem, or you bloat massively your exports as a result. Python is designed so applications are parts of its ecosystem and not the opposite. It definitely is nice to have Python support for some things (if you work on scientific or academic fields as an example) but it's not meant to be used for something like Godot. |
|
I get your point, Python might not be the best choice because of the size of its VM. My point is still relevant, writing a programming language is a full-time job for a whole team of dev. Meaning, either hiring more devs to work on GDScript, or dropping it and take another already existing programming language. I'm quite happy about what's GDScript is becoming, First class functions, static methods, new built-in types... but for the last 2 years, progress reports about GDScript are quite sparse ( https://duckduckgo.com/?sites=godotengine.org%2Farticle&q=GDScript&ia=web ). I mean I'd expect more update on GDScript since it is a programming language with quite some bugs ( https://github.com/godotengine/godot/issues?q=is%3Aopen+is%3Aissue+label%3Atopic%3Agdscript ). Some bugs are more critical than others and lasts for years ( godotengine/godot#7038 ). I'd like to state it is not a complain, because I use GDScript and I like the way it is, it is more like a realistic statement. My point is more, creating a programming language is too much work for Godot Engine atm, it might be better to think about dropping it, to focus on what's imprtant for the engine. You can focus on Python if you want, I was talking about dropping GDScript in favor of something else. |
|
@YuriSizov Its probably not a lot of work to have a Wasmer module that can load GDExtensions inside the runtime compiled for Webassembly. There is likely not a lot of merit either, but using it for running GDScript is definitely not a good solution. GDScript uses engine types for everything, so the isolation means the glue needs to either spend a lot of time converting the engine types to communicate with the engine, or it needs to keep types as external references and access to them is slower. This is what happens in C# as an example, where you need to be conscious on using engine types or native types depending on what you need for the most part (accessing data or passing it), which is fine for this language as users are expected to be more experienced. But for something easy to use like GDScript that is meant to "just work", it will cause a lot of friction. |
@xsellier |
|
@xsellier The problem with your logic is that, what I believe you fail to understand (likely due to lack of experience in the field) is that Integrating another programming language to Godot and keeping the fantastic "just works" usability GDScript is even more work than developing GDScript. Writing glue code is difficult and there are a lot of corner cases:
I already integrated other languages such as Lua, Squirrel and Python to Godot. It was a ton of work and generally there is more code to do the glue than there is GDScript runtime code, and inefficiencies everywhere when getting data one way or the other (same reason why I think Wasm as script runtime is a terrible idea). This without even getting into all the nice editor integrations GDScript has to the engine, which would be a massive amount of work to do in another language (where in GDScript they are tiny). Building software is not stacking lego bricks where you add something existing and it magically works. Getting the pieces to fit and keeping them together often requires as much effort as writing the pieces yourself. |
This comment was marked as off-topic.
This comment was marked as off-topic.
Indeed, speed is not the concern for most GDscript users I think, but I do think the fact that Godot compiles GDscript to plain text is an issue... which allows others with bad intention to open the game in Godot Engine as if the project is fully open source. C# does not fix this issue, and C++ is super complex. I wish Godot supported Swift instead of C#, as Miguel states, C# and its GC is a billion dollar mistake in gaming industry. |
|
You are probably saying this after watching Miguel de Icaza's video or something related. C# support was a political move iirc ; so Unity users would migrate to Godot. I believe it has had good results on that front. Swift is more of a c++ equivalent that is more developer-friendly, and has a future for making GDextension modules :) |
|
This is a bit offtopic here. If you are interested in Swift support, please open another proposal or start a discussion on another forum. However, Swift is already supported today with SwiftGodot: https://github.com/migueldeicaza/SwiftGodot You can even embed Godot into a SwiftUI app with it using the libgodot PR (now under review): https://github.com/migeran/libgodot_project |
|
You all do realize SwiftGodot already exists, right? C# currently exists as an optional module, but there are talks about porting it to a GDExtension instead, so it would be on identical footing to SwiftGodot. This is considered to be an improvement to the C# integration. Mind you that there's a hidden disadvantage to using Swift over C#: C# compiles to CLI, which is in turn executed by the runtime available on each target platform (Mono, .Net Core). Only a single CLI build is necessary, since CLI is an intermediate language. Basically, it boils down to: Swift is still too new, and cross-platform support is still incomplete. |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
As a quick reminder, Godot 4.x HTML5 export is no longer possible to be published to Cloudflare due to its huge size (godotengine/godot#70672 godotengine/godot#68647). TLDR; If any machine-code-related approaches will exist, I prefer external tool approach, as it's what other game engines have been doing and it doesn't conflict with Godot's design approach (as to be portable, self-contained, and has great backwards compatibility). External GDScript optimisation tool sounds like the most logical way to implement into Godot, as it's what other commercial tools have been doing for years. While having all-integrated tool that users can just download and use in single binary blob (Godot's design goal) is nice, but its size difference is too significant to let go especially if such approach is going to be used in platforms with specific requirements or very limited resources such as Apple iOS or HTML5. For comparison, GameMaker specifically requires its users to install platform-specific SDKs and at times also requires its users to have another machine being set up as a compiler machine instead of developing an AIO integration like what Unity has achieved with its IL2CPP solution, to workaround huge development time. While it sounds daunting, it isn't all that difficult if guided properly since the rest of the process, the tool will take the task and do rest of things for users already. There isn't really anything lower-level that users must have the knowledge in order to utilise such tools. Even better for Godot because it already has much better integration than GameMaker that still relies heavily on third party toolchains most of time. GDScript/C++ transpiler doesn't sound too foreign as it's what it was done previously. It's just that nobody really has any interests bringing it to the mainstream or improve it to become user-friendly enough (especially the proper GUI, I yet to have seen any tools with a proper user interface, including my own tools, which I planned to have them later). It's still huge work (especially proper debugging functionality) but at very least it's a lot less than other LLVM or all-integrated approaches that also introduce side effects and/or make existing problems even worse (notably, editor/export size and it breaking compatibility with some platforms). The traditional GDScript binary interpretation should always exist. I personally don't ever think that GDScript should become a full-fledged or drop-in programming language. It already works well enough with tasks it's designed for (as to be a high level scripting language), and even better with the recent optimisation patch, but it still should never be treated as a programming language that will perform immense amount of complex calculation tasks at any given seconds. Those, if possible, should be implemented in more performant languages either in the core if it's significant enough for majority of users or custom builds/GDExtension if it's task-specific. Even if ease of use is a priority, it should always be a task for talented community to make those solutions available and can be applied easily for non-tech-savvy users. Imagine AIO chunk generator, or MMD model loader/interpreter, or something that's rather small but has huge impact but only for server builds such as Argon2 hashing module. I'm honestly surprised that there yet to be any GDExtension related asset stores for Godot yet. |
|
Something like this?: GdScript2All |
|
Might be irrelevant, but Roblox's Luau fork has code generation. It also doesn't use LLVM and works on all platforms because... Roblox. |
Nah, thank their server using one platform for that.
|
I just meant the concept in general, although it might have some issues I can't think of. It appears to me that they made it in a way where they can add new platforms by simply adding new builders, eg. AssemblyBuilderA64, AssemblyBuilderX64, etc., and it takes a compiled function from the VM's stack at some index, and converts it into native code using one of the builders. Oh, and I just want to note that it's definitely intended to be used on clients at some point, because otherwise it would be pretty pointless to be on Roblox's servers where the architecture is always the same. I think that's their goal in the future based on one of their forum posts. |
|
Hello everyone, I have made a prototype JIT compiler for GDScript. On this example: func fib(iter: int) -> int:
var n1 := 0
var n2 := 1
for i in range(0, iter):
var n := n2
n2 = n2 + n1
n1 = n
return n2My JIT is around 60-70% faster. And it can call functions directly, but I haven't tested it with function pointers to the engine. I am using cranelift for codegen, which is written in rust, so likely won't be used in Godot. Cranelift is also better suited than LLVM for JIT compilation, because it takes fewer steps when producing machine code, while being only ~14% slower. I will continue working on it, and maybe at some point it will be good enough for use in production. I also would like to add something to the discussion:
|
|
LLVM is very easy to use. I made a new language with it in under a month when having no experience. If you guys already went through the effort of making a bytecode, using LLVM should be a piece of cake. You could generate LLVM IR straight from your bytecode format. As for linking with your C++ codebase... you can support the standard Itanium ABI scheme and compile using Clang to make sure its used as you expect. LLVM has a tool for determining mangled names in Itanium: https://llvm.org/docs/CommandGuide/llvm-cxxmap.html. |
|
I actually agree. I also started with zero knowledge of making compilers (and writing rust). And was able to make a prototype GDScript compiler. But I hit a roadblock, which was GDExtensions (wasn't able to find any documentation). Currently, LLVM seems (to me) like the best solution for Godot. Still, I would argue that a JIT, would be faster, since with JIT compilers you don't need PIC code, which makes it faster to call functions. And really most of the things GDScript does it calling functions. Edit: If compiled code will be statically linked, then this doesn't apply. But I am not sure how this would work I was able to only find this benchmark, which says that when calling via function pointer, -fPIC causes a 0 to 30% performance hit. And it's not a problem as long as they're not used in the innermost loops LLVM has a JIT, but LLVM takes it's time to compile, which can be a problem during startup, though this can be worked around with first using an interpreter then switching to compiled code, which adds complexity and makes it harder to maintain. Cranelift would be perfect for this, as it generates fast code quickly. If it wasn't as low level (btw, I am working on fixing this) and had C bindings. |
|
I disagree with the idea of native compilation being the solution for GDScript, at least at first. I think a great amount of work can be done with the VM as it is to improve performance. Here's my issues with native compilation:
This isn't to say I disagree with compiled languages - in fact, I prefer them - but GDScript is a scripting language and we should play to its strengths as a run-anywhere embedded platform. We need look no further for examples of very fast interpreted languages than Lua (although I recognize LuaJIT) or Wren. I think we can get competitive with them without static compilation. Here's what I propose:
I recognize a lot of this already increases complexity, but I would argue this is less of an ask than including LLVM, either internally or externally. Also, the less external dependencies, the better, particularly on platforms like Android. As for the JIT question - technically, that falls under the native code problems, since it executes platform-specific machine code. On the other hand, it is without a doubt a huge speed boost and valuable for games. I am on the fence about this. If we do go this route, it should be a tracing compiler, since so much of game flow runs on loops. I would love to contribute to GDScript, and if I should start poking away at this in my spare time, just give me the go-ahead. |
|
|
@SlashScreen Nice to hear you want to do some work in gdscript. I happen to be reading gdscript modules, here are some points which may help:
As far as I see, GDScript compiler is a one-pass compiler, which directly gen final form code from AST walking, if you are looking for sometype of IR you might be disappointed. This might be due to the need to compile code during game loading. And as long as we still need to compile on loading, we are facing resistance in doing complex optimizations.
These logic only present in debug build (
Actually we seem to lose performance for static typing, take Typed Array AssignOPCODE(OPCODE_ASSIGN_TYPED_ARRAY) {
CHECK_SPACE(6);
GET_VARIANT_PTR(dst, 0);
GET_VARIANT_PTR(src, 1);
GET_VARIANT_PTR(script_type, 2);
Variant::Type builtin_type = (Variant::Type)_code_ptr[ip + 4];
int native_type_idx = _code_ptr[ip + 5];
GD_ERR_BREAK(native_type_idx < 0 || native_type_idx >= _global_names_count);
const StringName native_type = _global_names_ptr[native_type_idx];
if (src->get_type() != Variant::ARRAY) {
OPCODE_BREAK;
}
Array *array = VariantInternal::get_array(src);
if (array->get_typed_builtin() != ((uint32_t)builtin_type)
|| array->get_typed_class_name() != native_type
|| array->get_typed_script() != *script_type) {
OPCODE_BREAK;
}
*dst = *src;
ip += 6;
}
DISPATCH_OPCODE;Untyped AssignOPCODE(OPCODE_ASSIGN) {
CHECK_SPACE(3);
GET_VARIANT_PTR(dst, 0);
GET_VARIANT_PTR(src, 1);
*dst = *src;
ip += 3;
}
DISPATCH_OPCODE;The type is checked at runtime and we still fall into the same path as untyped assign. I guess nothing has been done yet to speedup typed code.
I have no preference between stack and register based, but I support the idea of simplifying instruction set (even at the cost of some performance), a 3500 LOC function is definitely not something easy to deal with. Also the jump threading implementation could be reevaluated. Based on this paper, it may not providing enough performance gain to make up the maintainability cost. |
You're right, and I'm not arguing against the idea as a whole, despite what it may seem; I simply believe our efforts should be focused elsewhere in fixing the systemic issues of the language rather than slapping an LLVM bandaid on it. It should be optional (even a GDExtension) and not be default.
You're right about this, actually.
I figured this was the case. SSA might be doable in the AST stage, but I doubt it would be worth the trouble.
Perhaps I misunderstood what AOT means in this context; If it actually means compiling into bytecode on project export, I am actually for the idea. Although, for game mods and other things, there should retain the ability to do runtime compilation, and/or the ability to load bytecode directly. I'm worried, though, that it could become like C# if we take this too far (that is to say, cumbersome and kind of annoying).
That I understand; what I meant is that a lot of the logic is present in the LSP, and just needs to be copied and pasted and then modified to suit the environment.
Absolutely. The first changes I would probably make are
The register machine idea is attractive, but would also require a redesign on the entire VM and significant compiler adjustments. But, y'know, is that a bad thing? |
|
Idea that occurred to me. Have we considered just adding the option of running GDScript on top of the .NET VM? We already ship the runtime with the C# version and we already have all the plumbing with regards to the bindings. Seems to me like the least-effort way to get immediate performance. I don't think it's realistic to maintain something with better performance than .NET. I think there are a LOT of developers that would benefit from the .NET VM without having to learn C#, and developers that actually hit the garbage collection issues should be using a language with explicit memory semantics, which GDScript isn't. Maybe we shouldn't be trying to make GDScript into the N+1th compiled language. |
|
That isn't a bad idea, actually. A few comments, though:
I do absolutely agree about not making GDScript the umpteenth compiled language, especially since it wasn't designed as such and so trying to force it to become something it was never meant to be would be awkward. |
the .Net runtime is not shipped with the version of Godot used by most users, Moving it to a GDExtension means that users just download Godot and install the C# addon when needed ; |
If we do make GDScript run on the .net VM (called CLR), it would probably be with the goal to move to it definitively. Potential benefits and drawbacks of moving Gdscript to the CLR :
|
|
I'd rather have GDScript not depend on .NET at all. LLVM would be a good idea for CPP, but make Godot tell the user to download it from a certain link and a guide to set it up. |
It wouldn't be a dependency, it would be an optional backend, just as LLVM would. And we could also link dynamically to a system .NET installation too, if the concern is bloating the binary. Both should be plugins, though. |
|
If it becomes a CLI-based language, you might as well just adopt Boo as the language for Godot.
… Message ID: ***@***.***>
|
|
If you guys want to work on alternative backends for GDScript, you can, but I don't have much interest in doing it myself. What I'm going to do next is attempt to reduce the instruction set and refactor the main eval function to be easier to work with in a PR. I think that's an achievable first step. |
and others
Describe the project you are working on
Godot
Describe the problem or limitation you are having in your project
Many users have complained for a long time about the performance in GDScript.
For the GDScript 2.0 rewrite, a lot of work was put into making sure the VM has all the information required to generate optimized code, including type information, strict type checking, better addressing modes and the ability to access every operator, utility function, etc. as a C function pointer. This can be evidenced by the new codegen class used to emit code.
Unfortunately, just the changes to the parser and compiler ended up being very significant so almost no optimization work could take place for the VM.
Describe the feature / enhancement and how it helps to overcome the problem or limitation
The idea is to use the the new type information as well as all the C function pointers to engine to generate direct executable code and bypass the VM, allowing for much greater performance when typed code is used.
This is challenging to do in a practical way, however. Because of it, this is more of an open discussion on evaluating different alternatives.
Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams
There are a few general approaches that could be taken to optimize GDScript:
The problem with JIT and real-time AOT is that there are many platforms that do not support real-time code generation, such as iOS, WebAssembly or consoles. Because of this, if we followed this approach, Godot would not run well on those.
As a result, the ideal way to compile GDScript code to binary would be to compile the whole script to a shared object (DLL on Windows, .so on Linux, .dylib on MacOS, etc) that can be bundled together with the exported game.
Fortunately, GDExtension provides pretty much everything the compiled script needs:
The question that remains is how to build cross-platform code from the editor. There are a few alternatives to doing this:
External compiler
This is one of the easiest solutions, but it can be quite bothersome for the user to have to do this usability wise. It would make distributing Godot harder if the intention is to make sure code just runs fast. For some platforms like WebAssembly, I am not sure if there is any other way though.
There are tiny and embedded compilers that may be of use, but none really seems fit our needs:
LLVM / QBE
Another alternative would be to integrate LLVM or QBE into Godot and generate the binary on the fly.
LLVM looks like a lot of work though, since supposedly you have to implement the C ABI code yourself and its codebase is huge. QBE sounds like a much better project for this (it already has full C ABI support) but, while active, its developed by a smaller team and they don't use GitHub.
Lower level binary geneator
There are lower level libraries like SLJIT that do cross-platform codegen. This could work as an AOT compiler for platforms that can do this, and the code generated could be dumped to an shared object in advance on export for the platforms that do not support it (this would need some work).
The downside of this is that these libraries perform no optimization, so GDScript would need its own SSA form optimizer. This may not be terrible, but its some work.
So, what do you think? how would you tackle this problem in a way where the usability and maintainability balance is good in our favor?
If this enhancement will not be used often, can it be worked around with a few lines of script?
N/A
Is there a reason why this should be core and not an add-on in the asset library?
N/A
The text was updated successfully, but these errors were encountered: