ship even more of the compiler in source form #19063
Comments
andrewrk
added
the
proposal
|
Possibly related proposal This proposal would allow other zig projects to ship zig source code rather than compiled binaries as well. |
|
Shipping current version of Autodoc in source form would require also shipping ZIR and parts of InternPool |
|
I like this. It's always irked me that we have things like |
|
because that logic uses the cache but needs to be callable by |
|
does this proposal being implemented mean for example that |
No. It would be the same CLI. |
|
So same CLI, and it would re-build the relevant source files if necessary - like how |
part of #19063 This is a prerequisite for doing the same for Resinator.
part of #19063 This is a prerequisite for doing the same for Resinator.
part of #19063 This is a prerequisite for doing the same for Resinator.
Part of #19063. Primarily, this moves Aro from deps/ to lib/compiler/ so that it can be lazily compiled from source. src/aro_translate_c.zig is moved to lib/compiler/aro_translate_c.zig and some of Zig CLI logic moved to a main() function there. aro_translate_c.zig becomes the "common" import for clang-based translate-c. Not all of the compiler was able to be detangled from Aro, however, so it still, for now, remains being compiled with the main compiler sources due to the clang-based translate-c depending on it. Once aro-based translate-c achieves feature parity with the clang-based translate-c implementation, the clang-based one can be removed from Zig. Aro made it unnecessarily difficult to depend on with these .def files and all these Zig module requirements. I looked at the .def files and made these observations: - The canonical source is llvm .def files. - Therefore there is an update process to sync with llvm that involves regenerating the .def files in Aro. - Therefore you might as well just regenerate the .zig files directly and check those into Aro. - Also with a small amount of tinkering, the file size on disk of these generated .zig files can be made many times smaller, without compromising type safety in the usage of the data. This would make things much easier on Zig as downstream project, particularly we could remove those pesky stubs when bootstrapping. I have gone ahead with these changes since they unblock me and I will have a chat with Vexu to see what he thinks.
Part of #19063. Primarily, this moves Aro from deps/ to lib/compiler/ so that it can be lazily compiled from source. src/aro_translate_c.zig is moved to lib/compiler/aro_translate_c.zig and some of Zig CLI logic moved to a main() function there. aro_translate_c.zig becomes the "common" import for clang-based translate-c. Not all of the compiler was able to be detangled from Aro, however, so it still, for now, remains being compiled with the main compiler sources due to the clang-based translate-c depending on it. Once aro-based translate-c achieves feature parity with the clang-based translate-c implementation, the clang-based one can be removed from Zig. Aro made it unnecessarily difficult to depend on with these .def files and all these Zig module requirements. I looked at the .def files and made these observations: - The canonical source is llvm .def files. - Therefore there is an update process to sync with llvm that involves regenerating the .def files in Aro. - Therefore you might as well just regenerate the .zig files directly and check those into Aro. - Also with a small amount of tinkering, the file size on disk of these generated .zig files can be made many times smaller, without compromising type safety in the usage of the data. This would make things much easier on Zig as downstream project, particularly we could remove those pesky stubs when bootstrapping. I have gone ahead with these changes since they unblock me and I will have a chat with Vexu to see what he thinks.
Part of #19063. Primarily, this moves Aro from deps/ to lib/compiler/ so that it can be lazily compiled from source. src/aro_translate_c.zig is moved to lib/compiler/aro_translate_c.zig and some of Zig CLI logic moved to a main() function there. aro_translate_c.zig becomes the "common" import for clang-based translate-c. Not all of the compiler was able to be detangled from Aro, however, so it still, for now, remains being compiled with the main compiler sources due to the clang-based translate-c depending on it. Once aro-based translate-c achieves feature parity with the clang-based translate-c implementation, the clang-based one can be removed from Zig. Aro made it unnecessarily difficult to depend on with these .def files and all these Zig module requirements. I looked at the .def files and made these observations: - The canonical source is llvm .def files. - Therefore there is an update process to sync with llvm that involves regenerating the .def files in Aro. - Therefore you might as well just regenerate the .zig files directly and check those into Aro. - Also with a small amount of tinkering, the file size on disk of these generated .zig files can be made many times smaller, without compromising type safety in the usage of the data. This would make things much easier on Zig as downstream project, particularly we could remove those pesky stubs when bootstrapping. I have gone ahead with these changes since they unblock me and I will have a chat with Vexu to see what he thinks.
Part of ziglang#19063. Primarily, this moves Aro from deps/ to lib/compiler/ so that it can be lazily compiled from source. src/aro_translate_c.zig is moved to lib/compiler/aro_translate_c.zig and some of Zig CLI logic moved to a main() function there. aro_translate_c.zig becomes the "common" import for clang-based translate-c. Not all of the compiler was able to be detangled from Aro, however, so it still, for now, remains being compiled with the main compiler sources due to the clang-based translate-c depending on it. Once aro-based translate-c achieves feature parity with the clang-based translate-c implementation, the clang-based one can be removed from Zig. Aro made it unnecessarily difficult to depend on with these .def files and all these Zig module requirements. I looked at the .def files and made these observations: - The canonical source is llvm .def files. - Therefore there is an update process to sync with llvm that involves regenerating the .def files in Aro. - Therefore you might as well just regenerate the .zig files directly and check those into Aro. - Also with a small amount of tinkering, the file size on disk of these generated .zig files can be made many times smaller, without compromising type safety in the usage of the data. This would make things much easier on Zig as downstream project, particularly we could remove those pesky stubs when bootstrapping. I have gone ahead with these changes since they unblock me and I will have a chat with Vexu to see what he thinks.
This could be very interesting, I imagine specifying a 3rd party package containing a machine backend I need through my build.zig.zon, which could be very good for embedded projects. |
This moves .rc/.manifest compilation out of the main Zig binary, contributing towards ziglang#19063 Also: - Make resinator use Aro as its preprocessor instead of clang - Sync resinator with upstream
This moves .rc/.manifest compilation out of the main Zig binary, contributing towards ziglang#19063 Also: - Make resinator use Aro as its preprocessor instead of clang - Sync resinator with upstream
This moves .rc/.manifest compilation out of the main Zig binary, contributing towards ziglang#19063 Also: - Make resinator use Aro as its preprocessor instead of clang - Sync resinator with upstream
part of ziglang#19063 This is a prerequisite for doing the same for Resinator.
Part of ziglang#19063. Primarily, this moves Aro from deps/ to lib/compiler/ so that it can be lazily compiled from source. src/aro_translate_c.zig is moved to lib/compiler/aro_translate_c.zig and some of Zig CLI logic moved to a main() function there. aro_translate_c.zig becomes the "common" import for clang-based translate-c. Not all of the compiler was able to be detangled from Aro, however, so it still, for now, remains being compiled with the main compiler sources due to the clang-based translate-c depending on it. Once aro-based translate-c achieves feature parity with the clang-based translate-c implementation, the clang-based one can be removed from Zig. Aro made it unnecessarily difficult to depend on with these .def files and all these Zig module requirements. I looked at the .def files and made these observations: - The canonical source is llvm .def files. - Therefore there is an update process to sync with llvm that involves regenerating the .def files in Aro. - Therefore you might as well just regenerate the .zig files directly and check those into Aro. - Also with a small amount of tinkering, the file size on disk of these generated .zig files can be made many times smaller, without compromising type safety in the usage of the data. This would make things much easier on Zig as downstream project, particularly we could remove those pesky stubs when bootstrapping. I have gone ahead with these changes since they unblock me and I will have a chat with Vexu to see what he thinks.
This moves .rc/.manifest compilation out of the main Zig binary, contributing towards ziglang#19063 Also: - Make resinator use Aro as its preprocessor instead of clang - Sync resinator with upstream
Well, that explains it...
This is just speculation, but I feel like JIT would synergize really well with comptime. Especially for things like
I was thinking something similar. Custom backends sound like a big deal for non-mainstream platforms, including proprietary, niche, or even in-development architectures, including experiments and pet projects. Overall, I am a bit concerned about compile times, though. I know caching is a thing, but it's not perfect. |
|
I'm half-Indian so I think that last bullet could use a bit more heat. An observation: whatever gets compiled on-demand will have native opt, but whatever we ship as binary won't -- and if one of those things is the compiler itself, then compiling everything on-demand may incur a noticeable delay. A solution: ship the entire project as source, and provide only a single, tiny binary: the compiler, itself compiled for the absolute bare-bones no-extensions version of the host, with only the host platform backend, but capable of outputting code using all possible features. Upon install, this binary detects the host's features, compiles a copy of itself using native opt, then deletes itself and hands control to the copy. Now obviously this is pretty wacky: infeasible for the moment, annoying for the first time installer. When it becomes possible though, it will in fact make up the lost time the very first time a serious project is compiled. Also it's just cool to not have a single bit of externally-compiled code in the installed package. || |
You know what's really pleasant to work on? The build system. The zig build system is shipped in source form, and when you run
zig build, zig compiles the build runner from source, and then runs it. In order to do that, the compiler needs only 2 capabilities: compiling source into a native executable, and executing child processes.Here are some components that could be shipped in source form rather than compiled into the zig binary:
all the machine code backends other than the one used to produce native executablesseparate proposalAll of these components have simple interfaces that could be communicated over IPC or the file system. I admit however that last one is a bit spicy.
Implementing this issue would accomplish the following things:
-Denable-llvm=false -Doptimize=ReleaseSmallon x86_64 is 2.9 MiB (9.9 MiB uncompressed), while the compiler source code catted together into an xz stream is 1.6 MiB.The text was updated successfully, but these errors were encountered: