Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compiler: more progress on incremental #19273

Merged
merged 10 commits into from
Mar 14, 2024
Merged

Conversation

mlugg
Copy link
Member

@mlugg mlugg commented Mar 12, 2024

We're deep in the weeds now. As of this PR trivial updates are blocked on linker progress (or CBE/LLVM backend progress), but I can still make progress in the meantime.

Before I make more steps towards incremental, I need to have a stern talking-to with Zcu.Decl (see my proposed refactor) - but I think we're very close to something usable on the frontend.

@mlugg mlugg force-pushed the incremental-some-more branch 4 times, most recently from deabaa8 to 9f0d142 Compare March 14, 2024 00:27
mlugg added 10 commits March 14, 2024 07:40
There is no reason to perform this detection during semantic analysis.
In fact, doing so is problematic, because we wish to utilize detection
of existing decls in a namespace in incremental compilation.
This makes tracking easier across incremental updates: `scanDecl` can
now tell whether an existing decl in a namespace was mapped to the one
it's analyzing in the new ZIR.
Most basic re-analysis logic is now in place. Trivial updates are
hitting linker assertions.
This was an oversight in my original design. This new form of dependency
is invalidated when the resolved IES for a runtime function changes.
See comment. This slightly regresses a previous fix from this branch. A
proper solution will come soon, with the splitting up of `Decl` into
`Cau` and `Nav`.
@mlugg mlugg force-pushed the incremental-some-more branch from 9f0d142 to 7c32370 Compare March 14, 2024 07:40
@mlugg mlugg merged commit 39459e7 into ziglang:master Mar 14, 2024
10 checks passed
// Do a pass over the namespace contents and remove any decls from the last update
// which were removed in this one.
var i: usize = 0;
while (i < namespace.decls.count()) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing i += 1 ?

Copy link
Contributor

@nektro nektro Mar 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4294 should make it unnecessary but 4295 would immediately crash

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, you're totally right, good spot! Luckily this doesn't affect code built without incremental, and this is going to be changed a lot again soon anyway, but thank you for pointing out the issue.

@andrewrk
Copy link
Member

andrewrk commented Mar 14, 2024

In case you didn't think of it already, a nice stepping stone might be an incremental -fno-emit-bin build. No backend or linker coordination needed, but you can still make the frontend save its work as far as semantic analysis and state serialization/restoration. In fact this configuration might be the very one that IDEs are interested in - getting analysis results quickly with no codegen.

Furthermore - and this is the cool part - once frontend incremental is robust we could enable it by default when -fno-emit-bin is passed, before doing any of that work for linker/codegen components.

@mlugg
Copy link
Member Author

mlugg commented Mar 14, 2024

Yeah, that thought has crossed my mind. I need to step back and do some prereq work for now - i.e. mutilating comptime-mutable memory and Decl - but once I get back to this I'll probably look into doing -fno-emit-bin incremental first. Once that's usable, we could probably look more seriously at #615.

const decl = zcu.declPtr(decl_index);
if (!seen_decls.contains(decl.name)) {
// We must preserve namespace ordering for @typeInfo.
namespace.decls.orderedRemoveAt(i);
Copy link
Member

@andrewrk andrewrk Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't put a orderedRemove() inside a loop. That's O(N^2). I would have considered this to be a merge blocker.

Looks like this is some big strides forward otherwise though. Nice! 🙂

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, apologies - I didn't think of that. This code is only hit in the incremental path, and it'll probably be changed again anyway after the Decl changes, but I'll make sure this gets fixed!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants