Precompile all the CodeInstances

[timholy]

This is an attempt at allowing packages to save the entirety of their type-inferred code, which may result in a substantial reduction in latency for many packages. Currently we cache CodeInstances for the module in which they are defined, but not ones that get generated by being called from some other package. This is essentially a replacement for #32705, attempting to solve the "roots problem" without the massive duplication that doomed that approach. One interesting feature of the approach I've taken here is that it allows us to save type-inferred code even if it's hidden behind a runtime dispatch (no backedges from "owned" methods required).

Very briefly, our serialization ultimately refers to a list of objects (the roots associated with each method) by index. The problem is, fresh compilation might add to the roots table, and since it grows by order-of-insertion, there's a big risk:

• Imagine that PkgA calls Base.push!(::Vector{TypPkgA}, ::TypPkgA) for some new type defined in PkgA. Thus the corresponding Base method push!(a::Vector{T}, item) where T gets TypPkgA inserted in its roots table.
• Let's imagine that when precompiling PkgA on its own, this gets inserted at index 90 in that method's roots table. So, when precompilation saves the *.ji cache file, root index 90 will "mean" TypPkgA
• Start a fresh session, and say using PkgB first, then say using PkgA. Let's imagine that PkgB also generates a CodeInstance for the same method, and inserts some other type, TypPkgB. Since this package got loaded first, TypPkgB occupies index 90 in the roots table, and some code has already been deserialized and cached in the global code store assuming this to be the case (so we can't change it).
• Now say using PkgA. When deserialized, the CodeInstance for Base.push!(::Vector{TypPkgA}, ::TypPkgA) will refer to index 90 in the roots table, but unfortunately this now means TypPkgB. 💣

The starting point for this PR was the question, "If that's so bad, why can we get away with caching CodeInstances for internal methods?" The reason is that when a package is serialized, it has two properties: (1) the methods the package "owns" were never defined before loading the package, so there is no risk that some other package will "get there first" and insert things earlier in the roots table; and (2) it is serialized in a "clean" environment, with nothing extra added, so we can be sure that all the objects in the roots table are available in the package and the things it depends on.

This PR takes the approach that we might be able to safely cache new CodeInstances for old modules if we are careful about how we encode root indexes. The idea is to encode the "extra" root indexes not with an absolute index, but a relative one. Essentially, whenever you're about to extend the roots table from outside the module, you record the index of the "old" end of the table, and everything that gets added henceforth will be encoded via relative index, i.e., from the old end of the roots table rather than an absolute index. When you deserialize, the idea is that you use the relative offset from whatever the roots table end now happens to be. Since we still have the "clean environment" property (2) above, this seems likely to be fairly well controlled.

This is definitely not working yet, but I think it may be close. My sense is that mostly what's missing is mostly or entirely on the deserialization side:

• after deserializing a CodeInstance that makes reference to a TAG_EXTERN_METHODROOT, we'd want to re-serialize and update the global code cache with absolute (traditional) encoding of roots. This is necessary to enable the next package to exploit the same process. Effectively, the relative indexing can only last during the window in which a package is being loaded, and once that's over we have to set the state of the system to be the same as if we had never used the relative offsets.
• the restoration of the various backedges, external MethodInstances, etc, has not even been attempted yet.

I don't have a complete understanding of this aspect of dump.c, so I could really use some help. I do think this will be a pretty big step forward, so I hope one or more experts can coach me through the remainder. If, that is, the strategy isn't doomed for reasons I've failed to consider.

[timholy]

I am hoping I'll have time to get back to this over the holidays. If anyone who knows this part of the codebase has a chance to glance at this and see if it's moving in the right direction, I'd appreciate it.

[timholy]

This feels very close. I'm still having some memory-corruption issues but the overall shape seems essentially complete.

Some of the bigger changes:

• the roots table is now (nearly) fully relocatable in terms of module-loading order, etc. Roots are organized in blocks indexed by the build_id of the toplevel worklist module. One requirement is that each build_id can only appear once, so its roots have to be gathered together.
• when doing incremental compilation, disable IR compression until we serialize. (Currently this may be revealing a bug in the inlining code.) This is intended to keep like with like (roots in blocks).
• there's still a bit of global state (currently_serializing and currently_deserializing) I mean to strip out.

There's significant risk my window for working on this is about to close again, but it would be lovely to get it working so we can at least test it out.

[timholy]

I am thinking about splitting this into smaller PRs, to make it easier to debug & review. First one is up at #43759.