Add a blog post on precompilation

[timholy]

I'm preparing new material on how to reduce latency and improve precompilation in packages, but I realized I would benefit from some introductory material before discussing the tools (e.g., before returning to #1093). Hopefully this post will introduce all the concepts I need.

[github-actions]

Once the build has completed, you can preview your PR at this URL: https://julialang.netlify.app/previews/PR1111/ in ~15 minutes

[timholy]

Pinging some participants in https://discourse.julialang.org/t/understanding-and-optimizing-compiler-time-just-a-bit/51759 and https://discourse.julialang.org/t/understanding-precompilation-and-its-limitations-reducing-latency/48294: @tpapp, @chriselrod, @jkrumbiegel, @joa-quim.

[tlienart]

Small Franklin note which you could ignore: if you use a triple backtick with a ! then the code in there gets executed and the output is shown on the page, the interest in doing this is that if ever the output changes slightly, it will be updated automatically. So for instance:

```!
c64 = [1.0]
```

-->

[timholy]

Thanks for the tip! I'm not sure I'll use it due to changes in formatting and the fact that it seems to need an isolated statement, but I will keep that in mind.

I just pushed an update that may hint at something I'm trying but failing to do: I'd be interested in exploring having "boxes" for different types of content: asides, exercises/quizzes, and perhaps other topics. As you'll see, I pushed a .css file as well as my markdown, but this is me just imitating things I've found on the internet without actually understanding what I'm doing. Is this easy or hard? If the former, any pointers?

[tlienart]

You can do this with @@ in Franklin this will surround content in a div block of a given class (or classes): e.g.

@@note
content here
@@

Will inject the content in a <div class="note">...</div>". For several classes use @@Class1,class2,class3 content @@`. I think that should be all you need but don't hesitate to ping me from here :)

PS: and plug your custom CSS at the end of https://github.com/JuliaLang/www.julialang.org/blob/master/_css/franklin.css or, if it bothers you to put it there, you can also inject it at the top of your post with:

~~~
<style>
your css here
</style>
~~~

whichever one you think is easier (the advantage of putting your CSS in the _css/* folder is that everyone can re-use the stuff you define in there)

[chriselrod]

While a lot of work has gone into Julia 1.6 to reduce latency, users and developers naturally will want more.

More work, but less latency. ;)

I think it looks good and found it informative/helpful. I have some playing around to do!

[timholy]

If you're looking into fixing inference, the upcoming (on master) inference_triggers framework in SnoopCompile is the bomb 😆.

[tpapp]

This is very informative, thanks for writing it up! As I am still digesting this, I don't have much to add yet, except that it is unclear how to get a method instance object from the @snoopi_deep and similar output that will come in later installments.

[timholy]

Boxes are working great, thanks @tlienart!

Barring suggested changes, I'm currently targeting this for a Jan 5 release (just to give people a few days to get back in the swing).

[NHDaly]

🌟 This was great! 🌟

This was such a useful introduction to many of the topic at play, and I think will do a good job of helping people to understand what's going on under the hood. Thanks for a really nice read! :) :)

Thank you for writing this all up so clearly and explaining it so well!

[NHDaly]

which can cause the same MethodInstance to be re-inferred for different constant values.

Or even reinferred for the same constant value, across different inferrable sets! 😮

julia> using SnoopCompile

julia> h(x) = x == 0 ? 0 : h(x-1)+1
h (generic function with 1 method)

julia> f(x) = h(sizeof(x))
f (generic function with 1 method)

julia> t = @snoopi_deep f(0.0)
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.Compiler.Timings.ROOT()) with 1 children

julia> flatten_times(t)
8-element Vector{Tuple{Float64, Float64, Core.Compiler.Timings.InferenceFrameInfo}}:
 (5.1809e-5, 5.1809e-5, InferenceFrameInfo for -(8::Int64, 1::Int64))
 (5.8423e-5, 5.8423e-5, InferenceFrameInfo for ==(8::Int64, 0::Int64))
 (6.0468e-5, 6.0468e-5, InferenceFrameInfo for +(0::Int64, 1::Int64))
 (9.3104e-5, 9.3104e-5, InferenceFrameInfo for sizeof(::Float64))
 (0.000327155, 0.000387623, InferenceFrameInfo for h(::Int64))
 (0.000330497, 0.000440729, InferenceFrameInfo for h(8::Int64))
 (0.00037169, 0.001293146, InferenceFrameInfo for f(::Float64))
 (0.013208022, 0.014501168, InferenceFrameInfo for Core.Compiler.Timings.ROOT())

julia> t = @snoopi_deep f(0)
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.Compiler.Timings.ROOT()) with 1 children

julia> flatten_times(t)
5-element Vector{Tuple{Float64, Float64, Core.Compiler.Timings.InferenceFrameInfo}}:
 (3.51e-5, 3.51e-5, InferenceFrameInfo for -(8::Int64, 1::Int64))
 (6.2299e-5, 6.2299e-5, InferenceFrameInfo for ==(8::Int64, 0::Int64))
 (0.000144853, 0.000422476, InferenceFrameInfo for f(::Int64))
 (0.000180224, 0.000277623, InferenceFrameInfo for h(8::Int64))
 (0.00266502, 0.003087496, InferenceFrameInfo for Core.Compiler.Timings.ROOT())

^ Note that h(8) had to be inferred again, even though it was identical.

But within the same top-level call to inference, it will cache inference with constants, and so only infer h(8) once:

julia> using SnoopCompile

julia> g(x) = f(x) + f(Int(x))
g (generic function with 1 method)

julia> f(x) = x + h(sizeof(x))
f (generic function with 1 method)

julia> h(x) = x == 0 ? 0 : h(x-1)+1
h (generic function with 1 method)

julia> t = @snoopi_deep g(0.0)
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.Compiler.Timings.ROOT()) with 1 children

julia> flatten_times(t)
10-element Vector{Tuple{Float64, Float64, Core.Compiler.Timings.InferenceFrameInfo}}:
 (5.4203e-5, 5.4203e-5, InferenceFrameInfo for -(8::Int64, 1::Int64))
 (6.23e-5, 6.23e-5, InferenceFrameInfo for +(0::Int64, 1::Int64))
 (6.2768e-5, 6.2768e-5, InferenceFrameInfo for ==(8::Int64, 0::Int64))
 (8.6434e-5, 8.6434e-5, InferenceFrameInfo for sizeof(::Float64))
 (0.000180616, 0.000180616, InferenceFrameInfo for f(::Int64))
 (0.000284891, 0.000401862, InferenceFrameInfo for h(8::Int64))
 (0.000298722, 0.000361022, InferenceFrameInfo for h(::Int64))
 (0.000344378, 0.001193696, InferenceFrameInfo for f(::Float64))
 (0.000429948, 0.00180426, InferenceFrameInfo for g(::Float64))
 (0.017256332, 0.019060592, InferenceFrameInfo for Core.Compiler.Timings.ROOT())

---

I've asked Jameson, Jeff and Keno about this, and they explained it's a deliberate decision because they expect functions are rarely compiled with the same constant over multiple top-level inferences, and the overhead from caching it might be too expensive relative to gain.

BUT i think we may suffer from this in our database, because we are doing query compilation on the fly at runtime, and we want user-visible latency to be as low as possible, and i think we may be compiling some methodinstances repeatedly for every new query.

We suggested that it might make sense to at least keep a heuristic that tracks how many times a methodinstance has been inferred for the exact same constant, and after maybe the third time we start caching it? (This would make us behave a bit more like a profiling JIT!)

[NHDaly]

Anyway, up to you if this is too in the weeds, but you could consider updating this sentence to be something like:

Suggested change

Type inference behaves similarly: it caches its results, and thus infers each `MethodInstance` only once. (One wrinkle is [constant propagation](https://en.wikipedia.org/wiki/Constant_folding), which can cause the same `MethodInstance` to be re-inferred for different constant values.) As a consequence, inference also performs a depth-first search of the call graph.

Type inference behaves similarly: it caches its results, and thus infers each `MethodInstance` only once. (One wrinkle is [constant propagation](https://en.wikipedia.org/wiki/Constant_folding), which can cause the same `MethodInstance` to be re-inferred for different constant values, and sometimes even re-inferred for the same value.) As a consequence, inference also performs a depth-first search of the call graph.

[timholy]

If I were making other changes, I'd do this, but on its own but I think the question of persistence of inference's cache is a bit more detail than readers need.

[NHDaly]

Woah, this is WILD! Fascinating. Thanks, TIL! :)

[timholy]

Awesome! Let's write the next one together? I'd like to have one/two on the GUIs, the flamegraph and my (still morphing but mostly stable) "profile-guided despecialization" (PGDS) GUI.

The PGDS stuff gets into complex territory that I'm still figuring out, but probably best just to get out what we know and let the community help figure it out...