Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🧂 more precompile for dynamic functions! #1978

Merged
merged 12 commits into from
Mar 24, 2022
Merged

🧂 more precompile for dynamic functions! #1978

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
e8531ed
Update WebServer.jl
fonsp Mar 12, 2022
a12233e
Update MsgPack.jl
fonsp Mar 12, 2022
e693252
Update Dynamic.jl
fonsp Mar 12, 2022
b417fda
Revert "Update MsgPack.jl"
fonsp Mar 12, 2022
ce5ede9
Update MsgPack.jl
fonsp Mar 12, 2022
bc9bbf2
Update src/webserver/Dynamic.jl
fonsp Mar 23, 2022
b5fa292
Update src/webserver/MsgPack.jl
fonsp Mar 23, 2022
bc0a843
Update src/webserver/WebServer.jl
fonsp Mar 23, 2022
3c16a38
Merge branch 'main' into random-precompile-statements
fonsp Mar 23, 2022
847e309
fix notebook_path_suggestion precompilation
fonsp Mar 24, 2022
bee1811
blabla
fonsp Mar 24, 2022
9a2fa52
tests please
fonsp Mar 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions src/Configuration.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,17 @@ using Configurations # https://github.com/Roger-luo/Configurations.jl

import ..Pluto: tamepath

# This can't be a simple `const` because this would hard-code it into the precompile image.
const notebook_path_suggestion_ref = Ref{Union{Nothing,String}}(nothing)
# Using a ref to avoid fixing the pwd() output during the compilation phase. We don't want this value to be baked into the sysimage, because it depends on the `pwd()`. We do want to cache it, because the pwd might change while Pluto is running.
const pwd_ref = Ref{Union{Nothing,String}}()
function notebook_path_suggestion()
if notebook_path_suggestion_ref[] === nothing
notebook_path_suggestion_ref[] = let
preferred_dir = startswith(Sys.BINDIR, pwd()) ? homedir() : pwd()
joinpath(preferred_dir, "") # so that it ends with / or \
end
end
notebook_path_suggestion_ref[]
pwd_val = something(pwd_ref[], pwd())
preferred_dir = startswith(Sys.BINDIR, pwd_val) ? homedir() : pwd_val
# so that it ends with / or \
string(joinpath(preferred_dir, ""))
end

function __init__()
pwd_ref[] = pwd()
end

"""
Expand Down
2 changes: 2 additions & 0 deletions src/webserver/Dynamic.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ end
# All of the arrays in the notebook_to_js object are 'immutable' (we write code as if they are), so we can enable this optimization:
Firebasey.use_triple_equals_for_arrays[] = true


# the only possible Arrays are:
# - cell_order
# - cell_execution_order
Expand Down Expand Up @@ -169,6 +170,7 @@ function notebook_to_js(notebook::Notebook)
"cell_execution_order" => cell_id.(collect(topological_order(notebook))),
)
end
precompile(notebook_to_js, (Notebook,))
Copy link
Collaborator

@rikhuijzer rikhuijzer Mar 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slight improvement:

Without precompile

$ julia --project -ie 'using Pluto'

julia> @time @eval Pluto.notebook_to_js(Pluto.Notebook([Pluto.Cell("1")]));
  3.078880 seconds (6.69 M allocations: 376.505 MiB, 3.90% gc time, 91.05% compilation time)

With precompile

julia> @time @eval Pluto.notebook_to_js(Pluto.Notebook([Pluto.Cell("1")]));
  3.039852 seconds (6.00 M allocations: 338.446 MiB, 5.37% gc time, 91.06% compilation time)


"""
For each connected client, we keep a copy of their current state. This way we know exactly which updates to send when the server-side state changes.
Expand Down
3 changes: 2 additions & 1 deletion src/webserver/MsgPack.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,4 +85,5 @@ end

function unpack(args...)
MsgPack.unpack(args...) |> decode_extension_and_addbits
end
end
precompile(unpack, (Vector{UInt8},))
Copy link
Collaborator

@rikhuijzer rikhuijzer Mar 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
precompile(unpack, (Vector{UInt8},))

Minor effect

Without precompile

$ julia --project -ie 'using Pluto'
...

julia> @time @eval Pluto.MsgPack.unpack([0x00]);
  0.511228 seconds (1.27 M allocations: 67.586 MiB, 24.02% gc time, 99.97% compilation time)

With precompile

julia> @time @eval Pluto.MsgPack.unpack([0x00]);
  0.454856 seconds (1.24 M allocations: 65.688 MiB, 15.59% gc time, 99.98% compilation time)

2 changes: 2 additions & 0 deletions src/webserver/WebServer.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ function run(; kwargs...)
options = Configuration.from_flat_kwargs(; kwargs...)
run(options)
end
precompile(run, ())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one works. Although I cannot measure Pluto.run() easily, I can measure Configuration.from_flat_kwargs which is the biggest source of latency according to SnoopCompile

Without precompile(run, ())

$ julia --project -ie 'using Pluto'
...

julia> @time @eval Pluto.Configuration.from_flat_kwargs();
  2.158083 seconds (3.60 M allocations: 194.337 MiB, 7.19% gc time, 99.96% compilation time)

With precompile(run, ())

julia> @time @eval Pluto.Configuration.from_flat_kwargs();
  1.431652 seconds (2.64 M allocations: 143.595 MiB, 11.35% gc time, 99.92% compilation time)


function run(options::Configuration.Options)
session = ServerSession(; options)
Expand Down Expand Up @@ -323,6 +324,7 @@ function run(session::ServerSession, pluto_router)
end
end
end
precompile(run, (ServerSession, HTTP.Handlers.Router{Symbol("##001")}))

get_favorite_notebook(notebook:: Nothing) = nothing
get_favorite_notebook(notebook:: String) = notebook
Expand Down