store noinline declaration in method source
#41922
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
You also need to change the type of the value from Bool to UInt8 in jltypes.c |
|
|
This will help us quickly query if a method is declared as `@noinline` or not.
---
Not strictly related to this PR:
I don't like the remaining asymmetry between `@inline` and `@noinline`
i.e. we store `Expr(:meta, :noinline)` in statements while discarding
`@inline` information, so I tried to store `Expr(:meta, :inline)` also
(and implement the symmetric version of `is_declared_noinline`), but it
turned out that it slows down bootstrapping a bit and even worse, it
causes suspicious errors while sysimg creation. Maybe the later implies
that some of `@inline` declared methods body hit an existing error ?
MRE: with the following diff:
```diff
diff --git a/src/method.c b/src/method.c
index 0ff265cc87..a883c68e79 100644
--- a/src/method.c
+++ b/src/method.c
@@ -283,14 +283,14 @@ static void jl_code_info_set_ir(jl_co
jl_value_t *ma = jl_array_ptr_ref(meta, k)
if (ma == (jl_value_t*)pure_sym)
li->pure = 1;
- else if (ma == (jl_value_t*)inline_sym)
- li->inlineable = 1;
else if (ma == (jl_value_t*)propagate_inbo
li->propagate_inbounds = 1;
else if (ma == (jl_value_t*)aggressive_con
li->aggressive_constprop = 1;
else {
- if (ma == (jl_value_t*)noinline_sym)
+ if (ma == (jl_value_t*)inline_sym)
+ li->inlineable = 1;
+ else if (ma == (jl_value_t*)noinline_s
li->inlineable = 2;
jl_array_ptr_set(meta, ins++, ma);
}
```
`make` will yield something like this at sysimg creation:
```
...
jl_bounds_error_ints at /home/aviatesk/julia/src/rtutils.c:194
getindex at ./array.jl:895 [inlined]
find_curblock at ./compiler/ssair/passes.jl:58
compute_value_for_block at ./compiler/ssair/passes.jl:75
getfield_elim_pass! at ./compiler/ssair/passes.jl:859
run_passes at ./compiler/optimize.jl:303
optimize at ./compiler/optimize.jl:288 [inlined]
...
```
aviatesk
added a commit
that referenced
this pull request
Aug 19, 2021
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
## Understand `@nospecialize`
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable
to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can
cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into
inference in anyway. So there is still a possibility of "excess
inference"
when the compiler sees a considerable complexity of argument types
during
inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can
cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation
time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation
time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as
`@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer
effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
### 1. make `@nospecialize` avoid inference also
This is almost same as what Jameson has done at
<vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to
perform reasonably. For example, it's obvious that the following
definition of `getindex(@nospecialize(t::Tuple), i::Int)` would perform
very badly if `@nospecialize` blocks inference, because of a lack of
useful type information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
### 2. add noinfer effect when `@nospecialize`d method is annotated as
`@noinline` also
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when
`@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after
<#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
### 3. implement as `@pure`-like boolean annotation to request noinfer
effect on top of `@nospecialize`
So this is what this commit implements. It basically replaces the
previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to
`@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
# the compiler will always infer with `f::Any`
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or
not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained
way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
aviatesk
added a commit
that referenced
this pull request
Aug 19, 2021
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
## Understand `@nospecialize`
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
### 1. make `@nospecialize` avoid inference also
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
### 2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
### 3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
# the compiler will always infer with `f::Any`
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
aviatesk
added a commit
that referenced
this pull request
Aug 23, 2021
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
## Understand `@nospecialize`
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
### 1. make `@nospecialize` avoid inference also
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
### 2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
### 3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
# the compiler will always infer with `f::Any`
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
vtjnash
reviewed
Aug 24, 2021
Comment on lines
+120
to
+122
| return _any(source.code) do @nospecialize stmt | ||
| isexpr(stmt, :meta) && stmt.args[1] === :noinline | ||
| end |
There was a problem hiding this comment.
Having 2 ways to do the same thing is considered bad, so this either needs to be a flag or a meta statement. It is not both.
| """ | ||
| function is_inlineable(method::Method) | ||
| isdefined(method, :source) || return false | ||
| return ccall(:jl_ir_flag_inlineable, Bool, (Any,), method.source) |
There was a problem hiding this comment.
Perhaps make this jl_ir_flag_inlineable function return a UInt8 enum (or cost?) to DRY this pair of functions
vtjnash
added
the
compiler:optimizer
aviatesk
added a commit
that referenced
this pull request
Sep 20, 2021
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
aviatesk
added a commit
that referenced
this pull request
Sep 20, 2021
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
|
Superseded by #48250. |
aviatesk
added a commit
that referenced
this pull request
Apr 12, 2023
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
Apr 12, 2023
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
Apr 13, 2023
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
Apr 13, 2023
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
Apr 13, 2023
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
Apr 14, 2023
…ence
This commit introduces new compiler annotation named `@noinfer`, which
requests the compiler to avoid excess inference.
In order to discuss `@noinfer`, it would help a lot to understand the
behavior of `@nospecialize`.
Its docstring says simply:
> This is only a hint for the compiler to avoid excess code generation.
More specifically, it works by _suppressing dispatches_ with complex
runtime types of the annotated arguments. This could be understood with
the example below:
```julia
julia> function invokef(f, itr)
local r = 0
r += f(itr[1])
r += f(itr[2])
r += f(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inling
julia> f(a) = _isa(a, Function);
julia> g(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed invokef(f, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (f)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (f)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (f)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed invokef(g, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke f(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke f(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke f(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `f` remain to be `:call` expression (thus dispatched and
compiled at runtime) while the calls of `g` are resolved as `:invoke`
expressions. This is because `@nospecialize` requests the compiler to
give up compiling `g` with concrete argument types but with precisely
declared argument types, and in this way `invokef(g, dispatchonly)` will
avoid runtime dispatches and accompanying JIT compilations (i.e. "excess
code generation").
The problem here is, it influences dispatch only, does not intervene
into inference in anyway. So there is still a possibility of "excess
inference" when the compiler sees a considerable complexity of argument
types during inference:
```julia
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed invokef(f, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> @time @code_typed invokef(g, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
```
The purpose of this PR is basically to provide a more drastic way to
avoid excess compilation.
Here are some ideas to implement the functionality:
1. make `@nospecialize` avoid inference also
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline` also
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3. for now, because I think the
interface is ready to be experimented.
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
as `@noinline`.
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` also.
It's a reasonable choice, and could be implemented efficiently after <#41922>.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while _partially_ avoiding inference, e.g.:
```julia
@noinline function twof(@nospecialize(f), n) # we really want not to
inline this method body ?
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
So this is what this commit implements. It basically replaces the previous
`@noinline` flag with newly-introduced annotation named `@noinfer`. It's
still associated with `@nospecialize` and it only has effect when used
together with `@nospecialize`, but now it's not associated to `@noinline`
at least, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more reliably:
```julia
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach, if initial experiments show `@noinfer` is
useful.
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
Apr 14, 2023
…ence
This commit introduces a new compiler annotation called `@noinfer`,
which allows us to request the compiler to avoid excessive inference.
\## `@nospecialize` mechanism
T discuss `@noinfer`, let's first understand the behavior of
`@nospecialize`.
Its docstring says that
> This is only a hint for the compiler to avoid excess code generation.
, and it works by suppressing dispatches with complex runtime
occurrences of the annotated arguments. This could be understood with
the example below:
```julia
julia> function call_func_itr(func, itr)
local r = 0
r += func(itr[1])
r += func(itr[2])
r += func(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inlining
julia> func_specialize(a) = _isa(a, Function);
julia> func_nospecialize(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed call_func_itr(func_specialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (func)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (func)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (func)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed call_func_itr(func_nospecialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke func(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke func(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke func(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `func_specialize` remain to be `:call` expression (so that
they are dispatched and compiled at runtime) while the calls of
`func_nospecialize` are resolved as `:invoke` expressions. This is
because `@nospecialize` requests the compiler to give up compiling
`func_nospecialize` with runtime argument types but with the declared
argument types, allowing `call_func_itr(func_nospecialize, dispatchonly)`
to avoid runtime dispatches and accompanying JIT compilations
(i.e. "excess code generation").
The difference is evident when checking `specializations`:
```julia
julia> call_func_itr(func_specialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_specialize))))
3 # w/ runtime dispatch, multiple specializations
julia> call_func_itr(func_nospecialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_nospecialize))))
1 # w/o runtime dispatch, the single specialization
```
The problem here is that it influences dispatch only, and does not
intervene into inference in anyway. So there is still a possibility of
"excess inference" when the compiler sees a considerable complexity of
argument types during inference:
```julia
julia> func_specialize(a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_specialize)))) == 0;
julia> func_nospecialize(@nospecialize a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_nospecialize)))) == 0;
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed call_func_itr(func_specialize, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> length(Base.specializations(only(methods(func_specialize))))
4 # multiple method instances inferred
julia> @time @code_typed call_func_itr(func_nospecialize, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
julia> length(Base.specializations(only(methods(func_nospecialize))))
4 # multiple method instances inferred
```
The purpose of this PR is to implement a mechanism that allows us to
avoid excessive inference to reduce the compilation latency when
inference sees a considerable complexity of argument types.
\## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` block inference
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline`
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3.
\### 1. make `@nospecialize` block inference
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
\### 2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline`
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
with `@noinline` (i.e. when adding `@noinline`, we're usually fine with
disabling inter-procedural optimizations other than resolving dispatch).
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` too.
It's a reasonable choice and can be efficiently implemented with #41922.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while partly avoiding inference, e.g.:
```julia
\# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # this method body is very simple and should be eligible for inlining
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
This is what this commit implements. It basically replaces the previous
`@noinline` flag with a newly-introduced annotation named `@noinfer`.
It is still associated with `@nospecialize` and it only has effect when
used together with `@nospecialize`, but now it is not associated to
`@noinline`, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more safely:
```julia
\# the compiler will always infer with `f::Any`
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach but at the same time I'm afraid to have too
many annotations that are related to some sort (I expect we will
annotate both `@noinfer` and `@nospecialize` in this scheme).
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
May 9, 2023
…ence
This commit introduces a new compiler annotation called `@noinfer`,
which allows us to request the compiler to avoid excessive inference.
\## `@nospecialize` mechanism
T discuss `@noinfer`, let's first understand the behavior of
`@nospecialize`.
Its docstring says that
> This is only a hint for the compiler to avoid excess code generation.
, and it works by suppressing dispatches with complex runtime
occurrences of the annotated arguments. This could be understood with
the example below:
```julia
julia> function call_func_itr(func, itr)
local r = 0
r += func(itr[1])
r += func(itr[2])
r += func(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inlining
julia> func_specialize(a) = _isa(a, Function);
julia> func_nospecialize(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed call_func_itr(func_specialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (func)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (func)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (func)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed call_func_itr(func_nospecialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke func(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke func(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke func(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `func_specialize` remain to be `:call` expression (so that
they are dispatched and compiled at runtime) while the calls of
`func_nospecialize` are resolved as `:invoke` expressions. This is
because `@nospecialize` requests the compiler to give up compiling
`func_nospecialize` with runtime argument types but with the declared
argument types, allowing `call_func_itr(func_nospecialize, dispatchonly)`
to avoid runtime dispatches and accompanying JIT compilations
(i.e. "excess code generation").
The difference is evident when checking `specializations`:
```julia
julia> call_func_itr(func_specialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_specialize))))
3 # w/ runtime dispatch, multiple specializations
julia> call_func_itr(func_nospecialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_nospecialize))))
1 # w/o runtime dispatch, the single specialization
```
The problem here is that it influences dispatch only, and does not
intervene into inference in anyway. So there is still a possibility of
"excess inference" when the compiler sees a considerable complexity of
argument types during inference:
```julia
julia> func_specialize(a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_specialize)))) == 0;
julia> func_nospecialize(@nospecialize a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_nospecialize)))) == 0;
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed call_func_itr(func_specialize, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> length(Base.specializations(only(methods(func_specialize))))
4 # multiple method instances inferred
julia> @time @code_typed call_func_itr(func_nospecialize, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
julia> length(Base.specializations(only(methods(func_nospecialize))))
4 # multiple method instances inferred
```
The purpose of this PR is to implement a mechanism that allows us to
avoid excessive inference to reduce the compilation latency when
inference sees a considerable complexity of argument types.
\## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` block inference
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline`
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3.
\### 1. make `@nospecialize` block inference
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
\### 2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline`
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
with `@noinline` (i.e. when adding `@noinline`, we're usually fine with
disabling inter-procedural optimizations other than resolving dispatch).
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` too.
It's a reasonable choice and can be efficiently implemented with #41922.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while partly avoiding inference, e.g.:
```julia
\# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # this method body is very simple and should be eligible for inlining
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
This is what this commit implements. It basically replaces the previous
`@noinline` flag with a newly-introduced annotation named `@noinfer`.
It is still associated with `@nospecialize` and it only has effect when
used together with `@nospecialize`, but now it is not associated to
`@noinline`, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more safely:
```julia
\# the compiler will always infer with `f::Any`
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach but at the same time I'm afraid to have too
many annotations that are related to some sort (I expect we will
annotate both `@noinfer` and `@nospecialize` in this scheme).
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
May 9, 2023
…ence
This commit introduces a new compiler annotation called `@noinfer`,
which allows us to request the compiler to avoid excessive inference.
\## `@nospecialize` mechanism
T discuss `@noinfer`, let's first understand the behavior of
`@nospecialize`.
Its docstring says that
> This is only a hint for the compiler to avoid excess code generation.
, and it works by suppressing dispatches with complex runtime
occurrences of the annotated arguments. This could be understood with
the example below:
```julia
julia> function call_func_itr(func, itr)
local r = 0
r += func(itr[1])
r += func(itr[2])
r += func(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inlining
julia> func_specialize(a) = _isa(a, Function);
julia> func_nospecialize(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed call_func_itr(func_specialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (func)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (func)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (func)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed call_func_itr(func_nospecialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke func(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke func(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke func(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `func_specialize` remain to be `:call` expression (so that
they are dispatched and compiled at runtime) while the calls of
`func_nospecialize` are resolved as `:invoke` expressions. This is
because `@nospecialize` requests the compiler to give up compiling
`func_nospecialize` with runtime argument types but with the declared
argument types, allowing `call_func_itr(func_nospecialize, dispatchonly)`
to avoid runtime dispatches and accompanying JIT compilations
(i.e. "excess code generation").
The difference is evident when checking `specializations`:
```julia
julia> call_func_itr(func_specialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_specialize))))
3 # w/ runtime dispatch, multiple specializations
julia> call_func_itr(func_nospecialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_nospecialize))))
1 # w/o runtime dispatch, the single specialization
```
The problem here is that it influences dispatch only, and does not
intervene into inference in anyway. So there is still a possibility of
"excess inference" when the compiler sees a considerable complexity of
argument types during inference:
```julia
julia> func_specialize(a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_specialize)))) == 0;
julia> func_nospecialize(@nospecialize a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_nospecialize)))) == 0;
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed call_func_itr(func_specialize, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> length(Base.specializations(only(methods(func_specialize))))
4 # multiple method instances inferred
julia> @time @code_typed call_func_itr(func_nospecialize, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
julia> length(Base.specializations(only(methods(func_nospecialize))))
4 # multiple method instances inferred
```
The purpose of this PR is to implement a mechanism that allows us to
avoid excessive inference to reduce the compilation latency when
inference sees a considerable complexity of argument types.
\## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` block inference
2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline`
3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3.
\### 1. make `@nospecialize` block inference
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
\### 2. add noinfer effect when `@nospecialize`d method is annotated as `@noinline`
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
with `@noinline` (i.e. when adding `@noinline`, we're usually fine with
disabling inter-procedural optimizations other than resolving dispatch).
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` too.
It's a reasonable choice and can be efficiently implemented with #41922.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while partly avoiding inference, e.g.:
```julia
\# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # this method body is very simple and should be eligible for inlining
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 3. implement as `@pure`-like boolean annotation to request noinfer effect on top of `@nospecialize`
This is what this commit implements. It basically replaces the previous
`@noinline` flag with a newly-introduced annotation named `@noinfer`.
It is still associated with `@nospecialize` and it only has effect when
used together with `@nospecialize`, but now it is not associated to
`@noinline`, and it would help us reason about the behavior of `@noinfer`
and experiment its effect more safely:
```julia
\# the compiler will always infer with `f::Any`
Base.@noinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@noinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@noinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach but at the same time I'm afraid to have too
many annotations that are related to some sort (I expect we will
annotate both `@noinfer` and `@nospecialize` in this scheme).
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
May 10, 2023
…cess inference
This commit introduces a new compiler annotation called `@nospecializeinfer`,
which allows us to request the compiler to avoid excessive inference.
\## `@nospecialize` mechanism
T discuss `@nospecializeinfer`, let's first understand the behavior of
`@nospecialize`.
Its docstring says that
> This is only a hint for the compiler to avoid excess code generation.
, and it works by suppressing dispatches with complex runtime
occurrences of the annotated arguments. This could be understood with
the example below:
```julia
julia> function call_func_itr(func, itr)
local r = 0
r += func(itr[1])
r += func(itr[2])
r += func(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inlining
julia> func_specialize(a) = _isa(a, Function);
julia> func_nospecialize(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed call_func_itr(func_specialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (func)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (func)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (func)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed call_func_itr(func_nospecialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke func(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke func(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke func(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `func_specialize` remain to be `:call` expression (so that
they are dispatched and compiled at runtime) while the calls of
`func_nospecialize` are resolved as `:invoke` expressions. This is
because `@nospecialize` requests the compiler to give up compiling
`func_nospecialize` with runtime argument types but with the declared
argument types, allowing `call_func_itr(func_nospecialize, dispatchonly)`
to avoid runtime dispatches and accompanying JIT compilations
(i.e. "excess code generation").
The difference is evident when checking `specializations`:
```julia
julia> call_func_itr(func_specialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_specialize))))
3 # w/ runtime dispatch, multiple specializations
julia> call_func_itr(func_nospecialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_nospecialize))))
1 # w/o runtime dispatch, the single specialization
```
The problem here is that it influences dispatch only, and does not
intervene into inference in anyway. So there is still a possibility of
"excess inference" when the compiler sees a considerable complexity of
argument types during inference:
```julia
julia> func_specialize(a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_specialize)))) == 0;
julia> func_nospecialize(@nospecialize a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_nospecialize)))) == 0;
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed call_func_itr(func_specialize, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> length(Base.specializations(only(methods(func_specialize))))
4 # multiple method instances inferred
julia> @time @code_typed call_func_itr(func_nospecialize, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
julia> length(Base.specializations(only(methods(func_nospecialize))))
4 # multiple method instances inferred
```
The purpose of this PR is to implement a mechanism that allows us to
avoid excessive inference to reduce the compilation latency when
inference sees a considerable complexity of argument types.
\## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` block inference
2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3.
\### 1. make `@nospecialize` block inference
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
\### 2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
with `@noinline` (i.e. when adding `@noinline`, we're usually fine with
disabling inter-procedural optimizations other than resolving dispatch).
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` too.
It's a reasonable choice and can be efficiently implemented with #41922.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while partly avoiding inference, e.g.:
```julia
\# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # this method body is very simple and should be eligible for inlining
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
This is what this commit implements. It basically replaces the previous
`@noinline` flag with a newly-introduced annotation named `@nospecializeinfer`.
It is still associated with `@nospecialize` and it only has effect when
used together with `@nospecialize`, but now it is not associated to
`@noinline`, and it would help us reason about the behavior of `@nospecializeinfer`
and experiment its effect more safely:
```julia
\# the compiler will always infer with `f::Any`
Base.@nospecializeinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@nospecializeinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@nospecializeinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach but at the same time I'm afraid to have too
many annotations that are related to some sort (I expect we will
annotate both `@nospecializeinfer` and `@nospecialize` in this scheme).
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
May 11, 2023
…cess inference
This commit introduces a new compiler annotation called `@nospecializeinfer`,
which allows us to request the compiler to avoid excessive inference.
\## `@nospecialize` mechanism
T discuss `@nospecializeinfer`, let's first understand the behavior of
`@nospecialize`.
Its docstring says that
> This is only a hint for the compiler to avoid excess code generation.
, and it works by suppressing dispatches with complex runtime
occurrences of the annotated arguments. This could be understood with
the example below:
```julia
julia> function call_func_itr(func, itr)
local r = 0
r += func(itr[1])
r += func(itr[2])
r += func(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inlining
julia> func_specialize(a) = _isa(a, Function);
julia> func_nospecialize(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed call_func_itr(func_specialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (func)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (func)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (func)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed call_func_itr(func_nospecialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke func(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke func(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke func(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `func_specialize` remain to be `:call` expression (so that
they are dispatched and compiled at runtime) while the calls of
`func_nospecialize` are resolved as `:invoke` expressions. This is
because `@nospecialize` requests the compiler to give up compiling
`func_nospecialize` with runtime argument types but with the declared
argument types, allowing `call_func_itr(func_nospecialize, dispatchonly)`
to avoid runtime dispatches and accompanying JIT compilations
(i.e. "excess code generation").
The difference is evident when checking `specializations`:
```julia
julia> call_func_itr(func_specialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_specialize))))
3 # w/ runtime dispatch, multiple specializations
julia> call_func_itr(func_nospecialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_nospecialize))))
1 # w/o runtime dispatch, the single specialization
```
The problem here is that it influences dispatch only, and does not
intervene into inference in anyway. So there is still a possibility of
"excess inference" when the compiler sees a considerable complexity of
argument types during inference:
```julia
julia> func_specialize(a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_specialize)))) == 0;
julia> func_nospecialize(@nospecialize a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_nospecialize)))) == 0;
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed call_func_itr(func_specialize, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> length(Base.specializations(only(methods(func_specialize))))
4 # multiple method instances inferred
julia> @time @code_typed call_func_itr(func_nospecialize, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
julia> length(Base.specializations(only(methods(func_nospecialize))))
4 # multiple method instances inferred
```
The purpose of this PR is to implement a mechanism that allows us to
avoid excessive inference to reduce the compilation latency when
inference sees a considerable complexity of argument types.
\## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` block inference
2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3.
\### 1. make `@nospecialize` block inference
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
\### 2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
with `@noinline` (i.e. when adding `@noinline`, we're usually fine with
disabling inter-procedural optimizations other than resolving dispatch).
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` too.
It's a reasonable choice and can be efficiently implemented with #41922.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while partly avoiding inference, e.g.:
```julia
\# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # this method body is very simple and should be eligible for inlining
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
This is what this commit implements. It basically replaces the previous
`@noinline` flag with a newly-introduced annotation named `@nospecializeinfer`.
It is still associated with `@nospecialize` and it only has effect when
used together with `@nospecialize`, but now it is not associated to
`@noinline`, and it would help us reason about the behavior of `@nospecializeinfer`
and experiment its effect more safely:
```julia
\# the compiler will always infer with `f::Any`
Base.@nospecializeinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@nospecializeinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@nospecializeinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach but at the same time I'm afraid to have too
many annotations that are related to some sort (I expect we will
annotate both `@nospecializeinfer` and `@nospecialize` in this scheme).
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
May 12, 2023
…cess inference
This commit introduces a new compiler annotation called `@nospecializeinfer`,
which allows us to request the compiler to avoid excessive inference.
\## `@nospecialize` mechanism
T discuss `@nospecializeinfer`, let's first understand the behavior of
`@nospecialize`.
Its docstring says that
> This is only a hint for the compiler to avoid excess code generation.
, and it works by suppressing dispatches with complex runtime
occurrences of the annotated arguments. This could be understood with
the example below:
```julia
julia> function call_func_itr(func, itr)
local r = 0
r += func(itr[1])
r += func(itr[2])
r += func(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inlining
julia> func_specialize(a) = _isa(a, Function);
julia> func_nospecialize(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed call_func_itr(func_specialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (func)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (func)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (func)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed call_func_itr(func_nospecialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke func(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke func(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke func(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `func_specialize` remain to be `:call` expression (so that
they are dispatched and compiled at runtime) while the calls of
`func_nospecialize` are resolved as `:invoke` expressions. This is
because `@nospecialize` requests the compiler to give up compiling
`func_nospecialize` with runtime argument types but with the declared
argument types, allowing `call_func_itr(func_nospecialize, dispatchonly)`
to avoid runtime dispatches and accompanying JIT compilations
(i.e. "excess code generation").
The difference is evident when checking `specializations`:
```julia
julia> call_func_itr(func_specialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_specialize))))
3 # w/ runtime dispatch, multiple specializations
julia> call_func_itr(func_nospecialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_nospecialize))))
1 # w/o runtime dispatch, the single specialization
```
The problem here is that it influences dispatch only, and does not
intervene into inference in anyway. So there is still a possibility of
"excess inference" when the compiler sees a considerable complexity of
argument types during inference:
```julia
julia> func_specialize(a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_specialize)))) == 0;
julia> func_nospecialize(@nospecialize a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_nospecialize)))) == 0;
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed call_func_itr(func_specialize, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> length(Base.specializations(only(methods(func_specialize))))
4 # multiple method instances inferred
julia> @time @code_typed call_func_itr(func_nospecialize, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
julia> length(Base.specializations(only(methods(func_nospecialize))))
4 # multiple method instances inferred
```
The purpose of this PR is to implement a mechanism that allows us to
avoid excessive inference to reduce the compilation latency when
inference sees a considerable complexity of argument types.
\## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` block inference
2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3.
\### 1. make `@nospecialize` block inference
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
\### 2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
with `@noinline` (i.e. when adding `@noinline`, we're usually fine with
disabling inter-procedural optimizations other than resolving dispatch).
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` too.
It's a reasonable choice and can be efficiently implemented with #41922.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while partly avoiding inference, e.g.:
```julia
\# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # this method body is very simple and should be eligible for inlining
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
This is what this commit implements. It basically replaces the previous
`@noinline` flag with a newly-introduced annotation named `@nospecializeinfer`.
It is still associated with `@nospecialize` and it only has effect when
used together with `@nospecialize`, but now it is not associated to
`@noinline`, and it would help us reason about the behavior of `@nospecializeinfer`
and experiment its effect more safely:
```julia
\# the compiler will always infer with `f::Any`
Base.@nospecializeinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@nospecializeinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@nospecializeinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach but at the same time I'm afraid to have too
many annotations that are related to some sort (I expect we will
annotate both `@nospecializeinfer` and `@nospecialize` in this scheme).
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
aviatesk
added a commit
that referenced
this pull request
May 23, 2023
* introduce `@nospecializeinfer` macro to tell the compiler to avoid excess inference
This commit introduces a new compiler annotation called `@nospecializeinfer`,
which allows us to request the compiler to avoid excessive inference.
\## `@nospecialize` mechanism
T discuss `@nospecializeinfer`, let's first understand the behavior of
`@nospecialize`.
Its docstring says that
> This is only a hint for the compiler to avoid excess code generation.
, and it works by suppressing dispatches with complex runtime
occurrences of the annotated arguments. This could be understood with
the example below:
```julia
julia> function call_func_itr(func, itr)
local r = 0
r += func(itr[1])
r += func(itr[2])
r += func(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inlining
julia> func_specialize(a) = _isa(a, Function);
julia> func_nospecialize(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed call_func_itr(func_specialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = (func)(%2)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = (func)(%5)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = (func)(%8)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
julia> @code_typed call_func_itr(func_nospecialize, dispatchonly)
CodeInfo(
1 ─ %1 = π (0, Int64)
│ %2 = Base.arrayref(true, itr, 1)::Any
│ %3 = invoke func(%2::Any)::Any
│ %4 = (%1 + %3)::Any
│ %5 = Base.arrayref(true, itr, 2)::Any
│ %6 = invoke func(%5::Any)::Any
│ %7 = (%4 + %6)::Any
│ %8 = Base.arrayref(true, itr, 3)::Any
│ %9 = invoke func(%8::Any)::Any
│ %10 = (%7 + %9)::Any
└── return %10
) => Any
```
The calls of `func_specialize` remain to be `:call` expression (so that
they are dispatched and compiled at runtime) while the calls of
`func_nospecialize` are resolved as `:invoke` expressions. This is
because `@nospecialize` requests the compiler to give up compiling
`func_nospecialize` with runtime argument types but with the declared
argument types, allowing `call_func_itr(func_nospecialize, dispatchonly)`
to avoid runtime dispatches and accompanying JIT compilations
(i.e. "excess code generation").
The difference is evident when checking `specializations`:
```julia
julia> call_func_itr(func_specialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_specialize))))
3 # w/ runtime dispatch, multiple specializations
julia> call_func_itr(func_nospecialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_nospecialize))))
1 # w/o runtime dispatch, the single specialization
```
The problem here is that it influences dispatch only, and does not
intervene into inference in anyway. So there is still a possibility of
"excess inference" when the compiler sees a considerable complexity of
argument types during inference:
```julia
julia> func_specialize(a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_specialize)))) == 0;
julia> func_nospecialize(@nospecialize a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_nospecialize)))) == 0;
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed call_func_itr(func_specialize, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> length(Base.specializations(only(methods(func_specialize))))
4 # multiple method instances inferred
julia> @time @code_typed call_func_itr(func_nospecialize, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
julia> length(Base.specializations(only(methods(func_nospecialize))))
4 # multiple method instances inferred
```
The purpose of this PR is to implement a mechanism that allows us to
avoid excessive inference to reduce the compilation latency when
inference sees a considerable complexity of argument types.
\## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` block inference
2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3.
\### 1. make `@nospecialize` block inference
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
\### 2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
with `@noinline` (i.e. when adding `@noinline`, we're usually fine with
disabling inter-procedural optimizations other than resolving dispatch).
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` too.
It's a reasonable choice and can be efficiently implemented with #41922.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while partly avoiding inference, e.g.:
```julia
\# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # this method body is very simple and should be eligible for inlining
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
This is what this commit implements. It basically replaces the previous
`@noinline` flag with a newly-introduced annotation named `@nospecializeinfer`.
It is still associated with `@nospecialize` and it only has effect when
used together with `@nospecialize`, but now it is not associated to
`@noinline`, and it would help us reason about the behavior of `@nospecializeinfer`
and experiment its effect more safely:
```julia
\# the compiler will always infer with `f::Any`
Base.@nospecializeinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@nospecializeinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@nospecializeinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach but at the same time I'm afraid to have too
many annotations that are related to some sort (I expect we will
annotate both `@nospecializeinfer` and `@nospecialize` in this scheme).
---
experiment `@nospecializeinfer` on `Core.Compiler`
This commit adds `@nospecializeinfer` macro on various `Core.Compiler`
functions and achieves the following sysimage size reduction:
| | this commit | master | % |
| --------------------------------- | ----------- | ----------- | ------- |
| `Core.Compiler` compilation (sec) | `66.4551` | `71.0846` | `0.935` |
| `corecompiler.jl` (KB) | `17638080` | `18407248` | `0.958` |
| `sys.jl` (KB) | `88736432` | `89361280` | `0.993` |
| `sys-o.a` (KB) | `189484400` | `189907096` | `0.998` |
---------
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This will help us quickly query if a method is declared as
@noinlineor not.Not strictly related to this PR:
I don't like the remaining asymmetry between
@inlineand@noinlinei.e. we store
Expr(:meta, :noinline)in statements while discarding@inlineinformation, so I tried to storeExpr(:meta, :inline)also(and implement the symmetric version of
is_declared_noinline), but itturned out that it slows down bootstrapping a bit and even worse, it
causes suspicious errors while sysimg creation. Maybe the later implies
that some of
@inlinedeclared methods body hit an existing error ?MRE: with the following diff:
makewill yield something like this at sysimg creation: