Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

effects mistake with refining inconsistent errors to always-throws #53613

Closed
KristofferC opened this issue Mar 5, 2024 · 4 comments · Fixed by #54219
Closed

effects mistake with refining inconsistent errors to always-throws #53613

KristofferC opened this issue Mar 5, 2024 · 4 comments · Fixed by #54219
Assignees
@aviatesk
Labels
backport 1.11 Change should be backported to release-1.11 bug Indicates an unexpected problem or unintended behavior compiler:effects effect analysis
Milestone
1.11

Comments

@KristofferC
Copy link
Sponsor Member 

(NeuralNetworkReachability) pkg> test
...
     Testing Running tests...
Test Summary:                      | Pass  Total  Time
Optional dependencies (not loaded) |    2      2  0.2s

[43958] signal 4 (1): Illegal instruction: 4
in expression starting at /Users/kristoffercarlsson/PkgEvalAnalysis/dev/NeuralNetworkReachability.jl/test/ForwardAlgorithms/forward.jl:169
Verisig at /Users/kristoffercarlsson/PkgEvalAnalysis/dev/NeuralNetworkReachability.jl/src/ForwardAlgorithms/Verisig.jl:26
macro expansion at /Users/kristoffercarlsson/PkgEvalAnalysis/dev/NeuralNetworkReachability.jl/test/ForwardAlgorithms/forward.jl:184 [inlined]
macro expansion at /Users/kristoffercarlsson/julia1.11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1700 [inlined]
top-level scope at /Users/kristoffercarlsson/PkgEvalAnalysis/dev/NeuralNetworkReachability.jl/test/ForwardAlgorithms/forward.jl:170
_jl_invoke at /Users/kristoffercarlsson/julia1.11/src/gf.c:2945
jl_toplevel_eval_flex at /Users/kristoffercarlsson/julia1.11/src/toplevel.c:934
jl_toplevel_eval_flex at /Users/kristoffercarlsson/julia1.11/src/toplevel.c:886
ijl_toplevel_eval at /Users/kristoffercarlsson/julia1.11/src/toplevel.c:952 [inlined]
ijl_toplevel_eval_in at /Users/kristoffercarlsson/julia1.11/src/toplevel.c:994
eval at ./boot.jl:428 [inlined]
include_string at ./loading.jl:2486
_jl_invoke at /Users/kristoffercarlsson/julia1.11/src/gf.c:2926
_include at ./loading.jl:2546
include at ./sysimg.jl:38
unknown function (ip: 0x119403739)
_jl_invoke at /Users/kristoffercarlsson/julia1.11/src/gf.c:2926
jl_apply at /Users/kristoffercarlsson/julia1.11/src/./julia.h:2165 [inlined]
do_call at /Users/kristoffercarlsson/julia1.11/src/interpreter.c:126
eval_stmt_value at /Users/kristoffercarlsson/julia1.11/src/interpreter.c:174
eval_body at /Users/kristoffercarlsson/julia1.11/src/interpreter.c:657
eval_body at /Users/kristoffercarlsson/julia1.11/src/interpreter.c:539
@KristofferC KristofferC added this to the 1.11 milestone Mar 5, 2024
@vtjnash
Copy link
Sponsor Member

vtjnash commented Mar 6, 2024

That is a pretty sick effects mistake you found there:

julia> @noinline f() = @assert isdefined(Main, :y)
f (generic function with 4 methods)

julia> g()
ERROR: AssertionError: isdefined(Main, :y)
Stacktrace:
 [1] f()
   @ Main ./REPL[17]:1
 [2] g()
   @ Main ./REPL[15]:1
 [3] top-level scope
   @ REPL[18]:1

julia> y = 2
2

julia> g()
Unreachable reached at 0x7f3e339a4c2b

[2387242] signal 4 (2): Illegal instruction
in expression starting at REPL[20]:1
g at ./REPL[15]:1

The +c effect is being used to refine this "returns nothing" to "always-throws" via observing that concrete eval may throw, even though that is wrong (consistency does not apply to error paths).

 • %1 = < concrete eval > f()::Union{} (+c,+e,!n,+t,+s,!m,+u)

@vtjnash vtjnash added bug Indicates an unexpected problem or unintended behavior compiler:effects effect analysis backport 1.10 Change should be backported to the 1.10 release backport 1.11 Change should be backported to release-1.11 and removed bisect wanted MWE wanted backport 1.10 Change should be backported to the 1.10 release labels Mar 6, 2024
@vtjnash vtjnash changed the title Testing NeuralNetworkReachability crashes with illegal instruction on 1.11. effects mistake with refining inconsistent errors to always-throws Mar 6, 2024
@Keno
Copy link
Member

Keno commented Mar 6, 2024

This is possibly #53518

@aviatesk
Copy link
Sponsor Member

aviatesk commented Mar 7, 2024

It looks like we're having a two-part problem. First one is that there's an issue with f()'s !:consistent getting refined, and then, once we fix that, there will be another problem of the caller refining !:consistent propagated from f(). We can fix the first issue by using an augmented post domtree for visit_conditional_successors. As for the second part, it seems we might need to break down :consistent into two separate effect-bits :consistent_return and :consistent_termination.

aviatesk added a commit that referenced this issue Mar 7, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
362a0e4 
- fixes #53613
aviatesk added a commit that referenced this issue Mar 7, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
b757ee6 
- fixes #53613
aviatesk added a commit that referenced this issue Mar 7, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
80b3d53 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 7, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
6c7cdec 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 7, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
2b87cca 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 7, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
6b0630e 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 7, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
988b53b 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 7, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
113a555 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
@aviatesk aviatesk mentioned this issue Mar 7, 2024
Merged
aviatesk added a commit that referenced this issue Mar 8, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
8c2209f 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 8, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
f1d4535 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 8, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
c336656 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 8, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
769f1ef 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 8, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
cd7508b 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`, while also enforcing the
augmented control flow graph (`construct_augmented_cfg`) to have a
single exit node really. Since the augmented post domtree is now
enforced to have a single return, we can keep using the current
`postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 8, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
e11b31c 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`, while also enforcing the
augmented control flow graph (`construct_augmented_cfg`) to have a
single exit node really. Since the augmented post domtree is now
enforced to have a single return, we can keep using the current
`postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
@vtjnash vtjnash mentioned this issue Mar 12, 2024
Closed
aviatesk added a commit that referenced this issue Mar 13, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
95c91ae 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`, while also enforcing the
augmented control flow graph (`construct_augmented_cfg`) to have a
single exit node really. Since the augmented post domtree is now
enforced to have a single return, we can keep using the current
`postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 14, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
9928c45 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`, while also enforcing the
augmented control flow graph (`construct_augmented_cfg`) to have a
single exit node really. Since the augmented post domtree is now
enforced to have a single return, we can keep using the current
`postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
@schillic
Copy link

I just extracted a small example and reported in Requires and now wanted to report it also here, but then found this issue. Anyway, for the record: JuliaPackaging/Requires.jl#118

@schillic schillic mentioned this issue Mar 14, 2024
Closed
@topolarity topolarity mentioned this issue Mar 14, 2024
Merged
aviatesk added a commit that referenced this issue Mar 16, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
d0b2d32 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`, while also enforcing the
augmented control flow graph (`construct_augmented_cfg`) to have a
single exit node really. Since the augmented post domtree is now
enforced to have a single return, we can keep using the current
`postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk pushed a commit that referenced this issue Mar 19, 2024
@topolarity
Fix handling of virtual exit node in PostDomTree (#53739)
2775c9a 
This is an alternative to #53642

The `dom_edges()` for an exit block in the CFG are empty when computing
the PostDomTree so the loop below this may not actually run. In that
case, the right semidominator is the ancestor from the DFSTree, which is
the "virtual" -1 block.

This resolves half of the issue in
#53613:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∈ visited
           @test 3 ∈ visited
       end
Test Passed
```

This needs some tests (esp. since I don't think we have any DomTree
tests at all right now), but otherwise should be good to go.
aviatesk added a commit that referenced this issue Mar 19, 2024
@aviatesk
post-opt: use augmented post-domtree for visit_conditional_successors
c8e7e11 
This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`, while also enforcing the
augmented control flow graph (`construct_augmented_cfg`) to have a
single exit node really. Since the augmented post domtree is now
enforced to have a single return, we can keep using the current
`postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 19, 2024
@aviatesk
post-opt: add more test cases for visit_conditional_successors
3fecda9 
This commit was initially crafted to fix an issue with post-opt analysis
that came to light in #53613, planning to solve it by
incorporating an augmented CFG directly into the post-opt analysis. But,
with the issue now sorted out by #53739, this commit
shifted focus to just adding test cases.

=== The original commit message ===

post-opt: use augmented post-domtree for `visit_conditional_successors`

This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`, while also enforcing the
augmented control flow graph (`construct_augmented_cfg`) to have a
single exit node really. Since the augmented post domtree is now
enforced to have a single return, we can keep using the current
`postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too (#53613 (comment)).
aviatesk added a commit that referenced this issue Mar 19, 2024
@aviatesk
post-opt: add more test cases for visit_conditional_successors (#53642
9df47f2 
)

This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too
(#53613 (comment)).
aviatesk pushed a commit that referenced this issue Apr 23, 2024
@topolarity @aviatesk
Fix handling of virtual exit node in PostDomTree (#53739)
eb9c9b8 
This is an alternative to #53642

The `dom_edges()` for an exit block in the CFG are empty when computing
the PostDomTree so the loop below this may not actually run. In that
case, the right semidominator is the ancestor from the DFSTree, which is
the "virtual" -1 block.

This resolves half of the issue in
#53613:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∈ visited
           @test 3 ∈ visited
       end
Test Passed
```

This needs some tests (esp. since I don't think we have any DomTree
tests at all right now), but otherwise should be good to go.
aviatesk added a commit that referenced this issue Apr 23, 2024
@aviatesk
post-opt: add more test cases for visit_conditional_successors (#53642
678fe46 
)

This commit fixes the first problem that was found while digging into
#53613. It turns out that the post-domtree constructed
from regular `IRCode` doesn't work for visiting conditional successors
for post-opt analysis in cases like:
```julia
julia> let code = Any[
               # block 1
               GotoIfNot(Argument(2), 3),
               # block 2
               ReturnNode(Argument(3)),
               # block 3 (we should visit this block)
               Expr(:call, throw, "potential throw"),
               ReturnNode(), # unreachable
           ]
           ir = make_ircode(code; slottypes=Any[Any,Bool,Bool])
           visited = BitSet()
           @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int
               push!(visited, succ)
               return false
           end
           @test 2 ∉ visited
           @test 3 ∈ visited
       end
Test Failed at REPL[14]:16
  Expression: 2 ∉ visited
   Evaluated: 2 ∉ BitSet([2])
```

This might mean that we need to fix on the `postdominates` end, but for
now, this commit tries to get around it by using the augmented post
domtree in `visit_conditional_successors`. Since the augmented post
domtree is enforced to have a single return, we can keep using the
current `postdominates` to fix the issue.

However, this commit isn't enough to fix the NeuralNetworkReachability
segfault as reported in #53613, and we need to tackle the second issue
reported there too
(#53613 (comment)).
aviatesk added a commit that referenced this issue Apr 23, 2024
@aviatesk
post-opt: taint :consistent when statement may raise inconsistently
37393e8 
When an inconsistent statement doesn’t affect the return value, post-opt
analysis will try to refine it to `:consistent`. However this refinement
is invalid if the statement could throw, as `:consistent` requires
consistent termination.

For the time being, this commit implements the most conservative fix.
There might be a need to analyze `:nothrow` in a data-flow sensitive
way in the post-opt analysis as like we do for `:consistent`.

- closes #53613
@aviatesk aviatesk mentioned this issue Apr 23, 2024
Merged
Keno pushed a commit that referenced this issue Apr 24, 2024
@aviatesk
post-opt: taint :consistent when statement may raise inconsistently (#…
96866cb 
…54219)

When an inconsistent statement doesn’t affect the return value, post-opt
analysis will try to refine it to `:consistent`. However this refinement
is invalid if the statement could throw, as `:consistent` requires
consistent termination.

For the time being, this commit implements the most conservative fix.
There might be a need to analyze `:nothrow` in a data-flow sensitive way
in the post-opt analysis as like we do for `:consistent`.

- closes #53613
aviatesk added a commit that referenced this issue Apr 25, 2024
@aviatesk
post-opt: taint :consistent when statement may raise inconsistently (#…
6522986 
…54219)

When an inconsistent statement doesn’t affect the return value, post-opt
analysis will try to refine it to `:consistent`. However this refinement
is invalid if the statement could throw, as `:consistent` requires
consistent termination.

For the time being, this commit implements the most conservative fix.
There might be a need to analyze `:nothrow` in a data-flow sensitive way
in the post-opt analysis as like we do for `:consistent`.

- closes #53613
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aviatesk aviatesk

Labels
backport 1.11 Change should be backported to release-1.11 bug Indicates an unexpected problem or unintended behavior compiler:effects effect analysis
Projects
None yet
Milestone
1.11
Development

Successfully merging a pull request may close this issue.

post-opt: taint :consistent when statement may raise inconsistently JuliaLang/julia
5 participants
@vtjnash @KristofferC @Keno @schillic @aviatesk