Add Cthulhu.ascend integration (#648)

Closes #511 Also adds `reportkey` --------- Co-authored-by: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com>
aviatesk · Aug 17, 2024 · 0ec6218 · 0ec6218 · timholy · Aug 17, 2024
1 parent 8fe3d8d
commit 0ec6218
Show file tree

Hide file tree

Showing 9 changed files with 195 additions and 41 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,6 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 <!-- links start -->
+[0.9.8]: https://github.com/aviatesk/JET.jl/compare/v0.9.7...v0.9.8
 [0.9.6]: https://github.com/aviatesk/JET.jl/compare/v0.9.5...v0.9.6
 [0.9.5]: https://github.com/aviatesk/JET.jl/compare/v0.9.4...v0.9.5
 [0.9.4]: https://github.com/aviatesk/JET.jl/compare/v0.9.3...v0.9.4
@@ -25,6 +26,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 [0.8.0]: https://github.com/aviatesk/JET.jl/compare/v0.7.15...v0.8.0
 <!-- links end -->
 
+## [0.9.8]
+### Added
+- An extension that integrates `@report_opt` with Cthulhu (aviatesk/JET.jl#648)
+- `reportkey` for trimming multiple reports that resolve to the same runtime-dispatch caller/callee pair (aviatesk/JET.jl#648)
+
 ## [0.9.6]
 ### Fixed
 - `report_opt` no longer raises reports from callees on `throw` code path when the

diff --git a/Project.toml b/Project.toml
@@ -1,7 +1,7 @@
 name = "JET"
 uuid = "c3a54625-cd67-489e-a8e7-0a5a0ff4e31b"
 authors = ["Shuhei Kadowaki <aviatesk@gmail.com>"]
-version = "0.9.7"
+version = "0.9.8"
 
 [deps]
 CodeTracking = "da1fd8a2-8d9e-5ec2-8556-3022fb5608a2"
@@ -15,15 +15,18 @@ Preferences = "21216c6a-2e73-6563-6e65-726566657250"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
 
 [weakdeps]
+Cthulhu = "f68482b8-f384-11e8-15f7-abe071a5a75f"
 Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
 
 [extensions]
+JETCthulhuExt = "Cthulhu"
 ReviseExt = "Revise"
 
 [compat]
 Aqua = "0.8.2"
 BenchmarkTools = "1.3.2"
 CodeTracking = "1.3.1"
+Cthulhu = "2.14.0"
 Example = "0.5.3"
 InteractiveUtils = "1.10"
 JuliaInterpreter = "0.9"
@@ -43,6 +46,7 @@ julia = "1.10"
 [extras]
 Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
 BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
+Cthulhu = "f68482b8-f384-11e8-15f7-abe071a5a75f"
 Example = "7876af07-990d-54b4-ab0e-23690620f79a"
 Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
 Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
@@ -52,4 +56,4 @@ StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
 
 [targets]
-test = ["Aqua", "BenchmarkTools", "Example", "Libdl", "Logging", "Random", "Revise", "StaticArrays", "Test"]
+test = ["Aqua", "BenchmarkTools", "Cthulhu", "Example", "Libdl", "Logging", "Random", "Revise", "StaticArrays", "Test"]
diff --git a/docs/src/optanalysis.md b/docs/src/optanalysis.md
@@ -27,7 +27,7 @@ JET implements such an analyzer that investigates the optimized representation o
 anywhere the compiler failed in optimization. Especially, it can find where Julia creates captured variables, where
 runtime dispatch will happen, and where Julia gives up the optimization work due to unresolvable recursive function call.
 
-[SnoopCompile also detects inference failures](https://timholy.github.io/SnoopCompile.jl/stable/snoopi_deep_analysis/), but JET and SnoopCompile use different mechanisms: JET performs *static* analysis of a particular call, while SnoopCompile performs *dynamic* analysis of new inference. As a consequence, JET's detection of inference failures is reproducible (you can run the same analysis repeatedly and get the same result) but terminates at any non-inferable node of the call graph: you will miss runtime dispatch in any non-inferable callees. Conversely, SnoopCompile's detection of inference failures can explore the entire callgraph, but only for those portions that have not been previously inferred, and the analysis cannot be repeated in the same session.
+[SnoopCompile also detects inference failures](https://timholy.github.io/SnoopCompile.jl/stable/tutorials/snoop_inference_analysis/), but JET and SnoopCompile use different mechanisms: JET performs *static* analysis of a particular call, while SnoopCompile performs *dynamic* analysis of new inference. As a consequence, JET's detection of inference failures is reproducible (you can run the same analysis repeatedly and get the same result) but terminates at any non-inferable node of the call graph: you will miss runtime dispatch in any non-inferable callees. Conversely, SnoopCompile's detection of inference failures can explore the entire callgraph, but only for those portions that have not been previously inferred, and the analysis cannot be repeated in the same session.
 
 ## [Quick Start](@id optanalysis-quick-start)
 
@@ -164,6 +164,59 @@ using Test
 end
 ```
 
+## [Integration with Cthulhu](@id cthulhu-integration)
+
+If you identify inference problems, you may want to fix them. Cthulhu can be a useful tool for gaining more insight, and JET integrates nicely with Cthulhu.
+
+To exploit Cthulhu, you first need to split the overall report into individual inference failures:
+
+```@repl quickstart
+report = @report_opt sumup(sin);
+rpts = JET.get_reports(report)
+```
+
+!!! tip
+    If `rpts` is a long list, consider using `urpts = unique(reportkey, rpts)` to trim it.
+    See [`reportkey`](@ref).
+
+Now you can `ascend` individual reports:
+
+```
+julia> using Cthulhu
+
+julia> ascend(rpts[1])
+Choose a call for analysis (q to quit):
+     runtime dispatch to make_vals(%1::Any)::Any
+ >     sumup(::typeof(sin))
+
+Open an editor at a possible caller of
+  Tuple{typeof(make_vals), Any}
+or browse typed code:
+ > "REPL[7]", sumup: lines [4]
+   Browse typed code
+```
+
+`ascend` will show the full call-chain to reach a particular runtime dispatch; in this case, it was our entry point, but in other cases it may be deeper in the call graph. In this case, we've interactively moved the selector `>` down to the `sumup` call (you cannot descend into the `"runtime dispatch to..."` as there is no known code associated with it) and hit `<Enter>`, at which point Cthulhu showed us that the call to `make_vals(::Any)` occured only on line 4 of the definition of `sumup` (which we entered at the REPL). Cthulhu is now prompting us to either open the code in an editor (which will fail in this case, since there is no associated file!) or view the type-annoted code. If we select the "Browse typed code" option we see
+
+```
+sumup(f) @ Main REPL[7]:1
+ 1 function sumup(f::Core.Const(sin))::Any
+ 2     # this function uses the non-constant global variable `n` here
+ 3     # and it makes every succeeding operations type-unstable
+ 4     vals::Any = make_vals(n::Any)::Any
+ 5     s::Any = zero(eltype(vals::Any)::Any)::Any
+ 6     for v::Any in vals::Any::Any
+ 7         (s::Any += f::Core.Const(sin)(v::Any)::Any)::Any
+ 8     end
+ 9     return s::Any
+10 end
+Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
+⋮
+```
+
+with red highlighting to indicate the non-inferable arguments.
+
+For more information, you're encouraged to read Cthulhu's documentation, which includes a video tutorial better-suited to this interactive tool.
 
 ## [Entry Points](@id optanalysis-entry)
 

diff --git a/ext/JETCthulhuExt.jl b/ext/JETCthulhuExt.jl
@@ -0,0 +1,25 @@
+module JETCthulhuExt
+
+using JET: JET, InferenceErrorReport, VirtualFrame, PrintConfig, print_signature
+using Cthulhu: Cthulhu, Node, Data, callstring
+using Core: MethodInstance
+
+const _emptybackedges = MethodInstance[]
+
+struct CallFrames
+    frames::Vector{VirtualFrame}
+end
+
+function Cthulhu.treelist(r::InferenceErrorReport)
+    io = IOBuffer()
+    cf = CallFrames(r.vst)
+    print_signature(IOContext(io, :color=>true), r.sig, PrintConfig())
+    # printstyled(IOContext(io, :color=>true), r.sig.tt, color=:red)
+    Cthulhu.treelist!(Node(Data{Union{MethodInstance,Type}}("runtime dispatch to " * String(take!(io)), r.sig.tt)), io, cf, "", Base.IdSet{Union{MethodInstance,Nothing}}([nothing]))
+end
+
+Cthulhu.instance(cf::CallFrames) = isempty(cf.frames) ? nothing : cf.frames[end].linfo
+Cthulhu.backedges(cf::CallFrames) = isempty(cf.frames) ? _emptybackedges : [cf.frames[end].linfo]
+Cthulhu.nextnode(cf::CallFrames, ::MethodInstance) = CallFrames(cf.frames[1:end-1])
+
+end
diff --git a/src/JET.jl b/src/JET.jl
@@ -6,7 +6,7 @@ module JET
 export
     # jetanalyzer
     @report_call, report_call, @test_call, test_call,
-    report_file, test_file, report_package, test_package, report_text, test_text,
+    report_file, test_file, report_package, test_package, report_text, reportkey, test_text,
     watch_file,
     # optanalyzer
     @report_opt, report_opt, @test_opt, test_opt,
@@ -294,6 +294,8 @@ get_linfo(linfo::MethodInstance) = linfo
 is_constant_propagated(frame::InferenceState) = is_constant_propagated(frame.result)
 is_constant_propagated(result::InferenceResult) = CC.any(result.overridden_by_const)
 
+struct TypeUnassigned end    # for when inference doesn't bother assigning a type to a slot (e.g. dead code)
+
 # lattice
 
 ignorenotfound(@nospecialize(t)) = t === NOT_FOUND ? Bottom : t