serialization: fix relocatability bug #54738
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vtjnash
added
backport 1.10
|
Seems like this is not quite sufficient, since if an external client (e.g GPUCompiler) captures a reference to an CodeInstance, we may segfault later, since only those Method's that are present in new_ext_cis get serialized correctly (for other CI, the relocatability bit is computed wrong during serialization). I am not certain if we need to attempt to make that legal or recompute the bit. |
60 tasks
vtjnash
added a commit
that referenced
this pull request
Jun 14, 2024
Update the CodeInstance after optimizations with the code (if possible), but construct it early so that there will be the option of using it for edges, invoke, and rettype during optimizations, without problems with cycles in the call graph inhibiting that representability. The intent is that this will let us use CodeInstance as targets for `:invoke` calls, as discussed in https://hackmd.io/bpLnKm48TbWP_U_PGWgEVg. It complicates the life-cycle of a CodeInstance a little bit, so I don't think we have yet fully specified the thread-safety of this and which are valid transitions here (in general, it is only legal to narrow these properties or widen the worlds), but I think that is okay to postpone. It will also eventually require more bug-fixes to the way serialization is handled (such as #54738).
46 tasks
|
I'm not really familiar with the serializer, so I don't think I'm the best person to review, unless there's a specific GPUCompiler issue you're referring to? |
|
This is part of the reason why we needed an integrated cache. I can locally backport and validate it against GPUCompiler. I would like to see some extended tests, maybe added to the external owner precompilation tests. |
|
I mostly just wanted to tag you so that you are aware of it. It is not strictly required that the caches be integrated for this particular bug, it is just caring about the extending-external list contents right now, and whether the roots are getting saved for it. |
KristofferC
pushed a commit
that referenced
this pull request
Jun 22, 2024
(cherry picked from commit 323e725)
IanButterworth
removed
the
merge me
KristofferC
added a commit
that referenced
this pull request
Jun 25, 2024
Backported PRs: - [x] #54361 <!-- [LBT] Upgrade to v5.9.0 --> - [x] #54474 <!-- Unalias source from dest in copytrito --> - [x] #54548 <!-- Fixes for bitcast bugs with LLVM 17 / opaque pointers --> - [x] #54191 <!-- make `AbstractPipe` public --> - [x] #53402 <!-- Add `jl_getaffinity` and `jl_setaffinity` --> - [x] #53356 <!-- Rename at-scriptdir project argument to at-script and search upwards for Project.toml --> - [x] #54545 <!-- typeintersect: fix incorrect innervar handling under circular env --> - [x] #54586 <!-- Set storage class of julia globals to dllimport on windows to avoid auto-import weirdness. Forward port of #54572 --> - [x] #54587 <!-- Accomodate for rectangular matrices in `copytrito!` --> - [x] #54617 <!-- CLI: Use `GetModuleHandleExW` to locate libjulia.dll --> - [x] #54605 <!-- Allow libquadmath to also fail as it is not available on all systems --> - [x] #54634 <!-- Fix trampoline assembly for build on clang 18 on apple silicon --> - [x] #54635 <!-- Aggressive constprop in trevc! to stabilize triangular eigvec --> - [x] #54645 <!-- ensure we set the right value to gc_first_tid --> - [x] #54554 <!-- make elsize public --> - [x] #54648 <!-- Construct LazyString in error paths for tridiag --> - [x] #54658 <!-- fix missing uuid check on extension when finding the location of an extension --> - [x] #54594 <!-- Switch to Pkg mode prompt immediately and load Pkg in the background --> - [x] #54669 <!-- Improve error message in inplace transpose --> - [x] #54671 <!-- Add boundscheck in bindingkey_eq to avoid OOB access due to data race --> - [x] #54672 <!-- make: Fix `sed` command for LLVM libraries with no symbol versioning --> - [x] #54624 <!-- more precise aliasing checks for SubArray --> - [x] #54679 <!-- 🤖 [master] Bump the Distributed stdlib from 6a07d98 to 6c7cdb5 --> - [x] #54604 <!-- Fix tbaa annotation on union selector bytes inside of structs --> - [x] #54690 <!-- Fix assertion/crash when optimizing function with dead basic block --> - [x] #54704 <!-- LazyString in reinterpretarray error messages --> - [x] #54718 <!-- fix prepend StackOverflow issue --> - [x] #54674 <!-- Reimplement dummy pkg prompt as standard prompt --> - [x] #54737 <!-- LazyString in interpolated error messages involving types --> - [x] #54642 <!-- Document GenericMemory and AtomicMemory --> - [x] #54713 <!-- make: use `readelf` for LLVM symbol version detection --> - [x] #54760 <!-- REPL: improve prompt! async function handler --> - [x] #54606 <!-- fix double-counting and non-deterministic results in `summarysize` --> - [x] #54759 <!-- REPL: Fully populate the dummy Pkg prompt --> - [x] #54702 <!-- lowering: Recognize argument destructuring inside macro hygiene --> - [x] #54678 <!-- Don't let setglobal! implicitly create bindings --> - [x] #54730 <!-- Fix uuidkey of exts in fast path of `require_stdlib` --> - [x] #54765 <!-- Handle no-postdominator case in finalizer pass --> - [x] #54591 <!-- Don't expose guard pages to malloc_stack API consumers --> - [x] #54755 <!-- [TOML] remove Dates hack, replace with explicit usage --> - [x] #54721 <!-- add try/catch around scheduler to reset sleep state --> - [x] #54631 <!-- Avoid concatenating LazyString in setindex! for triangular matrices --> - [x] #54322 <!-- effects: add new `@consistent_overlay` macro --> - [x] #54785 - [x] #54865 - [x] #54815 - [x] #54795 - [x] #54779 - [x] #54837 Contains multiple commits, manual intervention needed: - [ ] #52694 <!-- Reinstate similar for AbstractQ for backward compatibility --> - [ ] #54649 <!-- Less restrictive copyto! signature for triangular matrices --> Non-merged PRs with backport label: - [ ] #54779 <!-- make recommendation clearer on manifest version mismatch --> - [ ] #54739 <!-- finish implementation of upgradable stdlibs --> - [ ] #54738 <!-- serialization: fix relocatability bug --> - [ ] #54574 <!-- Make ScopedValues public --> - [ ] #54457 <!-- Make `String(::Memory)` copy --> - [ ] #53957 <!-- tweak how filtering is done for what packages should be precompiled --> - [ ] #53452 <!-- RFC: allow Tuple{Union{}}, returning Union{} --> - [ ] #53286 <!-- Raise an error when using `include_dependency` with non-existent file or directory --> - [ ] #51479 <!-- prevent code loading from lookin in the versioned environment when building Julia -->
KristofferC
removed
the
backport 1.11
KristofferC
added a commit
that referenced
this pull request
Aug 13, 2024
Backported PRs: - [x] #51351 <!-- Remove boxing in pinv --> - [x] #52678 <!-- Profile: Improve module docstring --> - [x] #54201 <!-- Fix generic triangular solves with empty matrices --> - [x] #54605 <!-- Allow libquadmath to also fail as it is not available on all systems --> - [x] #54634 <!-- Fix trampoline assembly for build on clang 18 on apple silicon --> - [x] #54635 <!-- Aggressive constprop in trevc! to stabilize triangular eigvec --> - [x] #54645 <!-- ensure we set the right value to gc_first_tid --> - [x] #54671 <!-- Add boundscheck in bindingkey_eq to avoid OOB access due to data race --> - [x] #54672 <!-- make: Fix `sed` command for LLVM libraries with no symbol versioning --> - [x] #54704 <!-- LazyString in reinterpretarray error messages --> - [x] #54713 <!-- make: use `readelf` for LLVM symbol version detection --> - [x] #54781 <!-- [LinearAlgebra] Improve resilience to unknown libblastrampoline flags --> - [x] #54837 <!-- Do not add type tag size to the `alloc_typed` lowering for GC allocations --> - [x] #54815 <!-- add sticky task warning to `@task` and `schedule` --> - [x] #55141 <!-- Update the aarch64 devdocs to reflect the current state of its support --> - [x] #55178 <!-- Compat for `Base.@nospecializeinfer` --> - [x] #55197 <!-- compat notice for a[begin] indexing --> - [x] #55209 <!-- correction to compat notice for a[begin] --> - [x] #55203 <!-- document mutable struct const fields --> - [x] #54769 <!-- add missing compat entry to edit --> - [x] #54791 <!-- Bump libblastrampoline to v5.10.1 --> - [x] #55070 <!-- LinearAlgebra: LazyString in error messages for Diagonal/Bidiagonal --> - [x] #54624 <!-- more precise aliasing checks for SubArray --> - [x] #54690 <!-- Fix assertion/crash when optimizing function with dead basic block --> - [x] #55084 <!-- Use triple quotes in TOML.print when string contains newline --> Need manual backport: - [ ] #52505 <!-- fix alignment of emit_unbox_store copy --> - [ ] #53373 <!-- fix sysimage-native-code=no option with pkgimages --> - [ ] #53984 <!-- Profile: fix heap snapshot is valid char check --> - [ ] #54276 <!-- Fix solve for complex `Hermitian` with non-vanishing imaginary part on diagonal --> - [ ] #54669 <!-- Improve error message in inplace transpose --> - [ ] #54871 <!-- Make warn missed transformations pass optional --> Contains multiple commits, manual intervention needed: - [ ] #52854 <!-- Change to streaming out the heap snapshot data --> - [ ] #53218 <!-- Fix interpreter_exec.jl test --> - [ ] #53833 <!-- Profile: make heap snapshots viewable in vscode viewer --> - [ ] #54303 <!-- LinearAlgebra: improve type-inference in Symmetric/Hermitian matmul --> - [ ] #52694 <!-- Reinstate similar for AbstractQ for backward compatibility --> - [ ] #54737 <!-- LazyString in interpolated error messages involving types --> - [ ] #54738 <!-- serialization: fix relocatability bug --> - [ ] #55052 <!-- Fix `(l/r)mul!` with `Diagonal`/`Bidiagonal` --> Non-merged PRs with backport label: - [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows --> - [ ] #55169 <!-- `propertynames` for SVD respects private argument --> - [ ] #55013 <!-- [docs] change docstring to match code --> - [ ] #51479 <!-- prevent code loading from lookin in the versioned environment when building Julia --> - [ ] #50813 <!-- More doctests for Sockets and capitalization fix --> - [ ] #50157 <!-- improve docs for `@inbounds` and `Base.@propagate_inbounds` --> - [ ] #41244 <!-- Fix shell `cd` error when working dir has been deleted -->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.