Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports release 1.11 #53472

Merged
merged 25 commits into from
Mar 1, 2024
Merged

Backports release 1.11 #53472

merged 25 commits into from
Mar 1, 2024

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Feb 26, 2024

Backported PRs:

Contains multiple commits, manual intervention needed:

Non-merged PRs with backport label:

DilumAluthgeBot and others added 16 commits February 26, 2024 10:01
Stdlib: SparseArrays
URL: https://github.com/JuliaSparse/SparseArrays.jl.git
Stdlib branch: main
Julia branch: master
Old commit: c9f7293
New commit: cb602d7
Julia version: 1.12.0-DEV
SparseArrays version: 1.11.0(Does not match)
Bump invoked by: @dkarrasch
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)

Diff:
JuliaSparse/SparseArrays.jl@c9f7293...cb602d7

```
$ git log --oneline c9f7293..cb602d7
cb602d7 Add generic matmatmul for inplace sparse x sparse (#486)
95575c0 Speedup for `findmin()/findmax()` on sparse arrays (#510) (#511)
4cc31f2 Fix error message in getcommon() in cholmod (#509)
1748989 Move `fkeep!` docstring to the right function (#503)
1aa6431 Update ci.yml timeout (#507)
1f88ae1 Update ci.yml  - add codecov token (#504)
```

Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
(cherry picked from commit ddd7afb)
)

It has been possible for external abstract interpreters to keep custom
data in `codeinst.inferred` (together /w overloading `inlining_policy`).
After #52233, when such external absint uses
`InternalCodeCache`, this data is passed to `jl_ir_flag_inferred`,
leading to segfaults in assertion builds.

This commit resolves the issue by omitting `jl_ir_flag_inferred` checks
when the `cache_owner` is external. Nonetheless, a better resolution
might be necessary. It suggests that the current design of `code_owner`
and `InternalCodeCache` for the external cache system is somewhat
flawed. A conceivable approach could involve:
- Adding a layer similar to `inlining_policy` in
`CC.get(::WorldView{InternalCodeCache})` to enable safe redirection of
custom data to the native interpreter's implementation.
- Prohibiting custom data in the `inferred` field and directing such
data to be kept in `analysis_results`.

(cherry picked from commit 93876c9)
(cherry picked from commit 2197b5c)
Stdlib: Pkg
URL: https://github.com/JuliaLang/Pkg.jl.git
Stdlib branch: master
Julia branch: master
Old commit: 6dd0e7c9e
New commit: 76070d295
Julia version: 1.12.0-DEV
Pkg version: 1.11.0(Does not match)
Bump invoked by: @IanButterworth
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)

Diff:
JuliaLang/Pkg.jl@6dd0e7c...76070d2

```
$ git log --oneline 6dd0e7c9e..76070d295
76070d295 Prevent repl crash on invalid command (#3800)
d267986c2 RFC: stop testing non-ARM on mac on CI (#3794)
2571be6d1 CI: Add Apple Silicon (macOS aarch64) to the CI matrix (#3793)
3075fb78c Simplify Pkg.Registry APIs. (#3785)
e6f1e0902 Explain about Manifest.toml per julia version (#3791)
```

Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
(cherry picked from commit 59102aa)
Loading pkgimages would try to access the sysimage native code, which
will fail. Ensure that no code tries to load if the sysimage native code
is not available, as it may try to link against it.

Fixes #53147

(cherry picked from commit 02699bb)
Relands #53281 with some fixes noticed, though not the original causes
of the failure.

(cherry picked from commit fbc766a)
This was due to lowering keeping the same location info for the inserted
`return` or `goto` statement, even though the last seen location might
not have executed.

Also fixes inliner handling of the sentinel `0` value for code
locations.

(cherry picked from commit 61fc907)
Follow up to #53373, it seems this assert was broken for empty packages,
causing CI issues. It is not necessary.

Observed in CI here: #53395

https://buildkite.com/julialang/julia-master/builds/33860#018dc4dc-a603-4ad1-90cf-574540a41720
(cherry picked from commit 3742d33)
This aims to slightly simplify the synchronization by making
`n_threads_marking` the sole memory location of relevance for it, it
also removes the fast path, because being protected by the lock is
quite important so that the observed gc state arrays are valid.

Fixes: #53350
Fixes: #52757
Maybe fixes: #53026
Co-authored-by: Jameson Nash <vtjnash@gmail.com>

(cherry picked from commit a96726b)
… typevar. (#53429)

This should be safe as ∀ vars' bounds are frozen in env.
If there's no ∃ var, then the current env won't change after
`local_∀_∃_subtype`.
Thus, the slow path should be equivalent to the fast path if the latter
returns 1.
Close #53371.

(cherry picked from commit 37c48e8)
This prevents a race condition when building 'julia-cli-debug
julia-cli-release' simultaneously (as we do for libjulia_jll, and also
generally seems appropriate given what is done for all other source
files.

Motivated by JuliaPackaging/Yggdrasil#8151 so
I'll first see if it works there.

Closes #45002.

(cherry picked from commit fee198b)
The annotate! function provides a convenient way of adding annotations
to an AnnotatedString/AnnotatedChar without accessing any of the
implementation details of the Annotated* types.

When AnnotatedIOBuffer was introduced, an appropriate annotations method
was added, but annotate! was missed. To correct that, we refactor the
current annotate! method for AnnotatedString to a more general helper
function _annotate! that operates on the annotation vector itself, and
use this new helper method to easily provide an annotate! method for
AnnotatedIOBuffer.

(cherry picked from commit 6e1d062)
Memo to self:

* update the version in `stdlib/MozillaCACerts_jll/Project.toml`
* update `MOZILLA_CACERT_VERSION` in `deps/libgit2.version`
* generate new checksums with `make -f contrib/refresh_checksums.mk
mozillacert`
* manually delete old checksums in
`deps/checksums/cacert-<OLD-VERSIONS>`

(cherry picked from commit 669cbdc)
Memo to self:

* update version number in `stdlib/LibGit2_jll/Project.toml`
* update test result in `stdlib/LibGit2_jll/test/runtests.jl`
* update version number and commit sha in `deps/libgit2.version`
* refresh checksums with `make -f contrib/refresh_checksums.mk -j libgit2`

(cherry picked from commit 90d03b2)
@KristofferC KristofferC added the release Release management and versioning. label Feb 26, 2024
KristofferC and others added 6 commits February 26, 2024 10:04
…a precompile file against a set of custom flags (#53332)

(cherry picked from commit 9c0f1dc)
…53326)

This bypasses DEPOT_PATH, LOAD_PATH, and stale checks when loading
known stdlib code for known purposes from known locations, specifically
to avoid the problem that I cannot fix my tools because I have used my
tools to break my tools.

This helps avoid the situation that the user can break important Pkg,
REPL, Socket or similar features simply because they chose to remove
`@stdlib` from their environment. For example, if you make an edit to
REPL, then this will trigger recompilation and load the edited version:
```
$ ./julia -e 'using REPL' -iq
┌ Info: Precompiling REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb] (cache misses: include_dependency fsize change (2), invalid header (10), mismatched flags (1))
└ @ Base loading.jl:2643
julia>
```
But this will load the version that shipped with Julia, regardless of
the state of the cache or the source code (unless you delete the cache
files):
```
$ ./julia -iq
julia>
```

Fixes #53365

(cherry picked from commit ea1a0d2)
As discussed here:
#53286 (comment)

Readds the methods that were removed in
#12819.

(cherry picked from commit ea2b255)
This should fix the assertion failure that has been plaguing the Pkg
tests, as discussed in #52123 (comment)

(cherry picked from commit 6cbed31)
When we use options like code coverage, we can't use the native code
present in the cache file since it is not instrumented.

PR #52123 introduced the capability of skipping the native
code during loading, but created the issue that subsequent packages
could have an explicit or implicit dependency on the native code.

PR #53439 tainted the current process by setting
`use_sysimage_native_code`, but this flag is propagated to subprocesses
and lead to a regression in test time.

Move this to a process local flag to avoid the regression.
In the future we might be able to change the calling convention for
cross-image calls to `invoke(ci::CodeInstance, args...)` instead of
`ci.fptr(args...)` to handle native code not being present.

---------

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit b8a0a39)
Now that we take care of terminfo parsing ourselves, having a clear
origin and processing method for arriving at our reference terminfo data
seems somewhat important.

The original form was acquired by re-purposing some pre-processed
terminfo data from NCurses (I forget the exact source). I considered
adding a separate ingestion/processing script, but it occurred to me
that it would make sense to have the method for updating the data file
be _in_ the data file, by turning it into a quine.

This seems to work rather well, and so long as the NCurses source file
format stays the same, updating the terminfo data is now dead simple.

While working on the terminfo files, some minor refactors popped up as
"probably nice to have". One of which makes the reported number of
flags/numbers/strings actually accurate.

Lastly, to support the ergonomic use of capability variable names
instead of the short (read: uninformative) capname, we now also import
the NCurses-recognised extended capabilities, and generate/specify some
nice aliases for them.

-----

If we separate out the terminfo parser/data into a small stdlib, the
state here will be the initial state of the repo.

(cherry picked from commit 923fe2d)
@maleadt maleadt force-pushed the backports-release-1.11 branch from 91cf14a to ba189d8 Compare February 26, 2024 10:21
These changes are driven-by fixes I found during investigating into a
more complex issue related to precompilation with external abs int.
@KristofferC
Copy link
Member Author

@nanosoldier runtests(vs=":release-1.10")

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

…in create_expr_cache (#53387)

Fixes #53381

(cherry picked from commit a1db8da)
@KristofferC
Copy link
Member Author

#53387 was not backported here making the PkgEval result pretty much worthless.

Previously `conditional_successors_may_throw` performed post-domination
analysis not on the initially specified `bb` (which was given as the
argument), but on those BBs being analyzed that were popped from the
work-queue at the time. As a result, there were cases where not all
conditional successors were visited.

fixes #53508
@KristofferC KristofferC merged commit 2911937 into release-1.11 Mar 1, 2024
6 of 8 checks passed
@KristofferC KristofferC deleted the backports-release-1.11 branch March 1, 2024 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Release management and versioning.
Projects
None yet
Development

Successfully merging this pull request may close these issues.