Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-1.9: Backports for julia 1.9.0-alpha2 / 1.9.0-beta1 #47602

Merged
merged 57 commits into from
Dec 27, 2022

Commits on Nov 17, 2022

  1. Limit initial OpenBLAS thread count (#46844)

    * Limit initial OpenBLAS thread count
    
    We set OpenBLAS's initial thread count to `1` to prevent runaway
    allocation within OpenBLAS's initial thread startup.  LinearAlgebra will
    later call `BLAS.set_num_threads()` to the actual value we require.
    
    * Support older names
    
    (cherry picked from commit 58b559f)
    staticfloat authored and KristofferC committed Nov 17, 2022
    Configuration menu
    Copy the full SHA
    05c8f3b View commit details
    Browse the repository at this point in the history
  2. fix #46778, precompile() for abstract but compileable signatures (#47259

    )
    
    (cherry picked from commit fe81138)
    JeffBezanson authored and KristofferC committed Nov 17, 2022
    Configuration menu
    Copy the full SHA
    9606388 View commit details
    Browse the repository at this point in the history
  3. fix #47410, syntax error with anonfn inside elseif and short-circui…

    …t op (#47499)
    
    (cherry picked from commit 5f256e7)
    JeffBezanson authored and KristofferC committed Nov 17, 2022
    Configuration menu
    Copy the full SHA
    0e17295 View commit details
    Browse the repository at this point in the history
  4. ensure bindings handle write barriers for ty and globalref (#47580)

    This has probably been wrong for a long time (since being introduced in 7908246).
    
    (cherry picked from commit b369511)
    vtjnash authored and KristofferC committed Nov 17, 2022
    Configuration menu
    Copy the full SHA
    9dfd76b View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2022

  1. Turn on Intel jitevents by default on Linux (#47586)

    (cherry picked from commit bba41d4)
    vchuravy authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    d85e9ac View commit details
    Browse the repository at this point in the history
  2. Doc: The default sorting alg. is stable from 1.9 (#47579)

    * Update doc/src/base/sort.md
    
    * Update docs: The default sorting alg. is stable
    
    * Compat 1.9 for QuickSort to be stable
    
    * Specify the default algorithm
    
    * Use example from InlineStrings.jl
    
    * Change example to jldoctest
    
    * Remove "*appear* to be stable." as slightly misleading.
    
    Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com>
    (cherry picked from commit c5fe17b)
    petvana authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    b5d6b03 View commit details
    Browse the repository at this point in the history
  3. update MPFR (#47659)

    checksums updated via approach described in #47174.
    
    (cherry picked from commit 5996520)
    simonbyrne authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    3bc94a9 View commit details
    Browse the repository at this point in the history
  4. Add compat note for sortperm(x; dims) (#47657)

    (cherry picked from commit 4fa07cd)
    mcabbott authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    f8f0c63 View commit details
    Browse the repository at this point in the history
  5. build: add get-lld target (#47589)

    Fixes `make -C deps getall`
    
    (cherry picked from commit 3200219)
    petvana authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    382661c View commit details
    Browse the repository at this point in the history
  6. Print the detailed type on heap snapshot (#47503)

    Fixes #47502
    
    (cherry picked from commit 27ebaa7)
    gbaraldi authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    0f271d7 View commit details
    Browse the repository at this point in the history
  7. Remove typeinfer lock altogether (#46825)

    * Remove typeinfer lock altogether
    
    * Don't remove the typeinf lock functions
    
    * Track reentrancy in current task state
    
    * Fix up some git status
    
    * Initialize task variables
    
    * Promise that jl_typeinf_func is rooted somewhere
    
    (cherry picked from commit 113efb6)
    pchintalapudi authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    c615a49 View commit details
    Browse the repository at this point in the history
  8. Fix regression in generic_bitcast with Union{} arguments. (#47605)

    (cherry picked from commit 726bbd7)
    maleadt authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    3b414c6 View commit details
    Browse the repository at this point in the history
  9. Filesystem: rm(; recursive=true) should ignore UV_EACCES (#47668)

    The command-line program `rm` has no problem deleting an empty directory
    that we do not have listing permissions on, so we should follow suit.
    
    Example:
    
    ```
    mktempdir() do dir
        mkpath("$(dir)/foo")
        chmod("$(dir)/foo", 0o200)
        rm(dir; recursive=true)
    end
    ```
    
    (cherry picked from commit d0a211a)
    staticfloat authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    aba112c View commit details
    Browse the repository at this point in the history
  10. Fix overflow in pow5 (#47511)

    Fixup for #46764
    
    (cherry picked from commit 02aa0b0)
    LilithHafner authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    24505fc View commit details
    Browse the repository at this point in the history
  11. Fix GCExt test (#47699)

    * Add test/gcext to out-of-tree
    * Disable gcext test that uses jl_gc_internal_obj_base_ptr
    
    (cherry picked from commit 5495b8d)
    vchuravy authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    6fb45be View commit details
    Browse the repository at this point in the history
  12. Fix REPL keybinding CTRL-Q for stdlib methods (#47637)

    (cherry picked from commit 7514bcf)
    rashidrafeek authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    0865ae0 View commit details
    Browse the repository at this point in the history
  13. fix 5-arg mul! for vectors of vectors (#47665)

    Co-authored-by: N5N3 <2642243996@qq.com>
    (cherry picked from commit 902e8a7)
    ranocha authored and KristofferC committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    cfbb86a View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2022

  1. Add support for "package extensions" to code loading (#47695)

    * Add support for "glue packages" to code loading
    
    This allows packages to define "glue packages" which
    are modules that are automatically loaded when
    a set of other packages are loaded into the Julia
    session.
    
    (cherry picked from commit 495a004)
    KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    93587d7 View commit details
    Browse the repository at this point in the history
  2. 🤖 Bump the Pkg stdlib from ed6a5497e to 5d8b9ddb8 (#47828)

    Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
    (cherry picked from commit db00cc1)
    DilumAluthgeBot authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    dc1369e View commit details
    Browse the repository at this point in the history
  3. Fix and simplify inference timing logic (#47711)

    * Fix and simplify inference timing logic
    
    * Reduce task struct size
    
    (cherry picked from commit 88a0627)
    pchintalapudi authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    9b9a703 View commit details
    Browse the repository at this point in the history
  4. Fix nth_methtable tparam of -1 when n==0 (#47666)

    Fixes #47625
    
    (cherry picked from commit 3f9409c)
    apaz-cli authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    1b85c1f View commit details
    Browse the repository at this point in the history
  5. Fix generator-invocation legality check for varargs generators (#47739)

    This code was introduced by me back in #31025 to speed up evaluation
    of generated functions that didn't make use of all of their arguments to
    make generation decisions. However, it neglected to take into account the
    possibility that the generator could be varargs. As a result, an unfortunate
    coincidence of an unused slot in the correct position could have allowed
    expansion of generators that were not supposed to be expandable. This can
    cause incorrect inference with all the usual consequences. However, fortunately
    this coincidence appears to be pretty rare.
    
    Fixes JuliaDebug/CassetteOverlay.jl#12
    
    (cherry picked from commit 328dd57)
    Keno authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    81f8582 View commit details
    Browse the repository at this point in the history
  6. Bump libuv. (#47707)

    Adapts to the final version of the constrained/available memory APIs.
    
    (cherry picked from commit 41b73e2)
    maleadt authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    9da50d2 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    fcf02e8 View commit details
    Browse the repository at this point in the history
  8. strengthen setglobal to default to release-consume ordering (#47742)

    In looking at a TSAN report recently, I noticed that globals were
    getting stored as atomic-unordered (since c92ab5e #44182), instead
    of atomic-release as intended (since
    46135df #45484).
    
    (cherry picked from commit f4534d1)
    vtjnash authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    628c953 View commit details
    Browse the repository at this point in the history
  9. fix unescaping in global expressions (#47719)

    This fixes some issues around macro hygiene in `global` expressions.
    Apparently we always treat l-values in global expressions as being
    escaped, but we still need to be careful to handle type annotations and
    destructuring correctly.
    
    (cherry picked from commit cc25a13)
    simeonschaub authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    dfec160 View commit details
    Browse the repository at this point in the history
  10. Refactor and document sorting dispatch (#47383)

    * create an internal `_sort!` function and use it (rename the existing `_sort!` to `__sort!`)
    
    * test for several of bugs that slipped through test suite
    
    * Give each sorting pass and DEFAULT_STABLE a docstring
    
    * add pretty printing for the new algorithms that are much more flexible than the old ones
    
    * fix unexpected allocations in Radix Sort
    
    fixes #47474
    in this PR rather than separate to avoid dealing with the merge
    
    * support and test backwards compatibility with packages that depend in sorting internals
    
    * support 3-, 5-, and 6-argument sort! for backwards compatibility
    
    * overhall scratch space handling
    
    make _sort! return scratch space rather than sorted vector
    so that things like IEEEFloatOptimization can re-use the
    scratch space allocated on their first recursive call
    
    * test handling -0.0 in IEEEFloatOptimization
    
    * fix and test bug where countsort's correct overflow behavior triggers error due to unexpected promotion to UInt
    
    (cherry picked from commit cee0a04)
    LilithHafner authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    d8cbffd View commit details
    Browse the repository at this point in the history
  11. Comment out test in subtype that causes hang due to StackOverflow(#47792

    )
    
    (cherry picked from commit 327b7ac)
    vchuravy authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    930314e View commit details
    Browse the repository at this point in the history
  12. Prioritize build_dir for generated headers (#47783)

    (cherry picked from commit 0feaf5c)
    vchuravy authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    01ae8b7 View commit details
    Browse the repository at this point in the history
  13. Set OPENBLAS_NUM_THREADS=1 on local Distributed workers (#47803)

    This should prevent LinearAlgebra from trying to increase our OpenBLAS
    thread count in its `__init__()` method when we're not trying to enable
    threaded BLAS.
    
    (cherry picked from commit a8b3994)
    staticfloat authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    dba443d View commit details
    Browse the repository at this point in the history
  14. Add native julia fmod (#47501)

    * Add native julia rem
    
    Co-authored-by: Alex Arslan <ararslan@comcast.net>
    (cherry picked from commit cf5ae03)
    gbaraldi authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    ce7a372 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    712a123 View commit details
    Browse the repository at this point in the history
  16. Replace the .ji serialization with sysimage format

    This unifies two serializers, `dump.c` (used for packages)
    and `staticdata.c` (used for system images). It adopts the
    `staticdata` strategy, adding support for external linkage,
    uniquing of MethodInstances & types, method extensions,
    external specializations, and invalidation. This lays the
    groundwork for native code caching as done with system images.
    
    Co-authored-by: Valentin Churavy <v.churavy@gmail.com>
    Co-authored-by: Jameson Nash <vtjnash@gmail.com>
    Co-authored-by: Tim Holy <tim.holy@gmail.com>
    (cherry picked from commit cbfdb3f)
    timholy authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    d561447 View commit details
    Browse the repository at this point in the history
  17. Allow re-initialization and caching of foreign types (#47407)

    Co-authored-by: Tim Holy <tim.holy@gmail.com>
    Co-authored-by: Max Horn <max@quendi.de>
    (cherry picked from commit e06a591)
    vchuravy authored and KristofferC committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    f8a5cd6 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2022

  1. Fix physical_memory exports. (#47859)

    (cherry picked from commit 5a6c808)
    maleadt authored and KristofferC committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    12a4863 View commit details
    Browse the repository at this point in the history
  2. Fix missing GC root in Symbol construction (#47865)

    The `Symbol` constructor in boot.jl was not using the unsafe_convert mechanism,
    becuase it is unavailable at this point in bootstrap. However, it was also not
    GC-rooting the string some other way, resulting in potential memory corruption.
    Fix that by manually inlining the :foreigncall and setting up the root
    appropriately.
    
    (cherry picked from commit b5a6b0f)
    Keno authored and KristofferC committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    2866e26 View commit details
    Browse the repository at this point in the history
  3. TOML: print: handle mixed vector of dicts and non-dicts (#47876)

    (cherry picked from commit 4ff6288)
    fonsp authored and KristofferC committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    0b845b1 View commit details
    Browse the repository at this point in the history
  4. Fixups for #47383 (fixes runbenchmarks("sort")) (#47822)

    * add test demonstrating overflow in countsort
    
    * fix overflow in countsort
    
    * remove unnecessary type annotations (fixes tests)
    
    This fixes the test failure because it allows for automatic conversion.
    The manual for implementing the AbstractArray interface also does not recomend
    a type signature for the value arg in setindex!.
    
    Co-authored-by: Lilith Hafner <Lilith.Hafner@gmail.com>
    (cherry picked from commit 965bc7d)
    LilithHafner authored and KristofferC committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    3d80653 View commit details
    Browse the repository at this point in the history
  5. fixes for jl_rewrap_unionall

    This behaved a bit differently than Base.rewrap_unionall, which meant it
    might make types like `Any where T` from `supertype(struct A{T} <: Any)`.
    This can confuse subtyping, which does not expect other types to appear
    to be wider than Any.
    
    (cherry picked from commit c0d9367)
    vtjnash authored and KristofferC committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    a548ee3 View commit details
    Browse the repository at this point in the history
  6. call specialized method instance when encountering unspecialized sparams

    In some instances, the preferred compilation signature will require
    sparams to be provided at runtime. When we build the cache around these,
    we need to make sure the method instance we are calling has those values
    computed for the current signature, and not use the widened signature.
    But we can still compile for the widened signature, we just need to make
    sure we create a cache entry for every narrower call signature.
    
    Fix #47476
    
    (cherry picked from commit 16d3b92)
    vtjnash authored and KristofferC committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    31df7c8 View commit details
    Browse the repository at this point in the history
  7. ensure sparams are cached correctly for widened methods

    Follow-up issue found while working on #47476
    
    (cherry picked from commit 71ab5fa)
    vtjnash authored and KristofferC committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    9827f1d View commit details
    Browse the repository at this point in the history
  8. ensure types are UnionAll wrapped are cached correctly for widened Va…

    …rarg methods
    
    And fix a related accuracy issue in jl_isa_compileable_sig
    
    Follow-up issue found while working on #47476
    
    (cherry picked from commit 9e5e28f)
    vtjnash authored and KristofferC committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    a506f43 View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2022

  1. add back wordaround for Slot objects should not occur in an AST in …

    …Ipython mode (#47878)
    
    (cherry picked from commit 7b10d5f)
    KristofferC authored and KristofferC committed Dec 16, 2022
    Configuration menu
    Copy the full SHA
    3e1373e View commit details
    Browse the repository at this point in the history
  2. Reduce invalidations when loading JuliaData packages (#47889)

    (cherry picked from commit e84634e)
    timholy authored and KristofferC committed Dec 16, 2022
    Configuration menu
    Copy the full SHA
    5848e99 View commit details
    Browse the repository at this point in the history
  3. intersect: fix a minor soundness issue with supertypes (#47813)

    When doing intersection, we might end up with a value in `env` (as the
    only possible *value* for that parameter) without properly considering
    that the parameter might be a TypeVar.
    
    (cherry picked from commit 26a7dbb)
    vtjnash authored and KristofferC committed Dec 16, 2022
    Configuration menu
    Copy the full SHA
    86e8ef9 View commit details
    Browse the repository at this point in the history
  4. make Ctrl-C during sleeping work better (#47901)

    fixes #46635
    
    co-authored-by: Jameson Nash <vtjnash@gmail.com>
    (cherry picked from commit b6f32bc)
    JeffBezanson authored and KristofferC committed Dec 16, 2022
    Configuration menu
    Copy the full SHA
    8432d4f View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2022

  1. revert promotions of abstract arrays inside other arrays (#47893)

    (cherry picked from commit 427432e)
    KristofferC authored and KristofferC committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    eba98e5 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2022

  1. Revert "Emit safepoints at function entry (#41616)"

    This reverts commit 1a7a131.
    KristofferC committed Dec 20, 2022
    Configuration menu
    Copy the full SHA
    a16ffd6 View commit details
    Browse the repository at this point in the history
  2. Revert "improve performance issue of @nospecialize-d keyword func c…

    …all (#47059)"
    
    This reverts commit 95cfd62.
    KristofferC committed Dec 20, 2022
    Configuration menu
    Copy the full SHA
    5a684f0 View commit details
    Browse the repository at this point in the history
  3. only load extensions once dependencies have finished loading (#47927)

    (cherry picked from commit 9be3c85)
    KristofferC authored and KristofferC committed Dec 20, 2022
    Configuration menu
    Copy the full SHA
    327e081 View commit details
    Browse the repository at this point in the history
  4. Precompile cache: always add worklist CIs (#47924)

    We cache only those external CodeInstances that link back to the
    package being precompiled. Formerly we required a backedge;
    this PRs adds any whose `specTypes` could only link back to the
    package. This scoops up a few runtime-dispatched CodeInstances
    and their callees.
    
    (cherry picked from commit 1f0700a)
    timholy authored and KristofferC committed Dec 20, 2022
    Configuration menu
    Copy the full SHA
    95cb3a8 View commit details
    Browse the repository at this point in the history
  5. put back the old QuickSort, PartialQuickSort, and MergeSort algorithm…

    …s... (#47788)
    
    ...as they were in 1.8 and rename the new PartialQuickSort to QuickerSort
    Also improve the documentation and API for constructing QuickerSort and test
    the API
    
    Co-authored-by: Lilith Hafner <Lilith.Hafner@gmail.com>
    (cherry picked from commit 8cdb17b)
    LilithHafner authored and KristofferC committed Dec 20, 2022
    Configuration menu
    Copy the full SHA
    f17d1df View commit details
    Browse the repository at this point in the history
  6. add bounds check to Slices indexing (#47622)

    Co-authored-by: Simon Byrne <simonbyrne@gmail.com>
    (cherry picked from commit d7363d8)
    mcabbott authored and KristofferC committed Dec 20, 2022
    Configuration menu
    Copy the full SHA
    3ea7f6c View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2022

  1. Reduce codegen lock scope (#46836)

    (cherry picked from commit 09a6ff8)
    pchintalapudi authored and KristofferC committed Dec 27, 2022
    Configuration menu
    Copy the full SHA
    22789c0 View commit details
    Browse the repository at this point in the history
  2. Implement support for object caching through pkgimages (#47184)

    This implements caching native code in "package images" (pkgimages).
    We now write two serialization files, one ending in `*.ji` and the other with
    the platform dynamic library extension (e.g., `*.so`). The `*.ji` contains
    "extended" header information (include the source-code dump for Revise),
    whereas the dynamic library includes the Julia objects, including LLVM-generated
    native code.
    
    Native code is compiled once during precompilation and then a second time
    to build a "clean" module. When we find an edge to an external function (already
    cached in anloaded pkgimage), we emit a global variable which we will patch during
    loading with the address of the function to call. This allows us to leverage the
    standard multiversioning capabilities.
    
    Co-authored-by: Tim Holy <tim.holy@gmail.com>
    Co-authored-by: Kristoffer Carlsson <kristoffer.carlsson@chalmers.se>
    Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
    Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
    Co-authored-by: Max Horn <max@quendi.de>
    Co-authored-by: Michael Schlottke-Lakemper <michael@sloede.com>
    Co-authored-by: Alex Ames <alexander.m.ames@gmail.com>
    (cherry picked from commit a2db90f)
    vchuravy authored and KristofferC committed Dec 27, 2022
    Configuration menu
    Copy the full SHA
    9a592dd View commit details
    Browse the repository at this point in the history
  3. Apply InitialOptimizations more consistently in sorting & fix dispa…

    …tch bug (#47946)
    
    * Apply InitialOptimizations by default in several cases when it was previously present
    
    * fixup for MissingOptimization
    
    * fix stability in the sortperm union with missing case
    
    (cherry picked from commit 12e679c)
    LilithHafner authored and KristofferC committed Dec 27, 2022
    Configuration menu
    Copy the full SHA
    7880930 View commit details
    Browse the repository at this point in the history
  4. Restore libgcc_s symlinkin in !macOS (#47986)

    Commit c8b72e2 completely removed libgcc_s symlinking (I assume unintentionally) in !macOS.
    
    (cherry picked from commit ea13810)
    antonio-rojas authored and KristofferC committed Dec 27, 2022
    Configuration menu
    Copy the full SHA
    6adc428 View commit details
    Browse the repository at this point in the history