Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-1.10: Backports for 1.10-alpha2 #50508

Merged
merged 41 commits into from
Jul 24, 2023
Merged

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Jul 11, 2023

Backported PRs:

Need manual backport:

Non-merged PRs with backport label:

@KristofferC KristofferC added the release Release management and versioning. label Jul 11, 2023
@vchuravy vchuravy changed the title release-1.0: Backports for 1.10-alpha2 release-1.10: Backports for 1.10-alpha2 Jul 11, 2023
aviatesk and others added 12 commits July 17, 2023 12:48
`Core._svec_ref` has accepted `boundscheck`-value as the first argument
since it was added in #45062. Nonetheless, `Core._svec_ref` simply calls
`jl_svec_ref` in either the interpreter or the codegen, and thus the
`boundscheck` value isn't utilized in any optimizations. Rather, even
worse, this `boundscheck`-argument negatively influences the effect
analysis (xref #50167 for details) and has caused type inference
regressions as reported in #50544.

For these reasons, this commit simply eliminates the `boundscheck`
argument from `Core._svec_ref`. Consequently,
`getindex(::SimpleVector, ::Int)` is now being concrete-eval eligible.

closes #50544
…current dir is unwritable (#50476)

Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>
(cherry picked from commit 0f6bfd6)
In the presence of `attach_after` insertions, we have to be careful to
extend the basic block to include everything up to the last insertion.
We were accounting for "new" nodes (before the compaction point), but
not "pending" nodes (after the compaction point).

Fixes #50379.

(cherry picked from commit cdec4c2)
Ensure `internal_obj_base_ptr` checks whether objects past freelist
pointer are in freelist.

Fixes #50434

(cherry picked from commit dcca46b)
Fixes #50451

(cherry picked from commit f4f1ecf)
We do not care about this condition (the point of this fast path is to
skip checking it).

Fix #50450

(cherry picked from commit 15bcf1b)
Fixes the case from #50518, but we actually have two test cases in
the tests that also hit this (e.g. this one:
```
f40964(xs::Int...=1; k = 2) = (xs, k)
```), but just happened not to hit the bad codegen path. #50556,
once merged would have complained on those definitions as well,
without this fix.

(cherry picked from commit c272236)
At present we only record direct `innervars` (`T` -> `S<:Val{T}`). And chained `innervars` might be ignored (`T` -> `S<:Val{V<:T}`\
This commit fix it.

(cherry picked from commit cd74337)
Fixes #50532. The `read(io, Char)` method didn't correctly handle the
case where the lead byte starts with too many leading ones; this fix
makes it handle that case correctly, which makes `read(io, Char)` match
`collect(s)` in its interpretation of what a character is in all invalid
cases. Also fix and test `read(::File, Char)` which has the same bug.

(cherry picked from commit ffe1a07)
* Use SparseArrays.jl updated to work with SuiteSparse 7, and with Int32 indices
* SuiteSparse_jll Update to v7.2.0 (Using @Wimmerer's SuiteSparse fork for 32-bit QR)

---------

Co-authored-by: Francois-Xavier Coudert <fxcoudert@gmail.com>
Co-authored-by: Will Kimmerer <kimmerer@mit.edu>
(cherry picked from commit b26f3b2)
vtjnash and others added 4 commits July 18, 2023 16:31
This used to make a lot of references to design issues with the
SparseArrays package (#2326 /
#20815), which result in a
non-sensical dispatch arrangement, and contribute to a slow loading
experience do to the nonsense Unions that must be checked by subtyping.

(cherry picked from commit 5a922fa)
…wed (#50541)

Usually this is caught by use of `eval`, but we should try to move away
from that broad rule to specific functions such as this one, such that
eventually we can remove that rule from `eval`.

Fix #50538

(cherry picked from commit 3a9345c)
stevengj and others added 2 commits July 18, 2023 16:40
…=false

only check that values are finite in `generic_lufact`  when `check=true`

(cherry picked from commit 824cdf1)
@KristofferC
Copy link
Member Author

KristofferC commented Jul 18, 2023

@nanosoldier runtests(ALL, vs = ":release-1.9")

@maleadt
Copy link
Member

maleadt commented Jul 18, 2023

FWIW, Nanosoldier defaults to using Julia + LLVM assertions now, so no need for the complicated invocation.

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

vtjnash and others added 11 commits July 24, 2023 07:24
Previously, this was only a WARNING message, which was often missed
during CI runs.

Closes #50451

(cherry picked from commit de2c37a)
In #48919, the tid selection logic inside `enq_task` gained a
`!GC.in_finalizer()` condition. However, this made it possible for
`workqueue_at` to be reached with `tid==0`, which would attempt and
out-of-bounds write under `@inbounds`, corrupting memory. This was not
caught in the test suite despite `--check-bounds=yes`, because our
`--check-bounds=yes` is currently best effort. That would be fixed by
#50239, which exposed this bug. This PR attempts to
fix this by marking any tasks launched inside a finalizer as not sticky.
Finalizers don't have any thread they run on
semantically, so i don't think there's a meaningful sense in which tasks
launched inside finalizers could be sticky.

(cherry picked from commit bd8350b)
Trying for cross-compile and disabling CSL and realized both of those
configurations were broken now.

(cherry picked from commit 9dc1380)
Optimizes invalid object lookup in conservative GC by just looking at the
page metadata and GC bits.

(cherry picked from commit c82656d)
Stdlib: SparseArrays
URL: https://github.com/JuliaSparse/SparseArrays.jl.git
Stdlib branch: main
Julia branch: master
Old commit: b4b0e72
New commit: 99c99b4
Julia version: 1.11.0-DEV
SparseArrays version: 1.10.0 (Does not match)
Bump invoked by: @dkarrasch
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)

Diff:
JuliaSparse/SparseArrays.jl@b4b0e72...99c99b4

```
$ git log --oneline b4b0e72..99c99b4
99c99b4 Specialize 3-arg `dot` for sparse self-adjoint matrices (#398)
cb10c1e use unwrapping mechanism for triangular matrices (#396)
b3872c8 added warning for iterating while mutating a sparse matrix (#415)
f8f0f40 bring coverage of fixed SparseMatrixCSC to 100% (#392)
0eb9c04 fix typos (#414)
```

Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
(cherry picked from commit 6691a75)
Due to #50082, reverting the causative portion from #41099, which stored
MethodInstances in LineInfoNodes.

(cherry picked from commit ae798cd)
This fixes certain possible regressions in `isassigned` for ranges to
restore the 1.9-like behavior.
On v1.9
```julia
julia> r = 1:big(2)^65
1:36893488147419103232

julia> isassigned(r, lastindex(r))
true

julia> isassigned(r, true)
ERROR: ArgumentError: invalid index: true of type Bool
```
On v1.10.0-alpha1
```julia
julia> isassigned(r, lastindex(r))
ERROR: InexactError: Int64(36893488147419103232)

julia> isassigned(r, true) # Bool is converted to Int
true

julia> r[true] # but indexing with Bool doesn't work
ERROR: ArgumentError: invalid index: true of type Bool
```
This PR
```julia
julia> isassigned(r, lastindex(r))
true

julia> isassigned(r, true)
ERROR: ArgumentError: invalid index: true of type Bool
```
This still leaves
```julia
julia> isassigned(collect(1:3), true)
true
```
so that should perhaps be changed as well.

(cherry picked from commit d270a71)
This PR implements GC heuristics based on the amount of pages allocated
instead of live objects like was done before.
The heuristic for new heap target is based on
https://dl.acm.org/doi/10.1145/3563323 (in summary it argues that the
heap target should have square root behaviour).
From my testing this fixes
#49545 and
#49761

(cherry picked from commit 32aa29f)
@KristofferC

This comment was marked as outdated.

@KristofferC
Copy link
Member Author

@nanosoldier runtests(["KirUtil", "ExpressBase", "MatrixLMnet", "ASE", "SparseMatricesCSR", "WeatherReport", "AstroTime", "UnitfulCurrency", "SIMDPolynomials", "DiffieHellman", "AhoCorasick", "FourierFlows", "SnoopPrecompile", "PauliPopRec", "Wflow", "ClimaCache", "FHIRClient", "Elfel", "JosephsonCircuits", "Xicor", "MacroTools", "McCabeThiele", "Dojo", "InvertibleNetworks", "KCenters", "IRViz", "SurrogatesMOE", "MRIsim", "InspectDR", "COBREXA", "VCFTools", "SLEEF", "Brillouin", "SwapLiterals", "Convex", "NExOS", "HomalgProject", "LazyArrays", "AIBECS", "MimiCIAM", "Qecsim", "YAXArrayBase", "ConstrainedDynamicsVis", "LoweredCodeUtils", "Plotly", "FeatureTransforms", "Yunir", "ParallelStencil", "PeaceCypher", "BootlegCassette", "DocStringExtensions", "DynamicalSystemsBase", "QuadEig", "Dagger", "Hose", "Lens", "AutoExpe", "Monads", "Weave", "SafeREPL", "GeoParquet", "PALEOaqchem", "QEDbase", "JSON3", "BitConverter", "PiecewiseDeterministicMarkovProcesses", "PyMNE", "GenerateProperties", "PSSFSS", "AxisKeys", "Constructs", "Photosynthesis", "SymbolicPlanners", "DiskDataProviders", "FourierSpaces", "FSimZoo", "PerformanceTestTools", "CommonDataModel", "EHTNCDBase", "TaylorDiff", "OpenQuantumTools", "FileCmp", "ForwardDiffPullbacks", "ArgoData", "DiffEqBase", "GaussianVariationalInference", "TermoQuimica", "GenLinearAlgebra", "JLD2", "ArviZExampleData", "MoistAir", "RustFFT", "FunctionalTables", "FeedbackNets", "ControllerFormats", "Turf", "BinaryRecordReader", "SIMDMath", "AlgebraOfGraphics", "BoundingSphere", "Hecke", "AxisSets", "NCTiles", "OptimKit", "NeuralAttentionlib", "BiodiversityObservationNetworks", "ModelPredictiveControl", "PALEOocean", "ArrayAllocators", "Knockoffs", "ZXCalculus", "MCMCChains", "vSmartMOM", "SaferIntegers", "LanguageServer", "LazyBandedMatrices", "PhaseSpaceIO", "FlexiMaps", "ExproniconLite", "PerformanceProfilingHttpEndpoints", "NeuralArithmetic", "SmallCouplingDynamicCavity", "POMDPModelChecking", "StrBase", "IncompressibleNavierStokes", "TheFix", "SPICE", "ComponentArrays", "SoilHydraulics", "TreeTools", "BraidChains", "CircleFit", "HypertextLiteral", "FractionalDiffEq", "Scruff", "AxisArrays", "SLEEFInline", "openPMD", "DLMReader", "DifferentiableStateSpaceModels", "NEOs", "FinEtoolsFlexBeams", "Polylabel", "ExtraFun", "SimpleTraits", "NCDatasets", "MAT", "DIVAnd", "LinearMaps", "Pingouin", "D3TypeTrees", "RangeEnclosures", "PortfolioOpt", "CorrectMatch", "KomaMRIPlots", "PProf", "PolySignedDistance", "PrecompileSignatures", "CanopyRadiativeTransfer", "Pajarito", "DimensionfulAngles", "CUDASIMDTypes", "Expronicon", "RayTracing", "DaemonMode", "IRTools", "StellaratorOptimization", "StaticStorages", "DeformableBodies", "FixArgs", "MLUtils", "DECAES", "Metrics", "NoncommutativeGraphs", "MarketRisk", "BitBasis", "DynamicQuantities", "RobotVisualizer", "StatisticalGraphics", "GEOTRACES", "EqualitySampler", "ForwardDiff", "ValSplit", "EHTImages", "GPLikelihoods", "DistributedFactorGraphs", "MutableTypes", "EAGO", "CloudMicrophysics", "ExprTools", "BioSequences", "Eyeball", "GeneratorArrays", "IsDef", "PlateMotionRequests", "DynamicGrids", "ZipStreams", "DynamicPPL", "OpenSpiel", "KLU", "Dash", "DifferentiableCollisions", "ADIOS2", "NiLangCore", "LatticeUtilities", "FMIImport", "SuiteSparseGraphBLAS", "VLBIImagePriors", "GeneralizedTransferMatrixMethod", "DistributedArrays", "FuzzyCompletions", "HDF5", "SpatialAccessTrees", "Transducers", "Judycon", "CausalGPSLC", "ControlSystemsBase", "RelativisticDynamics", "QBase", "CMPlot", "BinaryTraits", "NonlinearEigenproblems", "RMLImaging", "CMBLensing", "WhereTraits", "Metadata", "DistributedSparseGrids", "MeshCatMechanisms", "PointCloudRasterizers", "WorldOceanAtlasTools", "ResourceContexts", "StaticTools", "DataWrangler", "MethodAnalysis", "Sunny", "SIMD", "BIGUQ", "CountTimeSeries", "SimilaritySearch", "MetaPhylo", "PALEOmodel", "Expectations", "HomotopyContinuation", "AstroLib", "IndividualDisplacements", "Nullables", "NeutralLandscapes", "MultiChannelColors", "BitcoinPrimitives", "CBFV", "Algoim", "PALEOboxes", "CausalELM", "Thermodynamics", "ImPlot", "RationalRoots", "ConstrainedSystems", "Umlaut", "GeoParams", "HiddenMarkovModels", "ProtoStructs", "ContextManagers", "GADM", "ReferenceImplementations", "NetworkDynamics", "HybridArrays", "PBWDeformations", "NASAPrecipitation", "SimplePolynomials", "Yota", "Sinograms", "KeplerTools", "Clarabel", "FunSQL", "Braket", "FlashWeave", "FluxArchitectures", "SymEngine", "Z3", "UInt12Arrays", "DataPipes", "RDMREopt", "Amb", "QuerySQLite", "ComplexDiff", "CoverageTools", "GraphMakie", "IrrationalConstants", "PlutoDevMacros", "Chain", "ContinuousTransformations", "GLPKMathProgInterface", "AffineMaps", "ODEInterfaceDiffEq", "Xyce", "SSMPlots", "ImageBase", "SpectralKit", "ONNXNaiveNASflux", "ONNX", "DynamicGridsInteract", "SDPA", "MonkeyLang", "Silico", "ParameterEstimocean", "SurfaceReactions", "ShallowWaters", "OMEinsumContractionOrders", "SimpleValueGraphs", "Unfold", "LeafOptics", "DiffEqDevTools", "StructArrays", "GLM", "SubpixelRegistration", "QuasiCopula", "Optimisers", "QRMumps", "BSplineKit", "PDDL", "SpatialIndexing", "NoRainbows", "ParameterEstimation", "PhyloDiamond", "RSDeltaSigmaPort", "BigRationals", "RankRevealing", "SurrealDB", "InfiniteLinearAlgebra", "BSON", "WellKnownGeometry", "Catalyst", "PlotlyDocumenter", "ClimaLSM", "TopologyPreprocessing", "ParameterSpacePartitions", "Jaynes", "AbstractLogic", "MPIReco", "ModelParameters", "ClimaCoreTempestRemap", "ColorTypes", "CryoGrid", "SurfaceFluxes", "RegressionDiscontinuity", "SeisMain", "PonchonSavarit", "Rimu", "YaoSubspaceArrayReg", "Intersections", "TuringPatterns", "BlockBandedMatrices", "LatticeDiracOperators", "EasyJobsBase", "InvertedFiles", "SoftGlobalScope", "NeidArchive", "MemoizedMethods", "Secp256k1", "PlanningDomains", "RecursiveArrayTools", "Casacore", "StomataModels", "RRTMGP", "ConcurrentSim", "EquilibriumMeasures", "RegularizationTools", "PrincipalMomentAnalysisApp", "ReactionNetworkImporters", "BVHFiles", "Flatten", "InMemoryDatasets", "RigidBodyDynamics", "SynchronicBallot", "RobustModels", "MuladdMacro", "StringManipulation", "QuantumOpticsBase", "HalfIntegers", "Pipe", "CMDimData", "DojoEnvironments", "MRIFiles", "FillArrays", "DynamicExpressions", "RobustMFit", "FastDifferentiation", "SparseArrays", "SpeedyWeather", "AlgebraicDynamics", "UpdatableQRFactorizations", "MatrixFactorizations", "Stopping", "Missings", "EFTfitter", "RAPIDS", "PGENFiles", "FMI", "NodesAndModes", "Bijectors", "SuperLU", "AlgamesDriving", "Mozi", "Liblet", "RecordedArrays", "TraceFuns", "SimpleWorkflows", "Muscade", "ChemometricsTools", "Rematch", "MuseInference", "CellListMap", "JobSchedulers", "Earth2014", "MParT", "CategoricalDistributions", "Parquet2", "Glimmer", "EDFPlus", "ImagePhantoms", "StableHashes", "Metida", "MPIFiles", "InlineStrings", "DynamicGridsGtk", "DiscreteEvents", "PhysOcean", "SyntheticEddyMethod", "TMLE", "TrajectoryOptimization", "TORA", "CxxCall", "CharacteristicInvFourier", "MagmaThermoKinematics", "Lighthouse", "RecursiveFactorization", "NiLang", "NighttimeLights", "StaticLint", "CompositeStructs", "StatProfilerHTML", "Thinkers", "SymbolServer", "MimiPAGE2020", "Circuitscape", "Spec", "Stencils", "InfiniteArrays", "GCMAES", "Polymake", "FuncPipelines", "StatsBase", "StableSpectralElements", "TrajOptPlots", "EasyABM", "FindClosest", "RestrictedBoltzmannMachines", "KomaMRICore", "REopt", "FrankWolfe", "MPSKitModels", "Polyhedra", "NaiveGAflux", "NTNk", "PackageCompiler"], vs = ":release-1.9", configuration = (buildflags = ["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],), vs_configuration = (buildflags = ["LLVM_ASSERTIONS=1", "FORCE_ASSERTIONS=1"],))

@nanosoldier

This comment was marked as outdated.

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

@KristofferC
Copy link
Member Author

#50578 is wrecking havoc but I guess that we just have to get through it...

@KristofferC KristofferC merged commit f9b2614 into release-1.10 Jul 24, 2023
@KristofferC KristofferC deleted the backports-release-1.10 branch July 24, 2023 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Release management and versioning.
Projects
None yet
Development

Successfully merging this pull request may close these issues.