Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make jl_*affinity tests more portable #55261

Merged
merged 1 commit into from
Jul 27, 2024
Merged

Make jl_*affinity tests more portable #55261

merged 1 commit into from
Jul 27, 2024

Conversation

ararslan
Copy link
Member

Changes made:

  • Use 0 for the thread ID to ensure it's always valid. The function expects 0 <= tid < jl_n_threads so 1 is incorrect if jl_n_threads is 1.
  • After retrieving the affinity mask with jl_getaffinity, pass that same mask back to jl_setaffinity. This ensures that the mask is always valid. Using a mask of all ones results in EINVAL on FreeBSD. Based on the discussion in Add jl_getaffinity and jl_setaffinity #53402, this change may also fix Windows, so I've tried reenabling it here.
  • To check whether jl_getaffinity actually did something, we can check that the mask is no longer all zeros after the call.

Fixes #54817

Changes made:
- Use 0 for the thread ID to ensure it's always valid. The function
  expects `0 <= tid < jl_n_threads` so 1 is incorrect if `jl_n_threads`
  is 1.
- After retrieving the affinity mask with `jl_getaffinity`, pass that
  same mask back to `jl_setaffinity`. This ensures that the mask is
  always valid. Using a mask of all ones results in `EINVAL` on FreeBSD.
  It may also have been the cause of the failure on Windows observed in
  PR 53402 before testing on Windows was disabled; to verify, we can
  reenable Windows and see what happens.
- To check whether `jl_getaffinity` actually did something, we can check
  that the mask is no longer all zeros after the call.

Fixes 54817
@ararslan ararslan added test This change adds or pertains to unit tests multithreading Base.Threads and related functionality bugfix This change fixes an existing bug backport 1.11 Change should be backported to release-1.11 labels Jul 26, 2024
@ararslan
Copy link
Member Author

Confirmed also fixes the invalid argument failure on Windows observed in #53402 (cc @carstenbauer). I guess it's just Linux that doesn't complain about an invalid mask.

@vtjnash vtjnash added the merge me PR is reviewed. Merge when all tests are passing label Jul 27, 2024
@ararslan ararslan merged commit 8a7e23d into master Jul 27, 2024
10 of 12 checks passed
@ararslan ararslan deleted the aa/jl_affinity-test branch July 27, 2024 03:03
@giordano giordano removed the merge me PR is reviewed. Merge when all tests are passing label Jul 27, 2024
KristofferC pushed a commit that referenced this pull request Aug 2, 2024
Changes made:
- Use 0 for the thread ID to ensure it's always valid. The function
expects `0 <= tid < jl_n_threads` so 1 is incorrect if `jl_n_threads` is
1.
- After retrieving the affinity mask with `jl_getaffinity`, pass that
same mask back to `jl_setaffinity`. This ensures that the mask is always
valid. Using a mask of all ones results in `EINVAL` on FreeBSD. Based on
the discussion in #53402, this change may also fix Windows, so I've
tried reenabling it here.
- To check whether `jl_getaffinity` actually did something, we can check
that the mask is no longer all zeros after the call.

Fixes #54817

(cherry picked from commit 8a7e23d)
@KristofferC KristofferC mentioned this pull request Aug 2, 2024
68 tasks
lazarusA pushed a commit to lazarusA/julia that referenced this pull request Aug 17, 2024
Changes made:
- Use 0 for the thread ID to ensure it's always valid. The function
expects `0 <= tid < jl_n_threads` so 1 is incorrect if `jl_n_threads` is
1.
- After retrieving the affinity mask with `jl_getaffinity`, pass that
same mask back to `jl_setaffinity`. This ensures that the mask is always
valid. Using a mask of all ones results in `EINVAL` on FreeBSD. Based on
the discussion in JuliaLang#53402, this change may also fix Windows, so I've
tried reenabling it here.
- To check whether `jl_getaffinity` actually did something, we can check
that the mask is no longer all zeros after the call.

Fixes JuliaLang#54817
KristofferC added a commit that referenced this pull request Aug 26, 2024
Backported PRs:
- [x] #54962 <!-- Add timing to precompile trace compile -->
- [x] #55180 <!-- compress jit debuginfo for easy memory savings -->
- [x] #54919 <!-- Fix annotated join with non-concrete eltype iters -->
- [x] #55013 <!-- [docs] change docstring to match code -->
- [x] #55017 <!-- TOML: Make `Dates` a type parameter -->
- [x] #54033 <!-- Fix a bug in `stack`'s DimensionMismatch error message
-->
- [x] #55242 <!-- fix at-main docstring to not code quote a compat box
-->
- [x] #55261 <!-- Make `jl_*affinity` tests more portable -->
- [x] #54736 <!-- specificity: ensure fast-path in `sub/eq_msp` handle
missing `UnionAll` wrapper correctly. -->
- [x] #55299 <!-- typeintersect: fix bounds merging during inner
`intersect_all`. -->
- [x] #55302 <!-- Add `lbt_forwarded_funcs()` to debug LBT forwarding
issues -->
- [x] #55148 <!-- Random: Mark unexported public symbols as public -->
- [x] #55303 <!-- avoid overflowing show for OffsetArrays around typemax
-->
- [x] #55317 <!-- Restrict argument to `isleapyear(::Integer)` -->
- [x] #55327 <!-- Profile: Fix stdlib paths -->
- [x] #55330 <!-- [libblastrampoline] Bump to v5.11.0 -->
- [x] #55310 <!-- Preserve structure in scaling triangular matrices by
NaN -->
- [x] #55329 <!-- mapreduce: don't inbounds unknown functions -->
- [x] #55356 <!-- Profile: close files when assembling heap snapshot -->
- [x] #55371 <!-- Fix tr for block SymTridiagonal -->
- [x] #55307 <!-- Make REPL.TerminalMenus public -->
- [x] #55362 <!-- inference: fix missing LimitedAccuracy markers -->
- [x] #55306 <!-- AllocOpt: Fix stack lowering where alloca continas
boxed and unboxed data -->
- [x] #55395 <!-- fix #55389: type-unstable `join` -->
- [x] #55226 <!-- re-add `unsafe_convert` for Reinterpret and Reshaped
array -->
- [x] #55405 <!-- handle unbound vars in NTuple fields -->
- [x] #55365 <!-- ml-matches: ensure all methods are included -->
- [x] #55428 <!-- codegen: move undef freeze before promotion point -->
- [x] #55419 <!-- `stale_cachefile`: handle if the expected cache file
is missing -->
- [x] #55470 <!-- Add push! implementation for AbstractArray depending
only on resize! -->
- [x] #55483 <!-- fix hierarchy level of "API reference" in `Dates`
documentation -->
- [x] #55268 <!-- simplify complex atanh and remove singularity
perturbation -->
- [x] #55441 <!-- fix Event to use normal Condition variable -->
- [x] #55413 <!-- subtyping: fast path for lhs union and rhs typevar -->
- [x] #55492 <!-- build: add missing dependencies for expmap -->
- [x] #55507 <!-- Fix fast getptls ccall lowering. -->
- [x] #55424 <!-- add missing clamp function for IOBuffer -->
- [x] #55504 <!-- Update symmetric docstring to reflect the type of uplo
-->
- [x] #55107 <!-- Make the memory GEP an inbounds GEP since the bounds
check has happened somewhere else -->
- [x] #55411 <!-- Vendor the terminfo database for use with
base/terminfo.jl -->
- [x] #55452 <!-- Do not load `ScopedValues` with `using` -->
- [x] #55407 <!-- Remove deprecated non string API for LLVM pass
pipeline and parse all options -->
- [x] #55461 <!-- 🤖 [master] Bump the StyledStrings stdlib from d7496d2
to f6035eb -->
- [x] #55433 <!-- Backport #55407
to 1.11 -->
- [x] #55225 <!-- [1.11 backport] trace-compile: don't generate
`precompile` statements for OpaqueClosure methods (#55072) -->
- [x] #55212 <!-- Make `Base.depwarn()` public -->
- [x] #552
- [x] #55052 <!-- Fix `(l/r)mul!` with `Diagonal`/`Bidiagonal` -->
- [x] #55251 <!-- Restrict binary ops for Diagonal and Symmetric to
Number eltypes -->95 <!-- LAPACK: Aggressive constprop to concretely
infer syev!/syevd! -->
- [x] #55522 <!-- Fix tr for Symmetric/Hermitian block matrices -->

Need manual backport:
- [x] #55342 <!-- Ensure bidiagonal setindex! does not read indices in
error message -->

Contains multiple commits, manual intervention needed:

- [ ] #55336 <!-- codegen: take gc roots (and alloca alignment) more
seriously -->


Non-merged PRs with backport label:
- [ ] #55506 <!-- Fix indexing in _mapreducedim for OffsetArrays -->
- [ ] #55500 <!-- make jl_thread_suspend_and_get_state safe -->
- [ ] #55499 <!-- propagate the terminal's `displaysize` to the
`IOContext` used by the REPL -->
- [ ] #55458 <!-- Allow for generically extracting unannotated string
-->
- [ ] #55457 <!-- Make AnnotateChar equality consider annotations -->
- [ ] #55453 <!-- Privatise the annotations API, for StyledStrings -->
- [ ] #55443 <!-- Add test for upper/lower/titlecase and fix call -->
- [ ] #55355 <!-- relocation: account for trailing path separator in
depot paths -->
- [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows -->
- [ ] #55169 <!-- `propertynames` for SVD respects private argument -->
- [ ] #54457 <!-- Make `String(::Memory)` copy -->
- [ ] #53957 <!-- tweak how filtering is done for what packages should
be precompiled -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
- [ ] #50813 <!-- More doctests for Sockets and capitalization fix -->
- [ ] #50157 <!-- improve docs for `@inbounds` and
`Base.@propagate_inbounds` -->
- [ ] #41244 <!-- Fix shell `cd` error when working dir has been deleted
-->
@KristofferC KristofferC removed the backport 1.11 Change should be backported to release-1.11 label Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix This change fixes an existing bug multithreading Base.Threads and related functionality test This change adds or pertains to unit tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Regression in threads tests on FreeBSD
4 participants