Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Option to pin GC threads #53073

Closed
carstenbauer opened this issue Jan 26, 2024 · 4 comments · Fixed by #53402
Closed

Option to pin GC threads #53073

carstenbauer opened this issue Jan 26, 2024 · 4 comments · Fixed by #53402
Assignees
@carstenbauer
Labels
GC Garbage collector multithreading Base.Threads and related functionality

Comments

@carstenbauer
Copy link
Member

I would like to add the ability to pin GC threads to ThreadPinning.jl. AFAICS, we (a user) can't run Julia code on GC threads. Hence, I think we need to add something like https://github.com/OpenMathLib/OpenBLAS/blob/4d0b7fbec04c95c90291938a5974f00673e10e68/driver/others/blas_server.c#L341-L369 to the C runtime.

Would it be possible/accepted to add such functions?

(cc @vchuravy)

PS: IIUC, we can't add this retrospectively to 1.10 but it would be great to get it into 1.11.
@giordano giordano added multithreading Base.Threads and related functionality GC Garbage collector labels Jan 26, 2024
@vchuravy vchuravy added the help wanted Indicates that a maintainer wants help on an issue or pull request label Jan 27, 2024
@carstenbauer
Copy link
Member Author

carstenbauer commented Feb 13, 2024 •
edited
Loading

Because I'm giving a Julia course at UCL this week, I couldn't implement this before 1.11 feature freeze (today), which is a bit sad. If I implement this next week, is there any chance that we can still get this into 1.11?

@gbaraldi
Copy link
Member

I believe so, if the code isn't super complex (I don't believe it would be)

@carstenbauer
Copy link
Member Author

carstenbauer commented Feb 19, 2024 •
edited
Loading

I'm trying to implement this but I have a problem. AFAICS, we don't seem to store the thread handles (uv_thread_t uvtid) of any of the threads, which I'd like to use for uv_thread_setaffinity and uv_thread_getaffinity calls later on.

Note that OpenBLAS stores these handles and it was thus pretty easy to add affinity functions, e.g. https://github.com/OpenMathLib/OpenBLAS/pull/3702/files#diff-2b77a530a65c94d60feb21d4ce2093df0a2bd3cb710d2dc82bab32f18e1c963a.

Should I add a new storage for them or is there a different/better way to get the handles? (@gbaraldi)

@carstenbauer
Copy link
Member Author

carstenbauer commented Feb 19, 2024 •
edited
Loading

Note to self: Ok, seems like we are storing the information in the TLS (specifically, in the field system_id). Guess I have to figure out now how to query the information from jl_all_tls_states. I guess via jl_atomic_load_relaxed?

@carstenbauer carstenbauer mentioned this issue Feb 20, 2024
Merged
@carstenbauer carstenbauer removed the help wanted Indicates that a maintainer wants help on an issue or pull request label Feb 22, 2024
@carstenbauer carstenbauer self-assigned this Feb 22, 2024
vchuravy added a commit that referenced this issue May 25, 2024
@carstenbauer @vchuravy @DilumAluthge
Add jl_getaffinity and jl_setaffinity (#53402)
065aeb6 
This PR adds two functions `jl_getaffinity` and `jl_setaffinity` to the
runtime, which are slim wrappers around `uv_thread_getaffinity` and
`uv_thread_setaffinity` and can be used to set the affinity of Julia
threads.

This will
* simplify thread pinning (ThreadPinning.jl currently pins threads by
spawning tasks that run the necessary ccalls) and
* enable users to also pin GC threads (or, more generally, all Julia
threads).

**Example:**
```julia
bauerc@n2lcn0146 julia git:(cb/affinity)  
➜ ./julia -q --startup-file=no --threads 2,3 --gcthreads 4,1
julia> cpumasksize = @CCall uv_cpumask_size()::Cint
1024

julia> mask = zeros(Cchar, cpumasksize);

julia> jl_getaffinity(tid, mask, cpumasksize) = ccall(:jl_getaffinity, Int32, (Int16, Ptr{Cchar}, Int32), tid, mask, cpumasksize)
jl_getaffinity (generic function with 1 method)

julia> jl_getaffinity(1, mask, cpumasksize)
0

julia> print(mask[1:Sys.CPU_THREADS])
Int8[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

julia> mask[1] = 0;

julia> jl_setaffinity(tid, mask, cpumasksize) = ccall(:jl_setaffinity, Int32, (Int16, Ptr{Cchar}, Int32), tid, mask, cpumasksize)
jl_setaffinity (generic function with 1 method)

julia> jl_setaffinity(1, mask, cpumasksize)
0

julia> fill!(mask, 0);

julia> jl_getaffinity(1, mask, cpumasksize)
0

julia> print(mask[1:Sys.CPU_THREADS])
Int8[0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
```
(cc @vchuravy, @gbaraldi)

Would be great to get this into 1.11 (despite feature freeze) because
otherwise we won't be able to pin GC threads until 1.12 (likely not
until the end of the year).

Closes #53073

---------

Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
DilumAluthge added a commit that referenced this issue Jun 3, 2024
@carstenbauer @vchuravy @DilumAluthge
Add jl_getaffinity and jl_setaffinity (#53402)
38ad67c 
This PR adds two functions `jl_getaffinity` and `jl_setaffinity` to the
runtime, which are slim wrappers around `uv_thread_getaffinity` and
`uv_thread_setaffinity` and can be used to set the affinity of Julia
threads.

This will
* simplify thread pinning (ThreadPinning.jl currently pins threads by
spawning tasks that run the necessary ccalls) and
* enable users to also pin GC threads (or, more generally, all Julia
threads).

**Example:**
```julia
bauerc@n2lcn0146 julia git:(cb/affinity)  
➜ ./julia -q --startup-file=no --threads 2,3 --gcthreads 4,1
julia> cpumasksize = @CCall uv_cpumask_size()::Cint
1024

julia> mask = zeros(Cchar, cpumasksize);

julia> jl_getaffinity(tid, mask, cpumasksize) = ccall(:jl_getaffinity, Int32, (Int16, Ptr{Cchar}, Int32), tid, mask, cpumasksize)
jl_getaffinity (generic function with 1 method)

julia> jl_getaffinity(1, mask, cpumasksize)
0

julia> print(mask[1:Sys.CPU_THREADS])
Int8[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

julia> mask[1] = 0;

julia> jl_setaffinity(tid, mask, cpumasksize) = ccall(:jl_setaffinity, Int32, (Int16, Ptr{Cchar}, Int32), tid, mask, cpumasksize)
jl_setaffinity (generic function with 1 method)

julia> jl_setaffinity(1, mask, cpumasksize)
0

julia> fill!(mask, 0);

julia> jl_getaffinity(1, mask, cpumasksize)
0

julia> print(mask[1:Sys.CPU_THREADS])
Int8[0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
```
(cc @vchuravy, @gbaraldi)

Would be great to get this into 1.11 (despite feature freeze) because
otherwise we won't be able to pin GC threads until 1.12 (likely not
until the end of the year).

Closes #53073

---------

Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
KristofferC pushed a commit that referenced this issue Jun 7, 2024
@carstenbauer
Add jl_getaffinity and jl_setaffinity (#53402)
6b4b5ef 
This PR adds two functions `jl_getaffinity` and `jl_setaffinity` to the
runtime, which are slim wrappers around `uv_thread_getaffinity` and
`uv_thread_setaffinity` and can be used to set the affinity of Julia
threads.

This will
* simplify thread pinning (ThreadPinning.jl currently pins threads by
spawning tasks that run the necessary ccalls) and
* enable users to also pin GC threads (or, more generally, all Julia
threads).

**Example:**
```julia
bauerc@n2lcn0146 julia git:(cb/affinity)
➜ ./julia -q --startup-file=no --threads 2,3 --gcthreads 4,1
julia> cpumasksize = @CCall uv_cpumask_size()::Cint
1024

julia> mask = zeros(Cchar, cpumasksize);

julia> jl_getaffinity(tid, mask, cpumasksize) = ccall(:jl_getaffinity, Int32, (Int16, Ptr{Cchar}, Int32), tid, mask, cpumasksize)
jl_getaffinity (generic function with 1 method)

julia> jl_getaffinity(1, mask, cpumasksize)
0

julia> print(mask[1:Sys.CPU_THREADS])
Int8[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

julia> mask[1] = 0;

julia> jl_setaffinity(tid, mask, cpumasksize) = ccall(:jl_setaffinity, Int32, (Int16, Ptr{Cchar}, Int32), tid, mask, cpumasksize)
jl_setaffinity (generic function with 1 method)

julia> jl_setaffinity(1, mask, cpumasksize)
0

julia> fill!(mask, 0);

julia> jl_getaffinity(1, mask, cpumasksize)
0

julia> print(mask[1:Sys.CPU_THREADS])
Int8[0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
```
(cc @vchuravy, @gbaraldi)

Would be great to get this into 1.11 (despite feature freeze) because
otherwise we won't be able to pin GC threads until 1.12 (likely not
until the end of the year).

Closes #53073

---------

Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
(cherry picked from commit 065aeb6)
lazarusA pushed a commit to lazarusA/julia that referenced this issue Jul 12, 2024
@carstenbauer @vchuravy
@DilumAluthge @lazarusA
Add jl_getaffinity and jl_setaffinity (JuliaLang#53402)
edbb9fa 
This PR adds two functions `jl_getaffinity` and `jl_setaffinity` to the
runtime, which are slim wrappers around `uv_thread_getaffinity` and
`uv_thread_setaffinity` and can be used to set the affinity of Julia
threads.

This will
* simplify thread pinning (ThreadPinning.jl currently pins threads by
spawning tasks that run the necessary ccalls) and
* enable users to also pin GC threads (or, more generally, all Julia
threads).

**Example:**
```julia
bauerc@n2lcn0146 julia git:(cb/affinity)  
➜ ./julia -q --startup-file=no --threads 2,3 --gcthreads 4,1
julia> cpumasksize = @CCall uv_cpumask_size()::Cint
1024

julia> mask = zeros(Cchar, cpumasksize);

julia> jl_getaffinity(tid, mask, cpumasksize) = ccall(:jl_getaffinity, Int32, (Int16, Ptr{Cchar}, Int32), tid, mask, cpumasksize)
jl_getaffinity (generic function with 1 method)

julia> jl_getaffinity(1, mask, cpumasksize)
0

julia> print(mask[1:Sys.CPU_THREADS])
Int8[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

julia> mask[1] = 0;

julia> jl_setaffinity(tid, mask, cpumasksize) = ccall(:jl_setaffinity, Int32, (Int16, Ptr{Cchar}, Int32), tid, mask, cpumasksize)
jl_setaffinity (generic function with 1 method)

julia> jl_setaffinity(1, mask, cpumasksize)
0

julia> fill!(mask, 0);

julia> jl_getaffinity(1, mask, cpumasksize)
0

julia> print(mask[1:Sys.CPU_THREADS])
Int8[0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
```
(cc @vchuravy, @gbaraldi)

Would be great to get this into 1.11 (despite feature freeze) because
otherwise we won't be able to pin GC threads until 1.12 (likely not
until the end of the year).

Closes JuliaLang#53073

---------

Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@carstenbauer carstenbauer

Labels
GC Garbage collector multithreading Base.Threads and related functionality
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add jl_getaffinity and jl_setaffinity carstenbauer/julia
4 participants
@vchuravy @carstenbauer @giordano @gbaraldi