Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to use sockets instead of numa_domains with pika's resource partitioner #1180

Merged
merged 18 commits into from
Sep 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions ci/.gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ include:
- local: 'ci/cpu/clang16_release.yml'
- local: 'ci/cpu/gcc11_codecov.yml'
- local: 'ci/cpu/gcc11_release.yml'
- local: 'ci/cpu/gcc11_release_stdexec.yml'
- local: 'ci/cpu/gcc11_debug_stdexec.yml'
# https://github.com/eth-cscs/DLA-Future/issues/1184
# - local: 'ci/cpu/gcc11_release_stdexec.yml'
# - local: 'ci/cpu/gcc11_debug_stdexec.yml'
- local: 'ci/cpu/gcc12_release_cxx20.yml'
- local: 'ci/cuda/gcc11_release.yml'
- local: 'ci/cuda/gcc11_release_scalapack.yml'
Expand Down
2 changes: 1 addition & 1 deletion ci/common-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ stages:
reports:
dotenv: build.env
variables:
SPACK_SHA: 1184de8352281ce1b34a8f3f6b54d7fadc2b216a
SPACK_SHA: 0905edf592752742eb4ddab3a528d3aee8f92930
SPACK_DLAF_REPO: ./spack
DOCKER_BUILD_ARGS: '[
"BASE_IMAGE",
Expand Down
2 changes: 1 addition & 1 deletion ci/docker/debug-cpu-stdexec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,5 @@ spack:
- 'malloc=system'
stdexec:
require:
- '@git.nvhpc-23.09.rc4=main'
- '@git.8bc7c7f06fe39831dea6852407ebe7f6be8fa9fd=main'
- 'build_type=Debug'
1 change: 1 addition & 0 deletions ci/docker/debug-cuda-scalapack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,5 +35,6 @@ spack:
- '+fortran'
pika:
require:
- '@0.19.1'
- 'build_type=Debug'
- 'malloc=system'
2 changes: 1 addition & 1 deletion ci/docker/release-cpu-stdexec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,4 @@ spack:
- '+stdexec'
stdexec:
require:
- '@git.48c52df0f81c6151eecf4f39fa5eed2dc0216204=main'
- '@git.8bc7c7f06fe39831dea6852407ebe7f6be8fa9fd=main'
2 changes: 1 addition & 1 deletion ci/docker/release-rocm533-stdexec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ spack:
- '+stdexec'
stdexec:
require:
- '@git.48c52df0f81c6151eecf4f39fa5eed2dc0216204=main'
- '@git.8bc7c7f06fe39831dea6852407ebe7f6be8fa9fd=main'
blas:
require:: openblas
lapack:
Expand Down
13 changes: 10 additions & 3 deletions include/dlaf/tune.h
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,11 @@

#include <cstddef>
#include <cstdint>
#include <exception>
#include <iosfwd>
#include <iostream>

#include <pika/init.hpp>
#include <pika/runtime.hpp>

#include <dlaf/types.h>
Expand Down Expand Up @@ -97,10 +100,14 @@ struct TuneParameters {
// Some parameters require the pika runtime to be initialized since they depend on the number of
// threads used by the runtime. We initialize them separately in the constructor after checking that
// pika is initialized.
#if PIKA_VERSION_FULL >= 0x001600 // >= 0.22.0
if (!pika::is_runtime_initialized()) {
std::cerr
<< "[ERROR] Trying to initialize DLA-Future tune parameters but the pika runtime is not initialized. Make sure pika is initialized first.\n";
std::terminate();
}
#endif

// TODO: Check upfront whether pika has been initialized to be able to give a better error message.
// If pika has not been initialized getting the default thread pool will either trigger an assertion
// or a segfault.
const auto default_pool_thread_count =
pika::resource::get_thread_pool("default").get_os_thread_count();
red2band_panel_nworkers = std::max<std::size_t>(1, default_pool_thread_count / 2);
Expand Down
1 change: 1 addition & 0 deletions spack/packages/dla-future/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ class DlaFuture(CMakePackage, CudaPackage, ROCmPackage):
depends_on("pika@0.17:", when="@0.2.1")
depends_on("pika@0.18:", when="@0.3")
depends_on("pika@0.19.1:", when="@0.4.0:")
conflicts("^pika@0.28:", when="@:0.6")
depends_on("pika-algorithms@0.1:", when="@:0.2")
depends_on("pika +mpi")
depends_on("pika +cuda", when="+cuda")
Expand Down
4 changes: 4 additions & 0 deletions src/init.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,10 @@ void initResourcePartitionerHandler(pika::resource::partitioner& rp,
// Create a thread pool with a single core that we will use for all
// communication related tasks
rp.create_thread_pool("mpi", pika::resource::scheduling_policy::static_priority, mode);
#if PIKA_VERSION_FULL >= 0x001C00 // >= 0.28.0
rp.add_resource(rp.sockets()[0].cores()[0].pus()[0], "mpi");
#else
rp.add_resource(rp.numa_domains()[0].cores()[0].pus()[0], "mpi");
#endif
}
}
Loading