Skip to content

Commit

Permalink
Set maximum default threads
Browse files Browse the repository at this point in the history
Set a limit of how many threads fd will use by default. On hosts that
have a large number of cores, using additional threads has diminishing
returns, and having large numbers of threads increases the setup cost.
Thus we don't necessarily want to use the same number of threads as we
have cores.

Fixes: sharkdp#1203
  • Loading branch information
tmccombs committed Oct 26, 2023
1 parent 1d57b3a commit 5ee6365
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 1 deletion.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,14 @@
- Breaking: `.git/` is now ignored by default when using `--hidden` / `-H`, use `--no-ignore` / `-I` or
`--no-ignore-vcs` to override, see #1387 and #1396 (@skoriop)


## Bugfixes

## Changes

- The default number of threads is now constrained to be at most 16. This should improve startup time on
systems with many CPU cores. (#1203)

## Other

# v8.7.1
Expand Down
15 changes: 14 additions & 1 deletion src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -719,7 +719,20 @@ fn default_num_threads() -> NonZeroUsize {
// Unfortunately, we can't do `NonZeroUsize::new(1).unwrap()`
// in a const context.
const FALLBACK_PARALLELISM: NonZeroUsize = NonZeroUsize::MIN;
std::thread::available_parallelism().unwrap_or(FALLBACK_PARALLELISM)
// As the number of threads increases, the startup time suffers from
// initializing the threads, and we get diminishing returns from additional
// parallelism. So set a maximum number of threads to use by default.
//
// This value is based on some empirical observations, but the ideal value
// probably depends on the exact hardware in use.
//
// Safety: The literal "20" is known not to be zero.
const MAX_DEFAULT_THREADS: NonZeroUsize = unsafe { NonZeroUsize::new_unchecked(20) };

std::cmp::min(
std::thread::available_parallelism().unwrap_or(FALLBACK_PARALLELISM),
MAX_DEFAULT_THREADS,
)
}

#[derive(Copy, Clone, PartialEq, Eq, ValueEnum)]
Expand Down

0 comments on commit 5ee6365

Please sign in to comment.