Set maximum default threads

Set a limit of how many threads fd will use by default. On hosts that
have a large number of cores, using additional threads has diminishing
returns, and having large numbers of threads increases the setup cost.
Thus we don't necessarily want to use the same number of threads as we
have cores.

Fixes: sharkdp#1203

CHANGELOG.md

@@ -5,10 +5,14 @@
 - Breaking: `.git/` is now ignored by default when using `--hidden` / `-H`, use `--no-ignore` / `-I` or
   `--no-ignore-vcs` to override, see #1387 and #1396 (@skoriop)
 
+
 ## Bugfixes
 
 ## Changes
 
+- The default number of threads is now constrained to be at most 16. This should improve startup time on
+  systems with many CPU cores. (#1203)
+
 ## Other
 
 # v8.7.1

src/cli.rs

@@ -719,7 +719,20 @@ fn default_num_threads() -> NonZeroUsize {
     // Unfortunately, we can't do `NonZeroUsize::new(1).unwrap()`
     // in a const context.
     const FALLBACK_PARALLELISM: NonZeroUsize = NonZeroUsize::MIN;
-    std::thread::available_parallelism().unwrap_or(FALLBACK_PARALLELISM)
+    // As the number of threads increases, the startup time suffers from
+    // initializing the threads, and we get diminishing returns from additional
+    // parallelism. So set a maximum number of threads to use by default.
+    //
+    // This value is based on some empirical observations, but the ideal value
+    // probably depends on the exact hardware in use.
+    //
+    // Safety: The literal "20" is known not to be zero.
+    const MAX_DEFAULT_THREADS: NonZeroUsize = unsafe { NonZeroUsize::new_unchecked(20) };
+
+    std::cmp::min(
+        std::thread::available_parallelism().unwrap_or(FALLBACK_PARALLELISM),
+        MAX_DEFAULT_THREADS,
+    )
 }
 
 #[derive(Copy, Clone, PartialEq, Eq, ValueEnum)]