set default blas num threads to Sys.CPU_THREADS / 2 in absence of OPENBLAS_NUM_THREADS

[Moelf]

fix #44449

[Moelf]

Maybe the right thing is to let you set any number explicitly as JULIA_NUM_THREADS, but if you don't specify then cap at 8 by default like we did before?

is OPENBLAS_NUM_THREADS still the ENV variable we want to use moving forward?

[ViralBShah]

This needs to not be tied to JULIA_NUM_THREADS, because even in single-threaded Julia, you may still want multi-threaded BLAS. We also don't want the cap to 8.

How about we set OPENBLAS_NUM_THREADS to Threads.nthreads() and if the number of julia threads is 1, then use (Sys.CPU_THREADS/2) for BLAS threads to match the physical CPU cores. Of course if the user has provided the openblas threading environment variable, we respect that.

[Moelf]

yeah this is not tied to JULIA_NUM_THREADS. I'm just unsure what @JeffBezanson 's comment was saying, I think he was saying do something like what we do with JULIA_NUM_THREADS, not actually using it for BLAS number of threads

[ViralBShah]

The problem with Julia threading defaults (-t auto) is that it also picks the hyperthreaded and not the physical cores.

[Moelf]

yes. Well, this PR only sets OPENBLAS_NUM_THREADS to 8 when it's not set by the user.

I always wanted -t auto to set # of threads to # of physical cores but alas that's a separate issue.

[ViralBShah]

Right, but that setting is almost always wrong - like on my laptop where I have only 2 physical and 4 hyperthreaded cores, it will pick 8. And on a big machine where we have more CPUs, it will pick too few. That's why my preference is just Julia's reported cores divided by 2.

Would be great to have @chriselrod and @staticfloat chime in.

[Moelf]

reusing:

julia/stdlib/LinearAlgebra/src/blas.jl

Line 109 in a5438f9

max(1, Sys.CPU_THREADS ÷ 2),

[ViralBShah]

I think we should we do this directly through the BLAS.set_num_threads() API and that LinearAlgebra.__init__() is the right place to do it.

[Moelf]

like this?

[chriselrod]

I think that's fine.

But with Apple M1 and Intel Alder Lake & future generations of both, we're going to have more and more cases where this is suboptimal.

We should use Julia's existing feature detection for this, but that's not something I've looked at yet.

[ViralBShah]

There needs to be (or already exists) a separate issue for doing feature detection in Julia for all packages to use.

[chriselrod]

There needs to be (or already exists) a separate issue for doing feature detection in Julia for all packages to use.

Yeah, no reason not to do this until then.

[ViralBShah]

I'm merging this so that it gives us time to experiment and see how it is working out.