Meta-ticket: support macOS 11 Big Sur

[slel]

This is to support macOS 11 Big Sur.

• switch the default mp library to gmp #30752 gmp instead of mpir
• GH Actions: Update macos test environments #30929 GH Actions: Update macos test environments
• Allow using arb, flint and ntl from Homebrew #30745 homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local
• Upgrade: wheel 0.36.2 #31050 upgrade wheel
• MACOSX_DEPLOYMENT_TARGET problems - https://groups.google.com/g/sage-devel/c/Sj3YBVJoHQM/m/Zt-Ih_hJAwAJ
• MR50: Upgrade cffi to 1.14.4 #31128 MR50: Upgrade cffi to 1.14.4
• Update setuptools and setuptools_scm #31134 upgrade setuptools and setuptools_scm
• Upgrade Python to 3.9.1, pip to 20.3.3 #30589 Upgrade Python to 3.9.1, pip to 20.3.3
• numpy: Fix for macOS 11 (Big Sur) #31166 numpy: Fix for macOS 11 (Big Sur)
• Fix scipy for MacOS 11 (Big Sur) #31183 scipy: fix for macOS 11 (Big Sur)
• macos-11.0: scipy build fails #31326 homebrew-macos-11.0-xcode-minimal: scipy build fails

Related:

• Meta-ticket: Support Xcode 12 #30494: Meta-ticket: Support Xcode 12
• Meta-ticket: Port to Apple silicon #30592: Meta-ticket: Port to Apple silicon

References:

• https://bugs.python.org/issue41116
• https://groups.google.com/g/sage-devel/c/5lBTr4gZuic/m/RkisHYLWCgAJ

Depends on #30494

CC: @dimpase @jhpalmieri @mkoeppe @slel @vbraun @williamstein

Component: porting

Keywords: macOS

Issue created by migration from https://trac.sagemath.org/ticket/30651

[wdjoyner]

comment:3

I'm trying to compile sage-9.2.rc0 on big sur with xcode 12.0.1.

I ran sage -f openblas. Here's the tail of that:

(see the openblas attachment).

After trying lots of things such as

$ ./configure \
  --enable-universalsdk=$(xcodebuild -version -sdk macosx Path) \
  --with-universal-archs=intel-64
$ source /Users/wdj/sagefiles/sage-9.2.rc0/.homebrew-build-env
$ ./config.status --recheck && ./config.status
$ ./sage -f gfortran

the compilation of sage -f gfortran stopped with an error compiling R.

However, Sage did start and crash with this error:

wdj@jeeves sage-9.2.rc0 % ./sage
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 9.2.rc0, Release Date: 2020-10-05                 │
│ Using Python 3.8.5. Type "help()" for help.                        │
└────────────────────────────────────────────────────────────────────┘
---------------------------------------------------------------------------
ModuleNotFoundErrorPython 3.8.5:
/Users/wdj/sagefiles/sage-9.2.rc0/local/bin/python3
Thu Oct  8 13:27:16 2020
A problem occurred executing Python code.  Here is the sequence of function
calls leading up to the error, with the most recent (innermost) call last.

(for the rest, see the error-in-start attachment).

It looks like Sage's Python doesn't work, right? Wrong:

wdj@jeeves sage-9.2.rc0 % ./sage -python3
Python 3.8.5 (default, Oct  7 2020, 13:59:53)
[Clang 12.0.0 (clang-1200.0.32.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 2+3
5

In case it's any use, here is what homebrew claims it has installed:

wdj@jeeves sage-9.2.rc0 % brew list --versions
autoconf 2.69
automake 1.16.1_1
bdw-gc 8.0.4
boost 1.71.0
cmake 3.15.5
fontconfig 2.13.1
freetype 2.10.1
gcc 9.2.0_1
gd 2.2.5
gettext 0.20.1 0.21
gmp 6.2.0 6.1.2_2
gnutls 3.6.15
gpatch 2.7.6
guile 3.0.4
icu4c 64.2
isl 0.22.1 0.21
jpeg 9c
libde265 1.0.3
libdvdcss 1.4.2
libevent 2.1.12
libffi 3.3 3.2.1
libidn2 2.3.0 2.2.0_1
libmpc 1.1.0 1.2.0
libomp 9.0.0
libpng 1.6.37
libtasn1 4.14 4.16.0
libtiff 4.0.10_1
libtool 2.4.6_1 2.4.6_2
libunistring 0.9.10
libyaml 0.2.5
mpfi 1.5.3
mpfr 4.1.0 4.0.2
nettle 3.4.1 3.6
ntl 11.4.1
openssl 1.0.2t
openssl@1.1 1.1.1g 1.1.1d
p11-kit 0.23.21 0.23.18.1
pari 2.11.4
pkg-config 0.29.2_3 0.29.2
readline 8.0.4
ruby 2.7.1_2
sqlite 3.33.0
unbound 1.11.0
webp 1.0.3
xz 5.2.4

[jhpalmieri]

comment:4

I've actually gotten this to build for me without patching Sage in any way, just building 9.2.rc0. Homebrew setup:

% brew list
autoconf    gd           haskell-stack  libpng        ncurses      r
automake    gdbm         icu4c          libtasn1      nettle       re2c
bdw-gc      gettext      ilmbase        libtiff       ninja        readline
bison       ghc          intltool       libtool       openblas     sphinx-doc
boost       ghostscript  isl            libunistring  openexr      sqlite
bzip2       git          jansson        libxml2       openssl@1.1  suite-sparse
cmake       glib         jpeg           meson         p11-kit      tbb
curl        glpk         libatomic_ops  metis         pandoc       texinfo
doxygen     gmp          libevent       mpc           pcre         unbound
fontconfig  gnutls       libffi         mpfi          pcre2        webp
freetype    gpatch       libidn2        mpfr          pkg-config   xz
gcc         gsl          libmpc         mpir          ppl          yasm
gcc@9       guile        libmpdclient   nauty         python@3.8   zeromq

Then I used the command

$ ./configure \
  --with-system-readline=no \
  --with-system-zlib=no \
  --with-system-gfortran=no \
  --with-system-openblas=no \
  --with-system-bzip2=no

after which make worked but make ptestlong
showed a bunch of failures, all like this:

Expected nothing
Got:
    ld: warning: dylib (/usr/local/lib/libgmp.dylib) was built for newer macOS version (10.15) than being linked (10.9)
    ld: warning: dylib (/usr/local/lib/libmpfr.dylib) was built for newer macOS version (10.15) than being linked (10.9)
    ld: warning: dylib (/usr/local/lib/libgmpxx.dylib) was built for newer macOS version (10.15) than being linked (10.9)

Maybe because MACOSX_DEPLOYMENT_TARGET is being set? I could try building Sage's mpfr and gmp. But in any case, I got Sage to build, unfortunately by not using a bunch of the system versions of things.

[jhpalmieri]

comment:5

Replying to @jhpalmieri:

Maybe because MACOSX_DEPLOYMENT_TARGET is being set? I could try building Sage's mpfr and gmp.

I could try that, but it won't work because our mpir is broken when using Xcode 12, with the usual error, and the latest release of mpir (git clone git://github.com/wbhart/mpir.git mpir) doesn't help.

[dimpase]

comment:6

mpir is basically not developed any more (for the time being at least), so we should not use it.

What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.

[dimpase]

comment:7

Replying to @wdjoyner:

I'm trying to compile sage-9.2.rc0 on big sur with xcode 12.0.1.

I ran sage -f openblas. Here's the tail of that:

[...]

David, could you please replace this 10-screen comment by an attachement?

[wdjoyner]

Attachment: openblas-tail.txt

openblas error

[wdjoyner]

Attachment: error-in-start.txt

error in start

[jhpalmieri]

comment:8

Replying to @dimpase:

mpir is basically not developed any more (for the time being at least), so we should not use it.

It's a standard package, so I'm confused about what the alternatives are.

Anyway, I hacked the spkg-install.in script for mpir, as in #30729, and I got it to build. That eliminated those doctest failures. I'm left with two:

sage -t --long --random-seed=0 src/sage/misc/gperftools.py
**********************************************************************
File "src/sage/misc/gperftools.py", line 118, in sage.misc.gperftools.Profiler._libc
Failed example:
    Profiler()._libc()
Exception raised:
    Traceback (most recent call last):
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.gperftools.Profiler._libc[1]>", line 1, in <module>
        Profiler()._libc()
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/misc/gperftools.py", line 129, in _libc
        raise ImportError('failed to open libc')
    ImportError: failed to open libc

and

sage -t --long --random-seed=0 src/sage/tests/parigp.py
    Timed out

What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.

I haven't changed Sage, so it's whatever gets set in sage-env: 10.9, I think.

[slel]

comment:9

Replying to @jhpalmieri:

Replying to @dimpase:

mpir is basically not developed any more (for the time being at least), so we should not use it.

It's a standard package, so I'm confused about what the alternatives are.

One can choose between gmp and mpir. Both are standard, but only one is needed.

Use the configure flag --with-mp=gmp or with-mp=mpir to choose.

Likewise, one can choose between ATLAS and OpenBLAS, and both are standard.

Use --with-blas=openblas or --with-blas=atlas for that.

In that case #30350 proposes to remove ATLAS.

[jhpalmieri]

comment:10

Sorry, I led us on a bit of a wild goose chase for the past few comments. Back in comment:4, I said the right thing:

I could try building Sage's mpfr and gmp.

But then (maybe I was tired or distracted), I started building Sage's mpir (and also Sage's gmp, which was appropriate). There is no reason to care about mpir: just use --with-system-gmp=no and Sage will build its own gmp and mpfr.

[mkoeppe]

comment:11

Replying to @jhpalmieri:

Replying to @dimpase:

What is the value of MACOSX_DEPLOYMENT_TARGET used in this build? I would not be surprised if there is an incompatibility barrier between macOS 10 and macOS 11.

I haven't changed Sage, so it's whatever gets set in sage-env: 10.9, I think.

Just a quick remark - MACOSX_DEPLOYMENT_TARGET only gets set in builds configured --without-system-python. (See #30724, which makes this code in sage-env cleaner.)

[jhpalmieri]

comment:12

Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own, so MACOSX_DEPLOYMENT_TARGET is relevant. (Also, Sage's Python can't find various system libraries to build the corresponding modules, which is why I'm passing the flags --with-system-zlib=no --with-system-bzip2=no to configure.)

[mkoeppe]

comment:13

Replying to @jhpalmieri:

Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own

Is this because of dependencies of the python3 package? In this case, #30559 may be relevant

[wdjoyner]

comment:14

Replying to @jhpalmieri:

I've actually gotten this to build for me without patching Sage in any way, just building 9.2.rc0.

...

Then I used the command

./configure --with-system-readline=no --with-system-zlib=no --with-system-gfortran=no --with-system-openblas=no --with-system-bzip2=no

make worked.

This configure command followed by make gave me a functioning sage, so
thank you!

R didn't compile but I don't use it.

[jhpalmieri]

comment:15

Replying to @mkoeppe:

Replying to @jhpalmieri:

Right. Since for me at least, Sage doesn't successfully detect the presence of the system Python on Big Sur, it always builds its own

Is this because of dependencies of the python3 package?

Yes, sqlite is not successfully detected. I'll try turning that dependency off.

[jhpalmieri]

comment:16

Sage built using the system Python after modifying the dependencies to avoid sqlite. I used

./configure --with-system-gfortran=no --with-system-openblas=no --with-system-gmp=no

Unfortunately the documentation won't build and many doctests fail , in both cases with a bus error. For example:

File "src/sage/algebras/quatalg/quaternion_algebra.py", line 107, in sage.algebras.quatalg.quaternion_algebra.QuaternionAlgebraFactory
Failed example:
    QuaternionAlgebra(sqrt(-1), sqrt(-3))
Exception raised:
    Traceback (most recent call last):
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.algebras.quatalg.quaternion_algebra.QuaternionAlgebraFactory[4]>", line 1, in <module>
        QuaternionAlgebra(sqrt(-Integer(1)), sqrt(-Integer(3)))
      File "sage/structure/factory.pyx", line 367, in sage.structure.factory.UniqueFactory.__call__ (build/cythonized/sage/structure/factory.c:2181)
        key, kwds = self.create_key_and_extra_args(*args, **kwds)
      File "sage/structure/factory.pyx", line 471, in sage.structure.factory.UniqueFactory.create_key_and_extra_args (build/cythonized/sage/structure/factory.c:3402)
        return self.create_key(*args, **kwds), {}
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/algebras/quatalg/quaternion_algebra.py", line 246, in create_key
        if a == 0 or b == 0:
      File "sage/symbolic/expression.pyx", line 3182, in sage.symbolic.expression.Expression.__nonzero__ (build/cythonized/sage/symbolic/expression.cpp:20543)
        res = self.test_relation()
      File "sage/symbolic/expression.pyx", line 3296, in sage.symbolic.expression.Expression.test_relation (build/cythonized/sage/symbolic/expression.cpp:21404)
        if self.lhs().is_algebraic() and self.rhs().is_algebraic():
      File "sage/symbolic/expression.pyx", line 2257, in sage.symbolic.expression.Expression.is_algebraic (build/cythonized/sage/symbolic/expression.cpp:15979)
        ex = sage.rings.all.QQbar(self)
      File "sage/structure/parent.pyx", line 900, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9336)
        return mor._call_(x)
      File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4515)
        return C._element_constructor(x)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/rings/qqbar.py", line 1496, in _element_constructor_
        return x._algebraic_(QQbar)
      File "sage/symbolic/expression.pyx", line 1754, in sage.symbolic.expression.Expression._algebraic_ (build/cythonized/sage/symbolic/expression.cpp:12658)
        return algebraic(self, field)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 1227, in algebraic
        return AlgebraicConverter(field)(ex)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 218, in __call__
        return self.arithmetic(ex, operator)
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/symbolic/expression_conversions.py", line 1061, in arithmetic
        return self.field(base**expt)
      File "sage/structure/element.pyx", line 2057, in sage.structure.element.Element.__pow__ (build/cythonized/sage/structure/element.c:14456)
        return coercion_model.bin_op(left, right, pow)
      File "sage/structure/coerce.pyx", line 1196, in sage.structure.coerce.CoercionModel.bin_op (build/cythonized/sage/structure/coerce.c:10205)
        return (<Action>action)._act_(y, x)
      File "sage/categories/action.pyx", line 173, in sage.categories.action.Action._act_ (build/cythonized/sage/categories/action.c:3900)
        cpdef _act_(self, g, x):
      File "/Users/palmieri/Sage/sage-9.2.rc0/local/lib/python3.8/site-packages/sage/rings/qqbar.py", line 6054, in _act_
        target_arg.sin() * target_abs)
      File "sage/rings/real_mpfi.pyx", line 4557, in sage.rings.real_mpfi.RealIntervalFieldElement.sin (build/cythonized/sage/rings/real_mpfi.c:26988)
        sig_on()
    cysignals.signals.SignalError: Bus error

[dimpase]

comment:17

Replying to @jhpalmieri:

Sorry, I led us on a bit of a wild goose chase for the past few comments. Back in comment:4, I said the right thing:

I could try building Sage's mpfr and gmp.

But then (maybe I was tired or distracted), I started building Sage's mpir (and also Sage's gmp, which was appropriate). There is no reason to care about mpir: just use --with-system-gmp=no and Sage will build its own gmp and mpfr.

They cannot be, and are not, "installed together" - mpir uses the "gmp compatibility" mode where it installs gmp-compatible libs etc. You need to run ./configure with --with-mp=gmpto select gmp as the package to use formp, (= multiple precision), as the default for mpismpir`.

Default mp should be changed to gmp, I think, as mpir is getting bitrotten more and more.
See #30752 (where a lie about mpir/gmp told by ./configure -h is uncovered, as well)

[jhpalmieri]

comment:22

I am having problems building scipy. This is after installing #31050, #30589, upgrades to setuptools and setuptools_scm, plus patches (related to parsing the OS X version number) to cffi and numpy. Then scipy fails with

  /usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so
  gfortran: warning: couldn't understand version 11
  ld: library not found for -lgcc_s.10.4
  collect2: error: ld returned 1 exit status
  error: Command "/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -bundle -undefined dynamic_lookup -L/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib -Wl,-rpath,/Users/palmieri/Desktop/Sage/sage_builds/TESTING/TEMP/sage-9.3.beta4/local/lib build/temp.macosx-11-x86_64-3.9/scipy/integrate/_quadpackmodule.o -L/usr/local/Cellar/openblas/0.3.13/lib -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/Cellar/gcc/10.2.0/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0/../../.. -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -Lbuild/temp.macosx-11-x86_64-3.9 -lquadpack -lmach -lopenblas -lopenblas -lgfortran -o build/lib.macosx-11-x86_64-3.9/scipy/integrate/_quadpack.cpython-39-darwin.so" failed with exit status 1
  Building wheel for scipy (PEP 517): finished with status 'error'
  ERROR: Failed building wheel for scipy

[isuruf]

comment:23

Try doing export MACOSX_DEPLOYMENT_TARGET=10.16 to fix the issue until gfortran understands 11

[jhpalmieri]

comment:48

Replying to @jhpalmieri:

After running make pillow, I tried make numpy and got an unexpected error:

Package 'numpy' is currently not installed
Uninstalling 'numpy' with legacy uninstaller
Traceback (most recent call last):
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/var/tmp/sage/build/numpy-1.19.4/src/../lapack_conf.py", line 7, in <module>
    pc_blas   = pkgconfig.parse('cblas blas')
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/pkgconfig/pkgconfig.py", line 248, in parse
    _raise_if_not_exists(package)
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.beta5/local/lib/python3.9/site-packages/pkgconfig/pkgconfig.py", line 103, in _raise_if_not_exists
    raise PackageNotFoundError(package)
pkgconfig.pkgconfig.PackageNotFoundError: cblas not found

Missing dependency? After running just make, it has now reached numpy and gotten past this problem.

I opened #31135 for this.

[jhpalmieri]

comment:50

I opened an issue with numpy for a fix as was done with cffi.

[jhpalmieri]

comment:51

Replying to @jhpalmieri:

I opened an issue with numpy for a fix as was done with cffi.

Should be fixed in numpy 1.20, which is in release-candidate stage.

[zlscherr]

comment:52

I found a patch for gfortran on Intel Macs at

gcc-mirror/gcc@96de87b

I'll try to open a pull request tomorrow with Homebrew to see if they can patch GCC. That should hopefully fix all Fortran issues until GCC 11 comes out.

[zlscherr]

comment:53

I noticed this morning that homebrew had updated their gcc. With the newest gcc formula you still need

MACOSX_DEPLOYMENT_TARGET=11.0 make scipy

in order for scipy to build, but now if you do

MACOSX_DEPLOYMENT_TARGET=11.0 ./sage -t src/sage/misc/inline_fortran.py

then the tests all pass.

[mkoeppe]

comment:54

Replying to @jhpalmieri:

Replying to @jhpalmieri:

I opened an issue with numpy for a fix as was done with cffi.

Should be fixed in numpy 1.20, which is in release-candidate stage.

We will have to backport the fix because numpy 1.20 has dropped support for Python 3.6, which I think we want to continue supporting in Sage 9.3

[mkoeppe]

comment:55

Opened #31166 for the numpy fix

[zlscherr]

comment:57

I was just curious if anyone was planning to open a ticket for scipy. I don't know enough about the sage build system, but it's possible to check if the user is on Big Sur by doing

sw_vers | grep 11

and then based off of the result you could set MACOSX_DEPLOYMENT_TARGET=11.0

[jhpalmieri]

comment:58

During the Sage build process, Sage already defines an environment variable MACOSX_VERSION by

MACOSX_VERSION=`uname -r | awk -F. '{print $1}'`

This should be '20' for Big Sur.

I'll open a scipy ticket.

[jhpalmieri]

comment:60

Replying to @jhpalmieri:

During the Sage build process, Sage already defines an environment variable MACOSX_VERSION by

MACOSX_VERSION=`uname -r | awk -F. '{print $1}'`

This should be '20' for Big Sur.

Actually, this is not correct: this variable is not universally defined anymore, so Sage packages that try to use it end up printing warnings. For example, r:

./spkg-install: line 90: [: -ge: unary operator expected
./spkg-install: line 94: [: -ge: unary operator expected

See #31186.

[vbraun]

comment:63

Fails on a the as-vanilla-as-possible buildbot machine while building scipy (I take it thats #31326):

    Building modules...
    	Building module "mvn"...
    		Constructing wrapper function "mvnun"...
    		  value,inform = mvnun(lower,upper,means,covar,[maxpts,abseps,releps])
    		Constructing wrapper function "mvnun_weighted"...
    		  value,inform = mvnun_weighted(lower,upper,means,weights,covar,[maxpts,abseps,releps])
    		Constructing wrapper function "mvndst"...
    		  error,value,inform = mvndst(lower,upper,infin,correl,[maxpts,abseps,releps])
    		Constructing COMMON block support for "dkblck"...
    		  ivls
    	Wrote C/API module "mvn" to file "build/src.macosx-10.9-x86_64-3.9/scipy/stats/mvnmodule.c"
    	Fortran 77 wrappers are saved to "build/src.macosx-10.9-x86_64-3.9/scipy/stats/mvn-f2pywrappers.f"
      adding 'build/src.macosx-10.9-x86_64-3.9/build/src.macosx-10.9-x86_64-3.9/scipy/stats/fortranobject.c' to sources.
      adding 'build/src.macosx-10.9-x86_64-3.9/build/src.macosx-10.9-x86_64-3.9/scipy/stats' to include_dirs.
      adding 'build/src.macosx-10.9-x86_64-3.9/scipy/stats/mvn-f2pywrappers.f' to sources.
    building extension "scipy.ndimage._nd_image" sources
    building extension "scipy.ndimage._ni_label" sources
    building extension "scipy.ndimage._ctest" sources
    building extension "scipy.ndimage._ctest_oldapi" sources
    building extension "scipy.ndimage._cytest" sources
    building extension "scipy._lib._ccallback_c" sources
    building extension "scipy._lib._test_ccallback" sources
    building extension "scipy._lib._fpumode" sources
    building extension "scipy._lib.messagestream" sources
    creating build/src.macosx-10.9-x86_64-3.9/scipy/_lib
    Traceback (most recent call last):
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
        main()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 161, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 253, in run_setup
        super(_BuildMetaLegacyBackend,
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 145, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 583, in <module>
        setup_package()
      File "setup.py", line 579, in setup_package
        setup(**metadata)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/core.py", line 169, in setup
        return old_setup(**new_attr)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/setuptools/command/dist_info.py", line 31, in run
        egg_info.run()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/egg_info.py", line 24, in run
        self.run_command("build_src")
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/build_src.py", line 144, in run
        self.build_sources()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/build_src.py", line 161, in build_sources
        self.build_extension_sources(ext)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/build_src.py", line 318, in build_extension_sources
        sources = self.generate_sources(sources, ext)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/build_src.py", line 378, in generate_sources
        source = func(extension, build_dir)
      File "scipy/_lib/setup.py", line 30, in get_messagestream_config
        if config_cmd.check_func('open_memstream', decl=True, call=True):
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/config.py", line 321, in check_func
        self._check_compiler()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/command/config.py", line 80, in _check_compiler
        self.fcompiler = new_fcompiler(compiler=self.fcompiler,
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 880, in new_fcompiler
        compiler = get_default_fcompiler(plat, requiref90=requiref90,
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 851, in get_default_fcompiler
        compiler_type =  _find_existing_fcompiler(matching_compiler_types,
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 802, in _find_existing_fcompiler
        c.customize(dist)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 507, in customize
        get_flags('opt', oflags)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 498, in get_flags
        flags.extend(getattr(self.flag_vars, tag))
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/environment.py", line 37, in __getattr__
        return self._get_var(name, conf_desc)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/environment.py", line 53, in _get_var
        var = self._hook_handler(name, hook)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 705, in _environment_hook
        return hook()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/gnu.py", line 229, in get_flags_opt
        v = self.get_version()
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 426, in get_version
        version = CCompiler.get_version(self, force=force, ok_status=ok_status)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/ccompiler.py", line 90, in <lambda>
        m = lambda self, *args, **kw: func(self, *args, **kw)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/ccompiler.py", line 657, in CCompiler_get_version
        version = matcher(output)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/gnu.py", line 278, in version_match
        v = self.gnu_version_match(version_string)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/numpy/distutils/fcompiler/gnu.py", line 80, in gnu_version_match
        raise ValueError(err + version_string)
    ValueError: A valid Fortran version was not found in this string:
    9.2.0
    Preparing wheel metadata: finished with status 'error'

[mkoeppe]

comment:64

You know the drill... full config.log, full log please

[dimpase]

comment:65

numpy/numpy#11822 ?

[kcrisman]

comment:66

For what it's worth, I am due for an upgrade and a test run of a Time Machine copy of my current (old) Mac onto a new Big Sur machine on an M1 chip apparently works - I'm not sure how, presumably using Rosetta somehow. Do we have any other data points about upgrading the Mac but leaving a working Sage install in place? (I guess upgrading that install would fall squarely under this ticket, though.)

[mkoeppe]

comment:68

I think we can close the ticket now - this task is done

[jhpalmieri]

comment:69

Yes, I agree.