Meta-ticket: Support Xcode 12

[mkoeppe]

Xcode 12 was publicly released on 2020-09-16.

Sage does not build from source using Xcode 12.2 beta, released 2020.09.17, clang version 12.0.0 (clang-1200.0.32.4)

• https://groups.google.com/d/msg/sage-support/ZLgu-1pi6nc/W7Hqe7AgAgAJ
• https://github.com/mkoeppe/sage/runs/1059961873

Failing packages (blockers):

• gf2x gf2x build with Xcode 12 #30593
• ecm fix ecm on macOS Xcode 12 #30600
• symmetrica fix symmetrica for xcode 12 #30608 or Upgrade to symmetrica-3.0.1 #29061
• ecl ecl build with Xcode 12 #30594
• scipy fix scipy for XCode 12 #30604

Failing packages with replacements available via homebrew:

• openblas Fix openblas to build with Xcode 12 #30610
• readline Upgrade readline to 8.0 #30603
• gmp upgrade GMP to version 6.2.0 #30625
• mpir
• gsl (?)
• r fix R on macOS xcode 12 #30631

Failing optional packages:

• rubiks (see also Downgrade rubiks to optional #28022)
• gap_packages (Fix gap_packages for Xcode 12 #30729)
• polylib (Update isl + polylib + barvinok packages #33758)

Failing experimental packages:

• gdb, valgrind (to be removed in Reduce outdated gdb and valgrind packages to dummy script packages #30158)
• gap3, cocoalib, awali, libtheora, surf, scipoptsuite https://github.com/mkoeppe/sage/actions/runs/289319580

Tickets:

• GH Actions: Test also with Xcode 12 beta #30487 - GH Actions: Test also with Xcode 12 beta
• Upgrade: Python 3.8.5 #27754 - Upgrade: Python 3.8.5
• Downgrade rubiks to optional #28022 - Downgrade rubiks to optional
• Accept /usr/bin/python3 from XCode 12.3 on macOS 10.15 (Catalina) and 11 (Big Sur) #31227 - Accept /usr/bin/python3 from XCode 12.3 on macOS 10.15 (Catalina)

Follow-up:

• Meta-ticket: support macOS 11 Big Sur #30651 - Meta-ticket: support macOS 11

CC: @jhpalmieri @dimpase @slel @vbraun

Component: porting

Keywords: macOS, Xcode

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/30494

[jhpalmieri]

comment:4

For what it's worth, I was trying to build without setting MACOSX_DEPLOYMENT_TARGET at all: I removed those lines from src/bin/sage-env.

[mkoeppe]

comment:5

Note that these lines are only executed when using python3 installed from spkg, not system python

[dimpase]

comment:6

looks like this should wait till 9.3

[jhpalmieri]

comment:7

Sure, Big Sur is still in beta.

[jhpalmieri]

comment:9

Xcode 12 has been released, and it causes breakage with Catalina, so perhaps this should be a blocker for 9.2. (For example I now cannot build Sage: I upgraded a laptop to a beta release of Big Sur, and on my main machine Xcode updated itself to version 12 overnight.)

[mkoeppe]

comment:10

I agree. It would be good to figure out if the upstream maintainers of these packages are already aware of these issues.

[dimpase]

comment:11

Can we score an https://en.wikipedia.org/wiki/Developer_Transition_Kit_(2020) - and Apple ARM prototype for developers somehow?
It might be very useful for testing. Probably a Raspberry Pi with a 64-bit OS comes close, but not as much.

[mkoeppe]

comment:12

This is unrelated to this ticket.

[mkoeppe]

comment:13

I have opened #30592 (Meta-ticket: Port to Apple silicon) for that. Let's keep the present ticket focused please. No new hardware is required.

[jhpalmieri]

comment:14

It seems that when configuring gf2x and ecl, at least, something like -Wall is being used. I see this kind of thing in the config.log files for those:

conftest.c:4:3: error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
  exit(0);
  ^
conftest.c:4:3: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
1 error generated.

Is there some setting in Xcode's command-line tools that has changed, causing these errors?

[jhpalmieri]

comment:15

Similar with ecm:

configure:13286: clang -O2 -pedantic -fomit-frame-pointer -m64 -mtune=nehalem -march=nehalem  -c conftes1.c >&5 && clang -O2 -pedantic -fomit-frame-pointer -m64 -mtune=nehalem -march=nehalem -c conftes2.s >&5 && clang -O2 -pedantic -fomit-frame-pointer -m64 -mtune=nehalem -march=nehalem  conftes1.o conftes2.o >&5
conftes1.c:4:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
main () { underscore_test(); }
^
conftes1.c:4:11: error: implicit declaration of function 'underscore_test' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
main () { underscore_test(); }
          ^
1 warning and 1 error generated.

[emmanuelthome]

comment:16

Hi,

I just learned about this ticket. (me = upstream gf2x)

I don't have access to a box of this kind, so I can't test. (all I have is macOS 10.15.5 + XCode 11.6).

It looks like latest xcode doesn't like the tests that are generated by autotools, right ? If yes, does a autoreconf with fresh autotools fix the problem ?

E.

[jhpalmieri]

comment:17

The Sage build log for symmetrica contains:

hiccup.c:478:3: error: implicit declaration of function 'set_multiplier' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                set_multiplier(S_MO_K(S_L_S(a)));
                ^

[jhpalmieri]

comment:18

Replying to @emmanuelthome:

Hi,

I just learned about this ticket. (me = upstream gf2x)

I don't have access to a box of this kind, so I can't test. (all I have is macOS 10.15.5 + XCode 11.6).

I'm seeing this on OS X 10.15.6 after an upgrade to Xcode 12. You could try upgrading to Xcode 12, if you're willing to take that risk.

It looks like latest xcode doesn't like the tests that are generated by autotools, right ? If yes, does a autoreconf with fresh autotools fix the problem ?

I'm not an expert in this sort of thing, but I don't think this will work. autoreconf would help if Sage's top-level configure script were running into problems, but the problems are coming from Sage packages. Those are just tarballs, and running autoreconf once won't help with those. For what it's worth, I unpacked the gf2x tarball and ran autoreconf on it, but I got the same error.

(This is with autoreconf 2.69 installed via homebrew. I did brew upgrade first. I'm not sure how to get a "fresh autotools".)

[jhpalmieri]

Attachment: config.log

gf2x config.log

[jhpalmieri]

comment:19

Here is the config.log for gf2x, in case it helps.

[jhpalmieri]

comment:20

I'm happy to run some experiments, if you have suggestions.

[mkoeppe]

comment:21

Replying to @jhpalmieri:

It seems that when configuring gf2x and ecl, at least, something like -Wall is being used. I see this kind of thing in the config.log files for those:

conftest.c:4:3: error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
  exit(0);
  ^
conftest.c:4:3: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
1 error generated.

Is there some setting in Xcode's command-line tools that has changed, causing these errors?

I think errors like this are usually from code incorrectly assuming that some header pulls in some other header. Adding the correct includes should fix this.

[emmanuelthome]

comment:22

Replying to @jhpalmieri:

I don't have access to a box of this kind, so I can't test. (all I have is macOS 10.15.5 + XCode 11.6).

I'm seeing this on OS X 10.15.6 after an upgrade to Xcode 12. You could try upgrading to Xcode 12, if you're willing to take that risk.

Not light-heartedly for sure. It's a mac mini that has been living in an office of (very) mac-averse researchers for years and years, crawling under the ram demands of the latest macOS (and actually last it can support).

It looks like latest xcode doesn't like the tests that are generated by autotools, right ? If yes, does a autoreconf with fresh autotools fix the problem ?

I'm not an expert in this sort of thing, but I don't think this will work. autoreconf would help if Sage's top-level configure script were running into problems, but the problems are coming from Sage packages.

I didn't make myself clear. Of course I mean autoreconf for gf2x.

Those are just tarballs, and running autoreconf once won't help with those. For what it's worth, I unpacked the gf2x tarball and ran autoreconf on it, but I got the same error.

ok.

(This is with autoreconf 2.69 installed via homebrew. I did brew upgrade first. I'm not sure how to get a "fresh autotools".)

doesn't say much. 2.69 is the banner under which all autoconf version have been presenting themselves for 8 years. IDK about the others, but I suspect autoconf is at fault here anyway.

[jhpalmieri]

comment:118

For what it's worth, I tried building rubiks and it seemed to hang. I'm trying again with a fresh Sage install, and also trying a few other packages at the same time.

[jhpalmieri]

comment:120

deformation succeeded for me, so I removed it from the ticket description. polylib fails on two different machines with the old familiar message error: implicit declaration of function. The rubiks build hangs on two different machines shortly after starting:

Building Rubiks cube solvers
Installing rubiks-20070912.p21
for dir in dietz/cu2 dietz/mcube dietz/solver dik reid; do \
		(cd ${dir} && make all)\
	done
g++ -std=gnu++11 -g -O2  -c cu2.cpp
g++ -std=gnu++11 -g -O2  -c main.cpp
g++ -std=gnu++11 -g -O2   -o cu2  cu2.o main.o  -L/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.6.rc1/local/lib -Wl,-rpath,/Users/jpalmier/Desktop/Sage/sage_builds/TESTING/sage-9.6.rc1/local/lib 
g++ -std=gnu++11 -g -O2  -c mcube.cpp
g++ -std=gnu++11 -g -O2  -c main.cpp
mcube.cpp:1966:38: warning: if statement has empty body [-Wempty-body]
        else if (fy == 3 && fz == 3) ;
                                     ^
mcube.cpp:1966:38: note: put the semicolon on a separate line to silence this warning
mcube.cpp:1977:38: warning: if statement has empty body [-Wempty-body]
        else if (fx == 3 && fy == 3) ;
                                     ^
mcube.cpp:1977:38: note: put the semicolon on a separate line to silence this warning

[jhpalmieri]

comment:122

boost is no longer a package, and fricas, e_antic, and csdp all work for me.

[jhpalmieri]

comment:123

The experimental packages don't matter much (since they're experimental), but lrslib works for me. The others don't. scipoptsuite is broken because the tarball can't be found.

[mkoeppe]

comment:124

I see the hanging build for rubiks now too. This must be a very recent regression. I built it quite recently (before upgrading to macOS Monterey)

[jhpalmieri]

comment:125

Replying to @jhpalmieri:

scipoptsuite is broken because the tarball can't be found.

Maybe this is intentional, given SPKG.rst:

We do not have permission to redistribute SCIP or SoPlex. Hence, you
must download it yourself from http://scip.zib.de and put the tarball
``scipoptsuite-VERSION.tgz`` in ``$SAGE_ROOT/upstream``, renaming
it to ``scipoptsuite-VERSION-do-not-distribute.tgz``.

If anyone cares about this package, they should create a better error message: maybe create spkg-preinst.in that tests for the presence of the tarball in upstream and exit with an appropriate message if it's not there. I don't know if that would actually work, or if the tarball is attempted to be downloaded before anything else happens. Anyway, I don't care about this package, so I am not likely to work on it.

[mkoeppe]

comment:126

Yes, it is intentional that there is no upstream_url and no tarball on our mirror.

[mkoeppe]

comment:127

There appears to be an effort to provide a version of scipoptsuite under a free software license. I'll wait for that before putting more effort into packaging it

[mkoeppe]

comment:128

Replying to @jhpalmieri:

boost is no longer a package, and fricas, e_antic, and csdp all work for me.

OK, I've removed them (and lrslib) from the ticket description

[jhpalmieri]

comment:131

Keep this open for polylib? Or close it?

[mkoeppe]

comment:132

Let's close it.

[dimpase]

Reviewer: Dima Pasechnik