Build fails on macOS with Xcode 13.0 (but passes with Xcode 12.5.1) #42346

jonas-schulze · 2021-09-22T13:41:28Z

Cloning and building Julia on the current master (5bf1ca5)

git clone ...
make

leads to

...
    JULIA usr/lib/julia/sys-o.a
/bin/sh: line 1: 11329 Segmentation fault: 11  JULIA_BINDIR=/Users/jonas/Projects/julia2/usr/bin WINEPATH="/Users/jonas/Projects/julia2/usr/bin;$WINEPATH" /Users/jonas/Projects/julia2/usr/bin/julia -O3 -C "native" --output-o /Users/jonas/Projects/julia2/usr/lib/julia/sys-o.a.tmp --startup-file=no --warn-overwrite=yes --sysimage /Users/jonas/Projects/julia2/usr/lib/julia/sys.ji /Users/jonas/Projects/julia2/contrib/generate_precompile.jl 1
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
make[1]: *** [/Users/jonas/Projects/julia2/usr/lib/julia/sys-o.a] Error 1
make: *** [julia-sysimg-release] Error 2

While working on my "old" clone, I encountered the same problem. make clean and make cleanall didn't fix the issue. I didn't try this on the new clone. I also tried creating a Make.user containing JULIA_PRECOMPILE=0, but that didn't work either.

I am running macOS 11.5.2 on a 2018 MacBook Pro. From my homebrew Julia installation:

julia> versioninfo()
Julia Version 1.6.2
Commit 1b93d53fc4 (2021-07-14 15:36 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)

The text was updated successfully, but these errors were encountered:

vtjnash · 2021-09-22T14:37:46Z

It is likely you have something broken in your environment. Check env for something being added.

jonas-schulze · 2021-09-22T17:37:05Z

I don't see anything suspicious. 🙁

$ env | cut -d= -f1
TERM_SESSION_ID
SSH_AUTH_SOCK
LC_TERMINAL_VERSION
COLORFGBG
ITERM_PROFILE
SQLITE_EXEMPT_PATH_FROM_VNODE_GUARDS
XPC_FLAGS
LANG
PWD
SHELL
__CFBundleIdentifier
SECURITYSESSIONID
TERM_PROGRAM_VERSION
TERM_PROGRAM
PATH
DISPLAY
LC_TERMINAL
COLORTERM
COMMAND_MODE
TERM
HOME
TMPDIR
USER
XPC_SERVICE_NAME
LOGNAME
LaunchInstanceID
__CF_USER_TEXT_ENCODING
ITERM_SESSION_ID
SHLVL
OLDPWD
EDITOR
TEXINPUTS
JULIA_NUM_THREADS
JULIA_PROJECT
JULIA_PKG_PRECOMPILE_AUTO
PAGER
LESS
LSCOLORS
LC_CTYPE
FZF_DEFAULT_COMMAND
HOMEBREW_GITHUB_API_TOKEN
_

Edit: thanks for the quick response!

vtjnash · 2021-09-22T17:48:39Z

JULIA_NUM_THREADS -- the build does not support explicitly launching threads

DilumAluthge · 2021-09-23T05:06:50Z

This seems like an error on our end. If the build doesn't support JULIA_NUM_THREADS, then it should ignore it, instead of crashing.

DilumAluthge · 2021-09-23T22:21:47Z

I'm reopening this because I think we need to fix this on our end.

vilterp · 2021-09-23T22:52:36Z

For what it's worth, this happens to me even with JULIA_NUM_THREADS unset.

NHDaly · 2021-09-23T23:30:43Z

@vilterp and, it's worth mentioning, it only started happening to you today, on a build that was working up til yesterday.

DilumAluthge · 2021-09-23T23:31:17Z

@vilterp and, it's worth mentioning, it only started happening to you today, on a build that was working up til yesterday.

It would be great to bisect this.

DilumAluthge · 2021-09-23T23:46:50Z

I am not able to reproduce this locally with the following setup:

macOS Big Sur version 11.6 (20G165)
Xcode version 12.5.1 (12E507)

I tried both with and without export JULIA_NUM_THREADS=4, but I wasn't able to reproduce the build failure either way.

@jonas-schulze or @vilterp would you be able to git bisect this, since you are able reproduce this failure locally?

vilterp · 2021-09-24T00:30:32Z

The thing is, I am now unable to build commits which I was able to build before. It seems like some state on my system is corrupt, which is preventing the build from succeeding.

I tried make clean, make cleanall, and even deleted my checkout and restarted, but to no avail.

The only thing I can think of which changed from when it was working to now is that I used to only be building the C source, with make -C src and make -j julia-src-release.

Eventually, I made a change to the Julia source (this is all on #42286), and issued make. However, at the time, my C source had a bug in it, so building the sysimage failed. Once I fixed the bug on the C side, I ran into this bug, there the C and sysimage build, but some step afterward segfaults.

Is there some state outside of the julia checkout directory itself which could be corrupted, causing my failed builds?

DilumAluthge · 2021-09-24T00:33:20Z

There shouldn't be any state outside of the Julia source directory.

To be safe, you can try doing e.g. rm -rf ~/.julia, but I doubt that will help.

Has anything changed on your system? For example:

macOS update?
Xcode update?

For example, Xcode 13 was released on September 20, 2021. If you installed Xcode from the Mac App Store, I think it auto-updates whenever a new version is released. So maybe that's what happened here?

vilterp · 2021-09-24T01:16:18Z

Hm, maybe it is an XCode update or something.

I pulled out an old laptop, and the same commits build just fine on it.

old laptop
- macOS 11.3
- clang -v:

clang version 11.0.0
Target: x86_64-apple-darwin20.4.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin

new laptop
- macOS 11.5.2
- clang -v:

Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Unblocked for now, thankfully… Thanks for helping look into this! Still mysterious… 🤔

DilumAluthge · 2021-09-24T01:18:39Z

If the Julia build is broken on Xcode 13, that is not great...

vilterp · 2021-09-24T01:20:59Z

Maybe it auto-updated under me. What is the macOS CI running? Will take a look…

vilterp · 2021-09-24T01:25:51Z

Can't tell from this build what version of xcode/clang is running in CI; not sure where the config lives https://build.julialang.org/#/builders/63/builds/3806

But yeah, maybe it's broken on clang 13

NHDaly · 2021-09-24T01:34:13Z

Great sleuthing y'all 👍👍

DilumAluthge · 2021-09-24T03:57:02Z

On my local machine, with otherwise identical build environments, I can confirm that the build passes with Xcode 12 and fails with Xcode 13. Both of these builds are using the exact same commit of Julia.

macOS	Xcode	Clang	Build
Big Sur 11.6 (20G165)	12.5.1 (12E507)	12.0.5 (clang-1205.0.22.11)	✅
Big Sur 11.6 (20G165)	13.0 (13A233)	13.0.0 (clang-1300.0.29.3)	❌

The error I get with Xcode 13.0 is the same as others have posted:

Sysimage built. Summary:
Total ───────  73.074822 seconds
Base: ───────  29.993381 seconds 41.0448%
Stdlibs: ────  43.078845 seconds 58.9517%
    JULIA usr/lib/julia/sys-o.a
/bin/sh: line 1:  6432 Segmentation fault: 11  JULIA_BINDIR=/Users/dilum/Downloads/julia/usr/bin WINEPATH="/Users/dilum/Downloads/julia/usr/bin;$WINEPATH" /Users/dilum/Downloads/julia/usr/bin/julia -O3 -C "native" --output-o /Users/dilum/Downloads/julia/usr/lib/julia/sys-o.a.tmp --startup-file=no --warn-overwrite=yes --sysimage /Users/dilum/Downloads/julia/usr/lib/julia/sys.ji /Users/dilum/Downloads/julia/contrib/generate_precompile.jl 1
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
make[1]: *** [/Users/dilum/Downloads/julia/usr/lib/julia/sys-o.a] Error 1
make: *** [julia-sysimg-release] Error 2

Note: the JULIA_NUM_THREADS environment variable was not defined during any of the above builds.

wookay · 2021-09-24T06:26:52Z

~~I just found the difference from git blame commands between two macs.~~
~~ref: https://github.com/JuliaLang/julia/blob/2c9e051/base/version_git.sh#L45~~

$ xcodebuild -version
Xcode 13.0
Build version 13A233

$ git --version
git version 2.30.1 (Apple Git-130)

$ git blame -sl VERSION
^5e8d7edfffbac8a69fad0f901089bbaa97d1043 1) 1.8.0-DEV

oh, it was my fault. I did git clone of depth 1.

jonas-schulze · 2021-09-24T09:30:50Z

It would be great to bisect this.

The last time I worked on Julia (#41834), everything built just fine. Now that commit is "broken", too. I don't know my version of Clang or Xcode back then but now it's 13:

macOS	Xcode	Clang	Build
Big Sur 11.5.2 (20G95)	13.0 (13A233)	13.0.0 (clang-1300.0.29.3)	💥

DilumAluthge · 2021-09-24T16:59:05Z

Yeah, based on the comments from @jonas-schulze and @vilterp, I don't think anything changed in Julia. I think the issue is with Xcode 12 versus Xcode 13.

jl_current_task is not constant after this point in the function, so we split the function so that the compiler won't try to optimize it incorrectly (hoisting the JL_GC_PUSH to the top of the function for example). Fixes #42346

IanButterworth · 2021-09-27T04:22:10Z

I just hit this but didn't find this issue until after a few hours of head scratching..
I saw no sign that MacOS had updated anything :/

jl_current_task is not constant after this point in the function, so we split the function so that the compiler won't try to optimize it incorrectly (hoisting the JL_GC_PUSH to the top of the function for example). Fixes #42346

jl_current_task is not constant after this point in the function, so we split the function so that the compiler won't try to optimize it incorrectly (hoisting the JL_GC_PUSH to the top of the function for example). Fixes #42346 (cherry picked from commit 4dea1c4)

jl_current_task is not constant after this point in the function, so we split the function so that the compiler won't try to optimize it incorrectly (hoisting the JL_GC_PUSH to the top of the function for example). Fixes JuliaLang#42346

vtjnash closed this as completed Sep 22, 2021

DilumAluthge mentioned this issue Sep 23, 2021

unable to build on macOS #42369

Closed

DilumAluthge reopened this Sep 23, 2021

DilumAluthge added bug Indicates an unexpected problem or unintended behavior system:mac Affects only macOS labels Sep 23, 2021

DilumAluthge added the building Build system, or building Julia or its dependencies label Sep 24, 2021

DilumAluthge changed the title ~~Build broken on macOS Big Sur 11.5.2~~ Build fails on macOS with Xcode 13.0 (but passes with Xcode 12.5.1) Sep 24, 2021

vtjnash mentioned this issue Sep 24, 2021

init: avoid an undesirable compiler optimization #42377

Merged

IanButterworth closed this as completed in #42377 Sep 28, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build fails on macOS with Xcode 13.0 (but passes with Xcode 12.5.1) #42346

Build fails on macOS with Xcode 13.0 (but passes with Xcode 12.5.1) #42346

jonas-schulze commented Sep 22, 2021

vtjnash commented Sep 22, 2021

jonas-schulze commented Sep 22, 2021 •

edited

Loading

vtjnash commented Sep 22, 2021

DilumAluthge commented Sep 23, 2021

DilumAluthge commented Sep 23, 2021

vilterp commented Sep 23, 2021

NHDaly commented Sep 23, 2021

DilumAluthge commented Sep 23, 2021

DilumAluthge commented Sep 23, 2021 •

edited

Loading

vilterp commented Sep 24, 2021

DilumAluthge commented Sep 24, 2021 •

edited

Loading

vilterp commented Sep 24, 2021 •

edited

Loading

DilumAluthge commented Sep 24, 2021 •

edited

Loading

vilterp commented Sep 24, 2021

vilterp commented Sep 24, 2021 •

edited

Loading

NHDaly commented Sep 24, 2021

DilumAluthge commented Sep 24, 2021 •

edited

Loading

wookay commented Sep 24, 2021 •

edited

Loading

jonas-schulze commented Sep 24, 2021 •

edited

Loading

DilumAluthge commented Sep 24, 2021

IanButterworth commented Sep 27, 2021

Build fails on macOS with Xcode 13.0 (but passes with Xcode 12.5.1) #42346

Build fails on macOS with Xcode 13.0 (but passes with Xcode 12.5.1) #42346

Comments

jonas-schulze commented Sep 22, 2021

vtjnash commented Sep 22, 2021

jonas-schulze commented Sep 22, 2021 • edited Loading

vtjnash commented Sep 22, 2021

DilumAluthge commented Sep 23, 2021

DilumAluthge commented Sep 23, 2021

vilterp commented Sep 23, 2021

NHDaly commented Sep 23, 2021

DilumAluthge commented Sep 23, 2021

DilumAluthge commented Sep 23, 2021 • edited Loading

vilterp commented Sep 24, 2021

DilumAluthge commented Sep 24, 2021 • edited Loading

vilterp commented Sep 24, 2021 • edited Loading

DilumAluthge commented Sep 24, 2021 • edited Loading

vilterp commented Sep 24, 2021

vilterp commented Sep 24, 2021 • edited Loading

NHDaly commented Sep 24, 2021

DilumAluthge commented Sep 24, 2021 • edited Loading

wookay commented Sep 24, 2021 • edited Loading

jonas-schulze commented Sep 24, 2021 • edited Loading

DilumAluthge commented Sep 24, 2021

IanButterworth commented Sep 27, 2021

jonas-schulze commented Sep 22, 2021 •

edited

Loading

DilumAluthge commented Sep 23, 2021 •

edited

Loading

DilumAluthge commented Sep 24, 2021 •

edited

Loading

vilterp commented Sep 24, 2021 •

edited

Loading

DilumAluthge commented Sep 24, 2021 •

edited

Loading

vilterp commented Sep 24, 2021 •

edited

Loading

DilumAluthge commented Sep 24, 2021 •

edited

Loading

wookay commented Sep 24, 2021 •

edited

Loading

jonas-schulze commented Sep 24, 2021 •

edited

Loading