Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error building julia from source #40585

Closed
KlausC opened this issue Apr 23, 2021 · 12 comments
Closed

Error building julia from source #40585

KlausC opened this issue Apr 23, 2021 · 12 comments
Assignees

Comments

@KlausC
Copy link
Contributor

KlausC commented Apr 23, 2021

On my installations (fedora34) making julia from a fresh clone of JuliaLang/julia fails with the following messages:

No problems with fedora33.

LINK usr/lib/libjulia-internal.so.1.7
/usr/bin/ld: ./codegen.o: in function `__gnu_cxx::new_allocator<llvm::Type*>::allocate(unsigned long, void const*)':
/usr/include/c++/11/ext/new_allocator.h:110: undefined reference to `std::__throw_bad_array_new_length()'
/usr/bin/ld: ./llvm-late-gc-lowering.o: in function `__gnu_cxx::new_allocator<int>::allocate(unsigned long, void const*)':
/usr/include/c++/11/ext/new_allocator.h:110: undefined reference to `std::__throw_bad_array_new_length()'

... some more of that kind

It came out, that the version of ~/julia/usr/lib/libstdc++.so.6.0.28 is responsible for that.
When I replace this library by the system version in /usr/lib64/libstdc++.6.0.29 julia links and runs like a charm.

ls -l libstdc*
lrwxrwxrwx. 1 crusius ems       19 23. Apr 16:14 libstdc++.so -> libstdc++.so.6.0.29
lrwxrwxrwx. 1 crusius ems       19 23. Apr 16:14 libstdc++.so.6 -> libstdc++.so.6.0.29
-rwxr-xr-x. 1 crusius ems 20907888 23. Apr 10:23 libstdc++.so.6.0.28
-rwxr-xr-x. 1 crusius ems  2227752 23. Apr 16:13 libstdc++.so.6.0.29

LINK usr/lib/libjulia-internal.so.1.7
LINK usr/lib/libjulia-internal.so.1
LINK usr/lib/libjulia-internal.so
JULIA usr/lib/julia/corecompiler.ji
JULIA usr/lib/julia/sys.ji
coreio.jl
@KlausC
Copy link
Contributor Author

KlausC commented Apr 23, 2021

details of the compiler version in use:

gcc -v
Es werden eingebaute Spezifikationen verwendet.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Ziel: x86_64-redhat-linux
Konfiguriert mit: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-11.0.1-20210324/obj-x86_64-redhat-linux/isl-install --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread-Modell: posix
Unterstützte LTO-Kompressionsalgorithmen: zlib zstd
gcc-Version 11.0.1 20210324 (Red Hat 11.0.1-0) (GCC) 

@KlausC
Copy link
Contributor Author

KlausC commented Apr 24, 2021

Problem seems to be, that the build process joins the include file /usr/include/c++/11/ext/new_allocator.h of the system release with the incompatible library provided by ~/julia/usr/libstdc++.so.6.

@guilleaf
Copy link

guilleaf commented Apr 30, 2021

I can reproduce the error using GCC 11.1 and Julia 1.6.1 both with git clone and downloading the tar package:

This is the error at the end of the compilation. The Make.inc was left unchanged.

    LINK usr/lib/libjulia-internal.so.1.6
    JULIA usr/lib/julia/corecompiler.ji
ERROR: Unable to load dependent library /gpfs20/shared/src/julia/usr/bin/../lib/libjulia-internal.so.1
Message:/gpfs20/shared/src/julia/usr/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /gpfs20/shared/src/julia/usr/bin/../lib/libjulia-internal.so.1)
make[1]: *** [sysimage.mk:61: /gpfs20/shared/src/julia/usr/lib/julia/corecompiler.ji] Error 1
make: *** [Makefile:82: julia-sysimg-ji] Error 2

It seems that the Julia build system copies and old version of libstdc++

$> ls -al usr/lib/libstdc++.so*
lrwxrwxrwx 1 build build       19 Apr 30 10:11 usr/lib/libstdc++.so.6 -> libstdc++.so.6.0.26
-r-xr-xr-x 1 build build 22521088 Apr 30 10:11 usr/lib/libstdc++.so.6.0.26

Instead of the libstdc++ declared on $LIBRARY_PATH

$> echo $LIBRARY_PATH
/shared/software/lang/gcc/11.1.0/lib64:/shared/software/lang/gcc/11.1.0/lib

The code is actually compiled with this one:

$> ls -l /shared/software/lang/gcc/11.1.0/lib64/libstdc++.so*
lrwxrwxrwx 1 build build       19 Apr 29 08:50 /shared/software/lang/gcc/11.1.0/lib64/libstdc++.so -> libstdc++.so.6.0.29
lrwxrwxrwx 1 build build       19 Apr 29 08:50 /shared/software/lang/gcc/11.1.0/lib64/libstdc++.so.6 -> libstdc++.so.6.0.29
-rwxr-xr-x 1 build build 14615584 Apr 29 08:50 /shared/software/lang/gcc/11.1.0/lib64/libstdc++.so.6.0.29
-rw-r--r-- 1 build build     2442 Apr 29 08:50 /shared/software/lang/gcc/11.1.0/lib64/libstdc++.so.6.0.29-gdb.py

@rjpower4
Copy link

rjpower4 commented May 7, 2021

This also occurs when using the precompiled binary along with packages like PyPlot while using the system python installation as the libstdc++ version used by julia does not match the system version.

Again with Julia v1.6.1 on Fedora 34 with system libstdc++.6.0.29:

julia> using PyPlot
ERROR: InitError: PyError (PyImport_ImportModule

The Python package matplotlib could not be imported by pyimport. Usually this means
that you did not install matplotlib in the Python version being used by PyCall.

PyCall is currently configured to use the Python version at:

/usr/bin/python3

and you should use whatever mechanism you usually use (apt-get, pip, conda,
etcetera) to install the Python package containing the matplotlib module.

One alternative is to re-configure PyCall to use a different Python
version on your system: set ENV["PYTHON"] to the path/name of the python
executable you want to use, run Pkg.build("PyCall"), and re-launch Julia.

Another alternative is to configure PyCall to use a Julia-specific Python
distribution via the Conda.jl package (which installs a private Anaconda
Python distribution), which has the advantage that packages can be installed
and kept up-to-date via Julia.  As explained in the PyCall documentation,
set ENV["PYTHON"]="", run Pkg.build("PyCall"), and re-launch Julia. Then,
To install the matplotlib module, you can use `pyimport_conda("matplotlib", PKG)`,
where PKG is the Anaconda package that contains the module matplotlib,
or alternatively you can use the Conda package directly (via
`using Conda` followed by `Conda.add` etcetera).

) <class 'ImportError'>
ImportError("/home/rpower/.local/opt/julia/julia-1.6.1/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /usr/lib64/python3.9/site-packages/matplotlib/_path.cpython-39-x86_64-linux-gnu.so)")
  File "/usr/lib64/python3.9/site-packages/matplotlib/__init__.py", line 107, in <module>
    from . import _api, cbook, docstring, rcsetup
  File "/usr/lib64/python3.9/site-packages/matplotlib/rcsetup.py", line 26, in <module>
    from matplotlib.colors import Colormap, is_color_like
  File "/usr/lib64/python3.9/site-packages/matplotlib/colors.py", line 82, in <module>
    from matplotlib import _api, cbook, scale
  File "/usr/lib64/python3.9/site-packages/matplotlib/scale.py", line 18, in <module>
    from matplotlib.ticker import (
  File "/usr/lib64/python3.9/site-packages/matplotlib/ticker.py", line 179, in <module>
    from matplotlib import transforms as mtransforms
  File "/usr/lib64/python3.9/site-packages/matplotlib/transforms.py", line 46, in <module>
    from matplotlib._path import (

Stacktrace:
 [1] pyimport(name::String)
   @ PyCall ~/.local/opt/julia/depot/packages/PyCall/BD546/src/PyCall.jl:550
 [2] pyimport_conda(modulename::String, condapkg::String, channel::String)
   @ PyCall ~/.local/opt/julia/depot/packages/PyCall/BD546/src/PyCall.jl:708
 [3] pyimport_conda
   @ ~/.local/opt/julia/depot/packages/PyCall/BD546/src/PyCall.jl:707 [inlined]
 [4] __init__()
   @ PyPlot ~/.local/opt/julia/depot/packages/PyPlot/XHEG0/src/init.jl:179
 [5] _include_from_serialized(path::String, depmods::Vector{Any})
   @ Base ./loading.jl:674
 [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
   @ Base ./loading.jl:760
 [7] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:998
 [8] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:914
 [9] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:901
during initialization of module PyPlot

Replacing the library in julia/lib/julia with the system one fixes the issue if a quick fix is needed. Without this library replacement, neither building from source nor downloading a pre-built binary will work in my testing.

@lmshk
Copy link

lmshk commented May 30, 2021

I have a similar issue (with both v1.6.0 and v1.6.1 on Arch Linux with GCC 11.1.0, also on a fresh Julia clone):

    LINK usr/lib/libjulia-internal.so.1.6
    JULIA usr/lib/julia/corecompiler.ji
ERROR: Unable to load dependent library /home/⟨user⟩/src/julia/usr/bin/../lib/libjulia-internal.so.1
Message:/home/⟨user⟩/src/julia/usr/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/⟨user⟩/src/julia/usr/bin/../lib/libjulia-internal.so.1)
make[1]: *** [sysimage.mk:61: /home/⟨user⟩/src/julia/usr/lib/julia/corecompiler.ji] Error 1
make: *** [Makefile:82: julia-sysimg-ji] Error 2

Using /usr/lib64/libstdc++.6.0.29 as described above lets me build without further problems.

@DoktorMike
Copy link

Same for me. I was able to build with this fix as well. Thanks!

@mauro3
Copy link
Contributor

mauro3 commented Jun 28, 2021

If I understand correctly, this is fixed by #41292, it just hasn't been backported yet.

@KlausC
Copy link
Contributor Author

KlausC commented Jul 12, 2021

seems fixed now

@KlausC KlausC closed this as completed Jul 12, 2021
@gdkrmr
Copy link
Contributor

gdkrmr commented Jul 15, 2021

I have a similar issue (with both v1.6.0 and v1.6.1 on Arch Linux with GCC 11.1.0, also on a fresh Julia clone):

    LINK usr/lib/libjulia-internal.so.1.6
    JULIA usr/lib/julia/corecompiler.ji
ERROR: Unable to load dependent library /home/⟨user⟩/src/julia/usr/bin/../lib/libjulia-internal.so.1
Message:/home/⟨user⟩/src/julia/usr/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/⟨user⟩/src/julia/usr/bin/../lib/libjulia-internal.so.1)
make[1]: *** [sysimage.mk:61: /home/⟨user⟩/src/julia/usr/lib/julia/corecompiler.ji] Error 1
make: *** [Makefile:82: julia-sysimg-ji] Error 2

Using /usr/lib64/libstdc++.6.0.29 as described above lets me build without further problems.

I am getting the same error when compiling 1.6.2 on Manjaro

EDIT: also, using the system libstdc++.so works.

@mauro3
Copy link
Contributor

mauro3 commented Jul 15, 2021

Apparently that is expected as the fix couldn't be backported: https://discourse.julialang.org/t/julia-1-6-2-cannot-be-compiled/64685/2

@KlausC KlausC reopened this Jul 15, 2021
@vtjnash
Copy link
Member

vtjnash commented Jul 16, 2021

@staticfloat can I assign you to this, for getting CSL from the system if the compiler is newer than our build shards? We should probably backport this also, since it is important that we keep the v1.6 LTS buildable for people for many years

@KristofferC
Copy link
Member

It is on the 1.6.3 backport branch now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants