Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for armv7l buildbots #51945

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Fixes for armv7l buildbots #51945

wants to merge 4 commits into from

Conversation

staticfloat
Copy link
Sponsor Member

We have a new armv7l build farm; this branch is collecting the fixes necessary to get Julia building on that platform again.

@giordano giordano added system:32-bit Affects only 32-bit systems system:arm ARMv7 and AArch64 labels Oct 30, 2023
src/stackwalk.c Outdated Show resolved Hide resolved
@staticfloat
Copy link
Sponsor Member Author

Alright, next up, a bus error during bootstrap:

[10767] signal (7.1): Bus error
in expression starting at compiler/bootstrap.jl:8
unknown function (ip: 0xee648e0c)
unknown function (ip: 0xee648f37)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xee69338b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xee693077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc295ab)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
unknown function (ip: 0xee63dd8b)
unknown function (ip: 0xee645b17)
unknown function (ip: 0xee646b1b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2902 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
unknown function (ip: 0xee63dd8b)
unknown function (ip: 0xee645b17)
unknown function (ip: 0xee646b1b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2902 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe17db)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe17db)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe17db)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc3b077)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc37127)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefc291d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf87cb)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf4cff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf45c3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbf37f3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbefa07)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe4347)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe3163)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbe1f9b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbdd157)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd7a2f)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd4557)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefbd31d3)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefba7273)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefb9c4ff)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
jl_apply at /cache/build/default-pibot5-0/julialang/julia-master/src/julia.h:2130 [inlined]
do_call at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:126
eval_value at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:223
eval_stmt_value at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:174 [inlined]
eval_body at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:647
jl_interpret_toplevel_thunk at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:787
jl_toplevel_eval_flex at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:938
jl_parse_eval_all at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:1062
ijl_load_ at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:1109
unknown function (ip: 0xefb4663b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xefb465bf)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
jl_apply at /cache/build/default-pibot5-0/julialang/julia-master/src/julia.h:2130 [inlined]
do_call at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:126
eval_value at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:223
eval_stmt_value at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:174 [inlined]
eval_body at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:647
jl_interpret_toplevel_thunk at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:787
jl_toplevel_eval_flex at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:938
jl_eval_module_expr at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:215 [inlined]
jl_toplevel_eval_flex at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:738
ijl_toplevel_eval_in at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:989
unknown function (ip: 0xefb4579b)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
jl_apply at /cache/build/default-pibot5-0/julialang/julia-master/src/julia.h:2130 [inlined]
do_call at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:126
eval_value at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:223
eval_stmt_value at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:174 [inlined]
eval_body at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:647
jl_interpret_toplevel_thunk at /cache/build/default-pibot5-0/julialang/julia-master/src/interpreter.c:787
jl_toplevel_eval_flex at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:938
jl_parse_eval_all at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:1062
ijl_load_ at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:1109
ijl_load at /cache/build/default-pibot5-0/julialang/julia-master/src/toplevel.c:1122
exec_program at /cache/build/default-pibot5-0/julialang/julia-master/src/jlapi.c:544
true_main at /cache/build/default-pibot5-0/julialang/julia-master/src/jlapi.c:601
jl_repl_entrypoint at /cache/build/default-pibot5-0/julialang/julia-master/src/jlapi.c:738
main at /cache/build/default-pibot5-0/julialang/julia-master/cli/loader_exe.c:58
__libc_start_main at /lib/arm-linux-gnueabihf/libc.so.6 (unknown line)
unknown function (ip: 0x107c7)
Allocations: 184790311 (Pool: 184789656; Big: 655); GC: 74
Bus error
make[1]: *** [sysimage.mk:61: /cache/build/default-pibot5-0/julialang/julia-master/usr/lib/julia/corecompiler.ji] Error 135
make[1]: Leaving directory '/cache/build/default-pibot5-0/julialang/julia-master'
make: *** [Makefile:103: julia-sysimg-ji] Error 2

@vtjnash
Copy link
Sponsor Member

vtjnash commented Oct 30, 2023

According to asm-generic/siginfo.h on my system, that is:

#define BUS_ADRALN      1       /* invalid address alignment */

@staticfloat
Copy link
Sponsor Member Author

I managed to catch it in gdb:

(gdb) handle SIGSEGV nostop noprint
Signal        Stop      Print   Pass to program Description
SIGSEGV       No        No      Yes             Segmentation fault
(gdb) r
Starting program: /build/julia/usr/bin/julia -C native --output-ji ..//usr/lib/julia/corecompiler.ji.tmp --startup-file=no --warn-overwrite=yes -g0 -O0 compiler/compiler.jl
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[Detaching after fork from child process 29]
[New Thread 0xf31ab460 (LWP 30)]

Thread 1 "julia" received signal SIGBUS, Bus error.
0xeea77e8c in ?? ()
(gdb) bt                                                                                                                                                     
#0  0xeea77e8c in ?? ()
#1  0xf77af5c0 in atomic_compare_exchange_n (fm=0, sm=0, weak_p=true, newval=<optimized out>, eptr=0xee6641f0, mptr=0xfffbfbf8)
    at /workspace/srcdir/gcc-13.2.0/libatomic/config/linux/arm/host-config.h:80
#2  libat_store_8_i2 (mptr=0xfffbfbf8, newval=<optimized out>, smodel=927126)
    at /workspace/srcdir/gcc-13.2.0/libatomic/config/linux/arm/../../../store_n.c:55
#3  0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

I'll see if the same happens with a debug build, just to try for a better stacktrace.

@staticfloat
Copy link
Sponsor Member Author

I tried running a julia-debug bootstrap under gdb but after 48 hours of a single core pegged it still hadn't finished, so I gave up.

@vtjnash
Copy link
Sponsor Member

vtjnash commented Nov 16, 2023

Try again with #52182 merged perhaps?

@staticfloat
Copy link
Sponsor Member Author

Sadly, it doesn't seem to help (yet)

@vtjnash
Copy link
Sponsor Member

vtjnash commented Dec 9, 2023

I let it run for about an hour on my ARMv7 board until I hit:

Thread 1 "julia" received signal SIGBUS, Bus error.
0x9da7d1b8 in ?? ()
(gdb) bt
#0  0x9da7d1b8 in ?? ()
0xac448330:     0xafdb8a30
(gdb) disas /r $pc-64,+128 
Dump of assembler code from 0x9da7d178 to 0x9da7d1f8:
   0x9da7d178:  eb0000f0        bl      0x9da7d540
   0x9da7d17c:  e59d3030        ldr     r3, [sp, #48]   @ 0x30
   0x9da7d180:  e59d2034        ldr     r2, [sp, #52]   @ 0x34
   0x9da7d184:  e59dc038        ldr     r12, [sp, #56]  @ 0x38
   0x9da7d188:  e1a01000        mov     r1, r0
   0x9da7d18c:  e59d0048        ldr     r0, [sp, #72]   @ 0x48
   0x9da7d190:  e58d103c        str     r1, [sp, #60]   @ 0x3c
   0x9da7d194:  e5013004        str     r3, [r1, #-4]
   0x9da7d198:  e581c000        str     r12, [r1]
   0x9da7d19c:  e1a03001        mov     r3, r1
   0x9da7d1a0:  e5a3c00c        str     r12, [r3, #12]!
   0x9da7d1a4:  f57ff05b        dmb     ish
   0x9da7d1a8:  e5814000        str     r4, [r1]
   0x9da7d1ac:  e302ca63        movw    r12, #10851     @ 0x2a63
   0x9da7d1b0:  e34acd9f        movt    r12, #44447     @ 0xad9f
   0x9da7d1b4:  eddc0b00        vldr    d16, [r12]
=> 0x9da7d1b8:  edc10b01        vstr    d16, [r1, #4]
   0x9da7d1bc:  f57ff05b        dmb     ish
   0x9da7d1c0:  e5832000        str     r2, [r3]
   0x9da7d1c4:  e3072718        movw    r2, #30488      @ 0x7718
   0x9da7d1c8:  e34a2daf        movt    r2, #44463      @ 0xadaf
   0x9da7d1cc:  e5926000        ldr     r6, [r2]
   0x9da7d1d0:  e58d12ac        str     r1, [sp, #684]  @ 0x2ac
   0x9da7d1d4:  e58d62a4        str     r6, [sp, #676]  @ 0x2a4
   0x9da7d1d8:  e5900008        ldr     r0, [r0, #8]
   0x9da7d1dc:  e3a01f63        mov     r1, #396        @ 0x18c
   0x9da7d1e0:  e3a02008        mov     r2, #8
   0x9da7d1e4:  e1a03006        mov     r3, r6
   0x9da7d1e8:  eb0000d4        bl      0x9da7d540
   0x9da7d1ec:  e59d2034        ldr     r2, [sp, #52]   @ 0x34
   0x9da7d1f0:  e59dc038        ldr     r12, [sp, #56]  @ 0x38
   0x9da7d1f4:  e59d303c        ldr     r3, [sp, #60]   @ 0x3c
End of assembler dump.
(gdb) p jl_($r1)
Core.Compiler.ConcreteResult(mi=!(Bool) from !(Bool), effects=Core.Compiler.Effects(consistent=0x70, effect_free=0x49, nothrow=true, terminates=true, notaskstate=true, inaccessiblememonly=0x49, noub=0x52, nonoverlayed=true), result=#<null>)
(gdb) info registers d16
d16            0                   (raw 0x0000000000000000)
(gdb) p (void) jl_gdblookup((void*)0x9da7d1b4)
ConcreteResult at ./compiler/stmtinfo.jl:71 [inlined]
concrete_eval_call at ./compiler/abstractinterpretation.jl:953
(gdb) p/x *(long long*) $r12
$33 = 0x0100_0001_0101_0000

Note that I think it is the statement earlier that failed, and gdb is showing the wrong address, as that vstr is clearly aligned, and that vldr from $r12 is clearly not aligned.

@vtjnash
Copy link
Sponsor Member

vtjnash commented Dec 9, 2023

The IR we emit (on x86) is correct with regards to this alignment being align=1, so seems like this might be an ARM backend mistake in LLVM

$ ./julia -e 'using InteractiveUtils; code_llvm(dump_module=true, optimize=false, ()) do; return Ref(Core.Compiler.EFFECTS_TOTAL); end'
; ModuleID = '#3'
source_filename = "#3"
target datalayout = "e-m:e-p:32:32-p270:32:32-p271:32:32-p272:64:64-f64:32:64-f80:32-n8:16:32-S128-ni:10:11:12:13"
target triple = "i686-unknown-linux-gnu"

@"+Main.Base.RefValue#428" = private unnamed_addr constant {}* @"+Main.Base.RefValue#428.jit"
@"_j_const#1" = private unnamed_addr constant [8 x i8] c"\00\00\01\01\01\00\00\01", align 1

@"+Main.Base.RefValue#428.jit" = private alias {}, inttoptr (i32 -435036416 to {}*)

; Function Signature: #3()
;  @ none:1 within `#3`
define nonnull {}* @"julia_#3_426"() #0 {
top:
  %pgcstack = call {}*** @julia.get_pgcstack()
  %0 = bitcast {}*** %pgcstack to {}**
  %current_task = getelementptr inbounds {}*, {}** %0, i32 -20
  %1 = bitcast {}** %current_task to i32*
  %world_age = getelementptr inbounds i32, i32* %1, i32 21
; ┌ @ refpointer.jl:143 within `Ref`
; │┌ @ refvalue.jl:10 within `RefValue` @ refvalue.jl:8
    %"+Main.Base.RefValue#428" = load {}*, {}** @"+Main.Base.RefValue#428", align 4
    %RefValue = ptrtoint {}* %"+Main.Base.RefValue#428" to i32
    %2 = inttoptr i32 %RefValue to {}*
    %3 = bitcast {}*** %pgcstack to {}**
    %current_task1 = getelementptr inbounds {}*, {}** %3, i32 -20
    %"new::RefValue" = call noalias nonnull align 1 dereferenceable(8) {}* @julia.gc_alloc_obj({}** %current_task1, i32 8, {}* %2) #2
    %4 = bitcast {}* %"new::RefValue" to [8 x i8]*
    %5 = bitcast [8 x i8]* %4 to i8*
    call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 1 %5, i8* align 1 getelementptr inbounds ([8 x i8], [8 x i8]* @"_j_const#1", i32 0, i32 0), i64 8, i1 false)
; └└
  ret {}* %"new::RefValue"
}

; Function Attrs: noinline optnone
define nonnull {}* @"jfptr_#3_427"({}* %"function::Core.Function", {}** noalias nocapture noundef readonly %"args::Any[]", i32 %"nargs::UInt32") #1 {
top:
  %pgcstack = call {}*** @julia.get_pgcstack()
  %0 = call nonnull {}* @"julia_#3_426"()
  ret {}* %0
}

declare {}*** @julia.get_pgcstack()

; Function Attrs: nounwind willreturn allockind("alloc,uninitialized") allocsize(1)
declare noalias nonnull {}* @julia.gc_alloc_obj({}**, i32, {}*) #2

; Function Attrs: argmemonly nocallback nofree nounwind willreturn
declare void @llvm.memcpy.p0i8.p0i8.i64(i8* noalias nocapture writeonly, i8* noalias nocapture readonly, i64, i1 immarg) #3

attributes #0 = { "frame-pointer"="all" "julia.fsig"="#3()" "probe-stack"="inline-asm" }
attributes #1 = { noinline optnone "frame-pointer"="all" "probe-stack"="inline-asm" }
attributes #2 = { nounwind willreturn allockind("alloc,uninitialized") allocsize(1) }
attributes #3 = { argmemonly nocallback nofree nounwind willreturn }

!llvm.module.flags = !{!0, !1, !2}

!0 = !{i32 2, !"Dwarf Version", i32 4}
!1 = !{i32 2, !"Debug Info Version", i32 3}
!2 = !{i32 2, !"julia.debug_level", i32 2}

$ ./julia -e 'using InteractiveUtils; code_llvm(dump_module=true, optimize=true, ()) do; return Ref(Core.Compiler.EFFECTS_TOTAL); end'
; ModuleID = '#3'
source_filename = "#3"
target datalayout = "e-m:e-p:32:32-p270:32:32-p271:32:32-p272:64:64-f64:32:64-f80:32-n8:16:32-S128"
target triple = "i686-unknown-linux-gnu"

@"+Main.Base.RefValue#428.jit" = private alias {}, inttoptr (i32 -435118336 to {}*)

; Function Signature: #3()
;  @ none:1 within `#3`
define nonnull {}* @"julia_#3_426"() #0 {
top:
  %thread_ptr = call i8* asm "movl %gs:0, $0", "=r"() #6
  %tls_ppgcstack = getelementptr i8, i8* %thread_ptr, i32 -4
  %0 = bitcast i8* %tls_ppgcstack to {}****
  %tls_pgcstack = load {}***, {}**** %0, align 4
; ┌ @ refpointer.jl:143 within `Ref`
; │┌ @ refvalue.jl:10 within `RefValue` @ refvalue.jl:8
    %ptls_field4 = getelementptr inbounds {}**, {}*** %tls_pgcstack, i32 2
    %1 = bitcast {}*** %ptls_field4 to i8**
    %ptls_load56 = load i8*, i8** %1, align 4
    %"new::RefValue" = call noalias nonnull align 4 dereferenceable(12) {}* @ijl_gc_pool_alloc_instrumented(i8* %ptls_load56, i32 404, i32 12, i32 -435118336) #5
    %2 = bitcast {}* %"new::RefValue" to i32*
    %"new::RefValue.tag_addr" = getelementptr inbounds i32, i32* %2, i32 -1
    store atomic i32 -435118336, i32* %"new::RefValue.tag_addr" unordered, align 4
    %3 = bitcast {}* %"new::RefValue" to i64*
    store i64 72057598349737984, i64* %3, align 1
; └└
  ret {}* %"new::RefValue"
}

; Function Attrs: noinline optnone
define nonnull {}* @"jfptr_#3_427"({}* %"function::Core.Function", {}** noalias nocapture noundef readonly %"args::Any[]", i32 %"nargs::UInt32") #1 {
top:
  %0 = call nonnull {}* @"julia_#3_426"()
  ret {}* %0
}

; Function Attrs: nounwind willreturn allockind("alloc,uninitialized") allocsize(1)
declare noalias nonnull {}* @julia.gc_alloc_obj({}**, i32, {}*) #2

; Function Attrs: argmemonly nocallback nofree nosync nounwind willreturn
declare void @llvm.lifetime.start.p0i8(i64 immarg, i8* nocapture) #3

; Function Attrs: argmemonly nocallback nofree nosync nounwind willreturn
declare void @llvm.lifetime.end.p0i8(i64 immarg, i8* nocapture) #3

; Function Attrs: nounwind willreturn allockind("alloc,uninitialized") allocsize(1)
declare noalias nonnull {}* @julia.gc_alloc_bytes(i8*, i32, i32) #2

; Function Attrs: inaccessiblemem_or_argmemonly
declare void @ijl_gc_queue_root({}*) #4

; Function Attrs: nounwind willreturn allockind("alloc,uninitialized") allocsize(2)
declare noalias nonnull {}* @ijl_gc_pool_alloc_instrumented(i8*, i32, i32, i32) #5

; Function Attrs: nounwind willreturn allockind("alloc,uninitialized") allocsize(1)
declare noalias nonnull {}* @ijl_gc_big_alloc_instrumented(i8*, i32, i32) #2

; Function Attrs: nounwind willreturn allockind("alloc,uninitialized") allocsize(1)
declare noalias nonnull {}* @ijl_gc_alloc_typed(i8*, i32, i32) #2

attributes #0 = { "frame-pointer"="all" "julia.fsig"="#3()" "probe-stack"="inline-asm" }
attributes #1 = { noinline optnone "frame-pointer"="all" "probe-stack"="inline-asm" }
attributes #2 = { nounwind willreturn allockind("alloc,uninitialized") allocsize(1) }
attributes #3 = { argmemonly nocallback nofree nosync nounwind willreturn }
attributes #4 = { inaccessiblemem_or_argmemonly }
attributes #5 = { nounwind willreturn allockind("alloc,uninitialized") allocsize(2) }
attributes #6 = { nounwind }

!llvm.module.flags = !{!0, !1, !2}

!0 = !{i32 2, !"Dwarf Version", i32 4}
!1 = !{i32 2, !"Debug Info Version", i32 3}
!2 = !{i32 2, !"julia.debug_level", i32 2}

@staticfloat
Copy link
Sponsor Member Author

I followed your steps as well on one of the pibot machines, I got a very similar result:

(gdb) disas /r $pc-32,+64
Dump of assembler code from 0xe52ae198 to 0xe52ae1d8:
   0xe52ae198:  04 30 01 e5     str     r3, [r1, #-4]
   0xe52ae19c:  00 c0 81 e5     str     r12, [r1]
   0xe52ae1a0:  01 30 a0 e1     mov     r3, r1
   0xe52ae1a4:  0c c0 a3 e5     str     r12, [r3, #12]!
   0xe52ae1a8:  5b f0 7f f5     dmb     ish
   0xe52ae1ac:  00 40 81 e5     str     r4, [r1]
   0xe52ae1b0:  63 ca 0d e3     movw    r12, #55907     ; 0xda63
   0xe52ae1b4:  0e c0 4f e3     movt    r12, #61454     ; 0xf00e
=> 0xe52ae1b8:  00 0b dc ed     vldr    d16, [r12]
   0xe52ae1bc:  01 0b c1 ed     vstr    d16, [r1, #4]
   0xe52ae1c0:  5b f0 7f f5     dmb     ish
   0xe52ae1c4:  00 20 83 e5     str     r2, [r3]
   0xe52ae1c8:  18 27 02 e3     movw    r2, #10008      ; 0x2718
   0xe52ae1cc:  1f 20 4f e3     movt    r2, #61471      ; 0xf01f
   0xe52ae1d0:  00 60 92 e5     ldr     r6, [r2]
   0xe52ae1d4:  ac 12 8d e5     str     r1, [sp, #684]  ; 0x2ac

Mine showed pc actually at the vldr, and if I inspect $r12 I get:

(gdb) p jl_($r12)        
<?#0xf00eda63::<?#0x747260::Core.Binding(value=Core.Compiler.widenlattice, globalref=Core.Compiler.widenlattice, owner=<circular reference @-1>, ty=#<null>, flags=0x01)>>

@eymay
Copy link

eymay commented Jan 29, 2024

Hi, I get a build error similar to the one below while building on host with docker buildx:

Alright, next up, a bus error during bootstrap:

[10767] signal (7.1): Bus error
in expression starting at compiler/bootstrap.jl:8
unknown function (ip: 0xee648e0c)
unknown function (ip: 0xee648f37)
_jl_invoke at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:2921 [inlined]
ijl_apply_generic at /cache/build/default-pibot5-0/julialang/julia-master/src/gf.c:3103
unknown function (ip: 0xee69338b)
...

What I get when I cancel a hang:

$ MARCH="armv7-a" CC=clang CXX=clang++  make debug -j25
JULIA usr/lib/julia/corecompiler.ji
^Cerror during bootstrap:
LoadError(at "compiler/compiler.jl" line 3: LoadError(at "compiler/optimize.jl" line 178: LoadError(at "compiler/ssair/inlining.jl" line 214: InterruptException())))
jl_ast_ctx_leave at /julia-src/src/ast.c:319
ijl_expand_with_loc_warn at /julia-src/src/ast.c:1318
jl_parse_eval_all at /julia-src/src/toplevel.c:1062
ijl_load_ at /julia-src/src/toplevel.c:1112
unknown function (ip: 0x3786c263)
jl_fptr_args at /julia-src/src/gf.c:2548
_jl_invoke at /julia-src/src/gf.c:2906
ijl_apply_generic at /julia-src/src/gf.c:3088
unknown function (ip: 0x3786d0f7)
jl_fptr_args at /julia-src/src/gf.c:2548
_jl_invoke at /julia-src/src/gf.c:2906
ijl_apply_generic at /julia-src/src/gf.c:3088
jl_apply at /julia-src/src/./julia.h:2150
do_call at /julia-src/src/interpreter.c:126
eval_value at /julia-src/src/interpreter.c:223
eval_stmt_value at /julia-src/src/interpreter.c:174
eval_body at /julia-src/src/interpreter.c:638
jl_interpret_toplevel_thunk at /julia-src/src/interpreter.c:796
top-level scope at compiler/optimize.jl:178
jl_toplevel_eval_flex at /julia-src/src/toplevel.c:941
jl_parse_eval_all at /julia-src/src/toplevel.c:1065
ijl_load_ at /julia-src/src/toplevel.c:1112
unknown function (ip: 0x3786c263)
jl_fptr_args at /julia-src/src/gf.c:2548
_jl_invoke at /julia-src/src/gf.c:2906
ijl_apply_generic at /julia-src/src/gf.c:3088
unknown function (ip: 0x3786d0f7)
jl_fptr_args at /julia-src/src/gf.c:2548
_jl_invoke at /julia-src/src/gf.c:2906
ijl_apply_generic at /julia-src/src/gf.c:3088
jl_apply at /julia-src/src/./julia.h:2150
do_call at /julia-src/src/interpreter.c:126
eval_value at /julia-src/src/interpreter.c:223
eval_stmt_value at /julia-src/src/interpreter.c:174
eval_body at /julia-src/src/interpreter.c:638
jl_interpret_toplevel_thunk at /julia-src/src/interpreter.c:796
top-level scope at compiler/compiler.jl:216
jl_toplevel_eval_flex at /julia-src/src/toplevel.c:941
jl_eval_module_expr at /julia-src/src/toplevel.c:215
jl_toplevel_eval_flex at /julia-src/src/toplevel.c:741
ijl_toplevel_eval at /julia-src/src/toplevel.c:950
ijl_toplevel_eval_in at /julia-src/src/toplevel.c:992
unknown function (ip: 0x3786b91f)
jl_fptr_args at /julia-src/src/gf.c:2548
_jl_invoke at /julia-src/src/gf.c:2906
ijl_apply_generic at /julia-src/src/gf.c:3088
jl_apply at /julia-src/src/./julia.h:2150
do_call at /julia-src/src/interpreter.c:126
eval_value at /julia-src/src/interpreter.c:223
eval_stmt_value at /julia-src/src/interpreter.c:174
eval_body at /julia-src/src/interpreter.c:638
jl_interpret_toplevel_thunk at /julia-src/src/interpreter.c:796
top-level scope at compiler/compiler.jl:3
jl_toplevel_eval_flex at /julia-src/src/toplevel.c:941
jl_parse_eval_all at /julia-src/src/toplevel.c:1065
ijl_load_ at /julia-src/src/toplevel.c:1112
ijl_load at /julia-src/src/toplevel.c:1125
exec_program at /julia-src/src/jlapi.c:544
true_main at /julia-src/src/jlapi.c:601
jl_repl_entrypoint at /julia-src/src/jlapi.c:738
jl_load_repl at /julia-src/cli/loader_lib.c:569
main at /julia-src/cli/loader_exe.c:58
unknown function (ip: 0x3f6e02d9)
__libc_start_main at /lib/arm-linux-gnueabihf/libc.so.6 (unknown line)
_start at /julia-src/usr/bin/julia-debug (unknown line)

make[1]: *** [sysimage.mk:64: /julia-src/usr/lib/julia/corecompiler.ji] Interrupt
make: *** [Makefile:103: julia-sysimg-ji] Interrupt

What might be interesting is when I follow a similar procedure on Raspberry Pi 4, I get more informative segfaults.

$ uname -a
Linux raspberrypi 6.1.0-rpi7-rpi-v8 #1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24) aarch64 GNU/Linux

$ CC=clang CXX=clang++ VERBOSE=1 make debug
make[1]: Leaving directory '/home/eymen/clones/julia/src'
make[1]: Entering directory '/home/eymen/clones/julia'
 cd /home/eymen/clones/julia/base && /home/eymen/clones/julia/usr/bin/julia-debug -C "native"  --output-ji /home/eymen/clones/julia/usr/lib/julia/corecompiler.ji.tmp --startup-file=no --warn-overwrite=yes -g0 -O0 compiler/compiler.jl

[21922] signal 11 (1): Segmentation fault
in expression starting at boot.jl:263
getNext at /home/eymen/clones/julia/usr/include/llvm/ADT/ilist_node_base.h:29
getNext at /home/eymen/clones/julia/usr/include/llvm/ADT/ilist_node.h:67
operator++ at /home/eymen/clones/julia/usr/include/llvm/ADT/ilist_iterator.h:157
incrementHelper<1U> at /home/eymen/clones/julia/usr/include/llvm/ADT/STLExtras.h:954
increment<0U, 1U> at /home/eymen/clones/julia/usr/include/llvm/ADT/STLExtras.h:968
operator++ at /home/eymen/clones/julia/usr/include/llvm/ADT/STLExtras.h:1015
operator++ at /home/eymen/clones/julia/usr/include/llvm/ADT/iterator.h:161
operator* at /home/eymen/clones/julia/usr/include/llvm/ADT/STLExtras.h:583
operator() at /home/eymen/clones/julia/src/jitlayers.cpp:708
withModuleDo<(lambda at /home/eymen/clones/julia/src/jitlayers.cpp:706:33)> at /home/eymen/clones/julia/usr/include/llvm/ExecutionEngine/Orc/ThreadSafeModule.h:136
validateExternRelocations at /home/eymen/clones/julia/src/jitlayers.cpp:706
CallImpl<llvm::Expected<llvm::orc::ThreadSafeModule> (*)(llvm::orc::ThreadSafeModule, llvm::orc::MaterializationResponsibility &)> at /home/eymen/clones/julia/usr/include/llvm/ADT/FunctionExtras.h:222
_ZN4llvm3orc16IRTransformLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so (unknown line)
Allocations: 0 (Pool: 0; Big: 0); GC: 0
Segmentation fault
make[1]: *** [sysimage.mk:64: /home/eymen/clones/julia/usr/lib/julia/corecompiler.ji] Error 139
make[1]: Leaving directory '/home/eymen/clones/julia'
make: *** [Makefile:103: julia-sysimg-ji] Error 2

When it's run with gdb:

$ cd /home/eymen/clones/julia/base
$ gdb --args /home/eymen/clones/julia/usr/bin/julia-debug -C "native"  --output-ji /home/eymen/clones/julia/usr/lib/julia/corecompiler.ji.tmp --startup-file=no --warn-overwrite=yes -g0 -O0 compiler/compiler.jl

Thread 1 "julia-debug" received signal SIGSEGV, Segmentation fault.
llvm::ilist_node_base<false>::getNext (this=0x0) at /home/eymen/clones/julia/usr/include/llvm/ADT/ilist_node_base.h:29
29	  ilist_node_base *getNext() const { return Next; }
(gdb) bt
#0  llvm::ilist_node_base<false>::getNext (this=0x0) at /home/eymen/clones/julia/usr/include/llvm/ADT/ilist_node_base.h:29
#1  0xf75ab60c in llvm::ilist_node_impl<llvm::ilist_detail::node_options<llvm::GlobalVariable, false, false, void> >::getNext (this=0x0)
    at /home/eymen/clones/julia/usr/include/llvm/ADT/ilist_node.h:67
#2  0xf75a9d9c in llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::GlobalVariable, false, false, void>, false, false>::operator++ (this=0xfffeda94)
    at /home/eymen/clones/julia/usr/include/llvm/ADT/ilist_iterator.h:157
#3  0xf75ac1a8 in llvm::concat_iterator<llvm::GlobalObject, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Function, false, false, void>, false, false>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::GlobalVariable, false, false, void>, false, false> >::incrementHelper<1u> (this=0xfffeda94)
    at /home/eymen/clones/julia/usr/include/llvm/ADT/STLExtras.h:954
#4  0xf75ac0a4 in llvm::concat_iterator<llvm::GlobalObject, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Function, false, false, void>, false, false>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::GlobalVariable, false, false, void>, false, false> >::increment<0u, 1u> (this=0xfffeda94)
    at /home/eymen/clones/julia/usr/include/llvm/ADT/STLExtras.h:968
#5  0xf75aa040 in llvm::concat_iterator<llvm::GlobalObject, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Function, false, false, void>, false, false>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::GlobalVariable, false, false, void>, false, false> >::operator++ (this=0xfffeda94) at /home/eymen/clones/julia/usr/include/llvm/ADT/STLExtras.h:1015
#6  0xf75b5df0 in llvm::iterator_facade_base<llvm::concat_iterator<llvm::GlobalObject, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Function, false, false, void>, false, false>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::GlobalVariable, false, false, void>, false, false> >, std::forward_iterator_tag, llvm::GlobalObject, int, llvm::GlobalObject*, llvm::GlobalObject&>::operator++ (this=0xfffeda94) at /home/eymen/clones/julia/usr/include/llvm/ADT/iterator.h:161
#7  0xf75b5950 in llvm::early_inc_iterator_impl<llvm::concat_iterator<llvm::GlobalObject, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Function, false, false, void>, false, false>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::GlobalVariable, false, false, void>, false, false> > >::operator* (this=0xfffeda94)
    at /home/eymen/clones/julia/usr/include/llvm/ADT/STLExtras.h:583
#8  0xf759b994 in validateExternRelocations(llvm::orc::ThreadSafeModule, llvm::orc::MaterializationResponsibility&)::$_2::operator()(llvm::Module&) const (this=0xfffedb6c, M=...)
    at jitlayers.cpp:708
#9  0xf759b8c0 in llvm::orc::ThreadSafeModule::withModuleDo<validateExternRelocations(llvm::orc::ThreadSafeModule, llvm::orc::MaterializationResponsibility&)::$_2>(validateExternRelocations(llvm::orc::ThreadSafeModule, llvm::orc::MaterializationResponsibility&)::$_2&&) (this=0xfffedb90, F=...)
    at /home/eymen/clones/julia/usr/include/llvm/ExecutionEngine/Orc/ThreadSafeModule.h:136
#10 0xf75995c8 in validateExternRelocations (TSM=..., R=...) at jitlayers.cpp:706
#11 0xf75c9ed4 in llvm::detail::UniqueFunctionBase<llvm::Expected<llvm::orc::ThreadSafeModule>, llvm::orc::ThreadSafeModule, llvm::orc::MaterializationResponsibility&>::CallImpl<llvm::Expected<llvm::orc::ThreadSafeModule> (*)(llvm::orc::ThreadSafeModule, llvm::orc::MaterializationResponsibility&)> (CallableAddr=0x456524, Params=..., Params=...)
    at /home/eymen/clones/julia/usr/include/llvm/ADT/FunctionExtras.h:222
#12 0xf588fef4 in llvm::orc::IRTransformLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, llvm::orc::ThreadSafeModule) () from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#13 0xf5899af8 in llvm::orc::BasicIRLayerMaterializationUnit::materialize(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >) () from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#14 0xf5840328 in llvm::orc::MaterializationTask::run() () from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#15 0xf5827774 in void llvm::detail::UniqueFunctionBase<void, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> > >::CallImpl<void (*)(std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >)>(void*, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >&) () from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#16 0xf584045c in llvm::orc::ExecutionSession::dispatchOutstandingMUs() () from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#17 0xf5846730 in llvm::orc::ExecutionSession::OL_completeLookup(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >, std::shared_ptr<llvm::orc::AsynchronousSymbolQuery>, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> >, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> > > > const&)>) () from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#18 0xf5846c3c in llvm::orc::InProgressFullLookupState::complete(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >) ()
   from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#19 0xf5834cd4 in llvm::orc::ExecutionSession::OL_applyQueryPhase1(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >, llvm::Error) ()
   from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#20 0xf5840754 in llvm::orc::ExecutionSession::lookup(llvm::orc::LookupKind, std::vector<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>, std::allocator<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags> > > const&, llvm::orc::SymbolLookupSet, llvm::orc::SymbolState, llvm::unique_function<void (llvm::Expected<llvm::DenseMap<llvm::orc::SymbolStringPt--Type <RET> for more, q to quit, c to continue without paging--
r, llvm::JITEvaluatedSymbol, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, llvm::JITEvaluatedSymbol> > >)>, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> >, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> > > > const&)>) ()
   from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#21 0xf5840938 in llvm::orc::ExecutionSession::lookup(std::vector<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>, std::allocator<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags> > > const&, llvm::orc::SymbolLookupSet, llvm::orc::LookupKind, llvm::orc::SymbolState, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> >, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> > > > const&)>) () from /home/eymen/clones/julia/usr/bin/../lib/libLLVM-15jl.so
#22 0xf7594f04 in JuliaOJIT::addModule (this=0x456078, TSM=...) at jitlayers.cpp:1881
#23 0xf7596040 in _jl_compile_codeinst (codeinst=0xf25e7490, src=0xf25e0b00, world=1, context=..., is_recompile=false) at jitlayers.cpp:227
#24 0xf7596d6c in jl_generate_fptr_for_unspecialized_impl (unspec=0xf25e7490) at jitlayers.cpp:591
#25 0xf77df948 in jl_compile_method_internal (mi=0xf25b7940, world=1) at gf.c:2506
#26 0xf77e2a14 in _jl_invoke (F=0x0, args=0x0, nargs=0, mfunc=0xf25b7940, world=1) at gf.c:2898
#27 0xf77e28d0 in ijl_invoke (F=0x0, args=0x0, nargs=0, mfunc=0xf25b7940) at gf.c:2913
#28 0xf7835540 in jl_toplevel_eval_flex (m=0xf25d0010, e=0xf255cf50, fast=1, expanded=1) at toplevel.c:932
#29 0xf7837960 in jl_parse_eval_all (module=0xf25d0010, text=0x521d00, filename=0xf25d9a70) at toplevel.c:1065
#30 0xf7837510 in ijl_load_ (module=0xf25d0010, filename=0xf25d9a70) at toplevel.c:1112
#31 0xf7837b44 in ijl_load (module=0xf25d0010, filename=0xf7a42d08 "boot.jl") at toplevel.c:1125
#32 0xf781196c in _finish_julia_init (rel=JL_IMAGE_JULIA_HOME, ptls=0x445100, ct=0xf15ec010) at init.c:876
#33 0xf78115d4 in julia_init (rel=JL_IMAGE_JULIA_HOME) at init.c:843
#34 0xf787f45c in jl_repl_entrypoint (argc=1, argv=0xfffef054) at jlapi.c:732
#35 0xf7e06318 in jl_load_repl (argc=10, argv=0xfffef054) at /home/eymen/clones/julia/cli/loader_lib.c:569
#36 0x00400940 in main (argc=10, argv=0xfffef054) at /home/eymen/clones/julia/cli/loader_exe.c:58

@staticfloat
Copy link
Sponsor Member Author

I've rebased in an attempt to see where we stand now that LLVM v16 has been merged.

@staticfloat
Copy link
Sponsor Member Author

Looks like we end up with a new error:

terminate called after throwing an instance of 'std::system_error'
what():  Resource deadlock avoided

@ViralBShah
Copy link
Member

Are we likely to get further if remove neon, vfp4 and all the vector instruction stuff?

staticfloat and others added 4 commits June 19, 2024 10:31
Jameson points out that ARM doesn't use the usual structure for these
registers, and so we have to manually typecast this structure.

X-ref: #47345 (comment)
@staticfloat
Copy link
Sponsor Member Author

staticfloat commented Jul 13, 2024

Jameson and I have been debugging this today; so far we've discovered the following:

  1. The error is due to a lock, taken within withModuleDo. The error occurs because the mutex is of type PTHREAD_MUTEX_ERRORCHECK (2).
  2. The mutex does not start out with that type, it starts out as type PTHREAD_MUTEX_RECURSIVE (1).
  3. The mutex's __kind field gets updated by the following instruction:
   0xf48ae520 <+512>:   b       0xf48ae4b4 <_ZN4llvm3orc21IRMaterializationUnitC2ERNS0_16ExecutionSessionERKNS0_14IRSymbolMapper15ManglingOptionsENS0_16ThreadSafeModuleE+404>
   0xf48ae524 <+516>:   dmb     ish         
   0xf48ae528 <+520>:   add     r3, r5, #28
   0xf48ae52c <+524>:   ldrex   r2, [r3]                                                                                                                                                                              
   0xf48ae530 <+528>:   add     r2, r2, #1                                                                 
=> 0xf48ae534 <+532>:   strex   r1, r2, [r3]                                                                                                                                                                          
   0xf48ae538 <+536>:   cmp     r1, #0                                                                                                                                                                                
   0xf48ae53c <+540>:   bne     0xf48ae52c <_ZN4llvm3orc21IRMaterializationUnitC2ERNS0_16ExecutionSessionERKNS0_14IRSymbolMapper15ManglingOptionsENS0_16ThreadSafeModuleE+524>
  1. We thought this might be a compiler bug as that offset of 28 is really strange, but rebuilding LLVM with GCC 13 didn't change much:
   0xf54177e8 <+344>:   beq     0xf5417854 <_ZN4llvm3orc21IRMaterializationUnitC2ERNS0_16ExecutionSessionERKNS0_14IRSymbolMapper15ManglingOptionsENS0_16ThreadSafeModuleE+452>                                        
   0xf54177ec <+348>:   mov     r0, r4                                                                     
   0xf54177f0 <+352>:   add     sp, sp, #28
   0xf54177f4 <+356>:   pop     {r4, r5, r6, r7, r8, r9, pc}     
   0xf54177f8 <+360>:   dmb     ish   
   0xf54177fc <+364>:   add     r3, r5, #28                                                                                                                                                                           
   0xf5417800 <+368>:   ldrex   r1, [r3]
   0xf5417804 <+372>:   add     r1, r1, #1                                                                                                                                                                            
=> 0xf5417808 <+376>:   strex   r2, r1, [r3]                                                               
   0xf541780c <+380>:   cmp     r2, #0                                                                     
   0xf5417810 <+384>:   bne     0xf5417800 <_ZN4llvm3orc21IRMaterializationUnitC2ERNS0_16ExecutionSessionERKNS0_14IRSymbolMapper15ManglingOptionsENS0_16ThreadSafeModuleE+368>

@vtjnash
Copy link
Sponsor Member

vtjnash commented Jul 15, 2024

Tracked this down. That failure comes from an ABI decision that gets made during the configure of libstdc++ depending on what features it can autodetect of the platform architecture was being used (-march value) at the time. (this is supposedly an improvement from when it used be an ABI decision changed by the -march and other values passed to the compiler, as it at least means different libraries can be linked together correctly if built with the exact same compiler, as was the case before gcc-mirror/gcc@da29d2a)

So to fix, IIUC, you need to configure the compiler with --with-libstdcxx-lock-policy=atomics to prevent the ABI from changing depending on the autodetected behavior for that. And/or ensure that the libstdc++ library is being compiled with atomics correctly enabled for the current architectural target (e.g. we make sure armv6 and armv7hf are using different compiler shards, since they have mutually incompatible ABIs for both atomics and float registers)

@giordano
Copy link
Contributor

(e.g. we make sure armv6 and armv7hf are using different compiler shards, since they have mutually incompatible ABIs for both atomics and float registers)

Ah, at the moment we're using the same toolchain, only with different -march switches

@staticfloat
Copy link
Sponsor Member Author

So it sounds like the path forward here is to rebuild our C++ dependencies (likely just LLVM, in practice) with toolchains that have a libstdc++ configured with the correct ABI. In this case, that means an armv7l-specific toolchain, not reusing the armv6 toolchain with a different -march flag, since that -march flag can't change the ABI baked into the libstdc++ that comes with the compiler. This will likely have to wait until I've gotten BB2 far enough along that I can do that rebuild.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
system:arm ARMv7 and AArch64 system:32-bit Affects only 32-bit systems
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants