Enable JITLink in aarch64 linux. #49745

giordano · 2023-05-10T23:02:52Z

Reland of #45859, which had some conflicts with f11bfc6 (#49530). I hope I got the logic right. CC: @sunho.

Fix #42295, fix #47399, fix #41852, fix #49474.

src/jitlayers.h

giordano · 2023-05-10T23:08:30Z

src/jitlayers.h

 // #define JL_FORCE_JITLINK

 #if defined(_COMPILER_ASAN_ENABLED_) || defined(_COMPILER_MSAN_ENABLED_) || defined(_COMPILER_TSAN_ENABLED_)
 # define HAS_SANITIZER
 #endif
 // The sanitizers don't play well with our memory manager

-#if defined(_OS_DARWIN_) && defined(_CPU_AARCH64_) || defined(JL_FORCE_JITLINK) || JL_LLVM_VERSION >= 150000 && defined(HAS_SANITIZER)
+#if defined(_CPU_AARCH64_) || defined(JL_FORCE_JITLINK) || JL_LLVM_VERSION >= 150000 && defined(HAS_SANITIZER)


@vchuravy @pchintalapudi just to be sure: if we're using a sanitizer with LLVM 15+ you want to force JITLink? I think we may want to remove defined(_CPU_AARCH64_) here and handle that below?

Yes we want JITLink if santitizers are enabled on >= 150000, because we hit a bug in our memory manager for Rtdyld. I'm not opposed to handling that below.

src/jitlayers.h

vchuravy · 2023-05-10T23:24:55Z

@pchintalapudi the only listener integration that is missing is the perf one right? GDB works with JITLink on 15?

pchintalapudi · 2023-05-10T23:27:02Z

Oh we might want #47037 since we need a different plugin for ELF debugger support.

sunho · 2023-05-11T04:41:09Z

Looks good to me regarding the code here!

pchintalapudi · 2023-05-11T17:13:06Z

Might just want to rebase with master since #47037 just merged.

pchintalapudi · 2023-05-11T21:28:43Z

Looks like we're running out of memory of the aarch64 tester?

oschulz · 2023-05-12T09:08:34Z

Would this be a 1.9 backport candidate?

giordano · 2023-05-12T09:13:01Z

This requires LLVM 15, so no.

oschulz · 2023-05-12T09:16:24Z

This requires LLVM 15, so no.

Oh, right! :-) Sorry, should have looked more closely.

minecraft2048 · 2023-05-15T02:30:57Z

Do we have a compiled build for this PR? I want to see if this fixes #49474 as I want to get qemu working

giordano · 2023-05-15T06:51:12Z

Go to the aarch64-linux-gnu build job and open the artifacts tab: https://buildkite.com/julialang/julia-master/builds/23816#01880bd0-6922-44b0-b77c-67fcaf2c1c66

giordano · 2023-06-07T22:02:35Z

I captured with rr on an M1 MacBook running Asahi Linux the crash (for the record, Base.runtests("LinearAlgebra/triangular") is enough to reproduce the error)

      From worker 2:	JIT session error: Cannot allocate memory
      From worker 2:	JIT session error: Cannot allocate memory
      From worker 2:
      From worker 2:	[213] signal (11.1): Segmentation fault
      From worker 2:	in expression starting at /cache/build/default-armageddon-6/julialang/julia-master/julia-ae73ae2440/share/julia/stdlib/v1.10/LinearAlgebra/test/triangular.jl:21

while running

julia/stdlib/LinearAlgebra/test/triangular.jl

Line 21 in 4200aa9

    
           for elty1 in (Float32, Float64, BigFloat, ComplexF32, ComplexF64, Complex{BigFloat}, Int)

For anyone interested with playing with the the trace: trace.tar.xz. It fails on Complex{BigFloat} but removing the other types makes the crash go away.

giordano · 2023-06-07T22:53:07Z

More breadcrumbs: backtrace and strace log. Highlights are

#0  0x0000000070000004 in syscall_traced ()
#1  0x0000ffff77a96ba0 in _raw_syscall () at /workspace/srcdir/rr/src/preload/raw_syscall.S:161
#2  0x0000ffff77a910b8 in traced_raw_syscall (call=0x687ffd50) at /workspace/srcdir/rr/src/preload/syscallbuf.c:338
#3  0x0000ffff77a969a8 in syscall_hook (call=0x687ffd50) at /workspace/srcdir/rr/src/preload/syscallbuf.c:3949
#4  syscall_hook (call=0x687ffd50) at /workspace/srcdir/rr/src/preload/syscallbuf.c:3933
#5  0x0000ffff77a90cc4 in _syscall_hook_trampoline () at /workspace/srcdir/rr/src/preload/syscall_hook.S:763
#6  0x0000ffff77a90dc4 in _syscall_hook_trampoline_raw () at /workspace/srcdir/rr/src/preload/syscall_hook.S:937
#7  0x0000ffff779865bc in mmap64 () from /usr/lib/libc.so.6
#8  0x0000ffff7322374c in llvm::sys::Memory::allocateMappedMemory(unsigned long, llvm::sys::MemoryBlock const*, unsigned int, std::error_code&) ()
   from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#9  0x0000ffff74f616d0 in llvm::orc::InProcessMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#10 0x0000ffff74f585ec in llvm::orc::MapperJITLinkMemoryManager::allocate(llvm::jitlink::JITLinkDylib const*, llvm::jitlink::LinkGraph&, llvm::unique_function<void (llvm::Expected<std::unique_ptr<llvm::jitlink::JITLinkMemoryManager::InFlightAlloc, std::default_delete<llvm::jitlink::JITLinkMemoryManager::InFlightAlloc> > >)>) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#11 0x0000ffff74e85854 in llvm::jitlink::JITLinkerBase::linkPhase1(std::unique_ptr<llvm::jitlink::JITLinkerBase, std::default_delete<llvm::jitlink::JITLinkerBase> >) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#12 0x0000ffff74ea8b28 in llvm::jitlink::link_ELF_aarch64(std::unique_ptr<llvm::jitlink::LinkGraph, std::default_delete<llvm::jitlink::LinkGraph> >, std::unique_ptr<llvm::jitlink::JITLinkContext, std::default_delete<llvm::jitlink::JITLinkContext> >) ()
   from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#13 0x0000ffff74ea6210 in llvm::jitlink::link_ELF(std::unique_ptr<llvm::jitlink::LinkGraph, std::default_delete<llvm::jitlink::LinkGraph> >, std::unique_ptr<llvm::jitlink::JITLinkContext, std::default_delete<llvm::jitlink::JITLinkContext> >) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#14 0x0000ffff74e7daa8 in llvm::jitlink::link(std::unique_ptr<llvm::jitlink::LinkGraph, std::default_delete<llvm::jitlink::LinkGraph> >, std::unique_ptr<llvm::jitlink::JITLinkContext, std::default_delete<llvm::jitlink::JITLinkContext> >) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#15 0x0000ffff74f71bac in llvm::orc::ObjectLinkingLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >) ()
   from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#16 0x0000ffff76d16878 in JuliaOJIT::LockLayerT::emit (this=<optimized out>, R=..., O=...) at /usr/include/c++/12.1.0/bits/unique_ptr.h:189
#17 0x0000ffff74f33778 in llvm::orc::IRCompileLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, llvm::orc::ThreadSafeModule) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#18 0x0000ffff74f34078 in llvm::orc::IRTransformLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, llvm::orc::ThreadSafeModule) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#19 0x0000ffff76d0b550 in JuliaOJIT::OptSelLayerT::emit (this=0x8000, R=..., TSM=...) at /home/mose/repo/julia/src/jitlayers.cpp:690
#20 0x0000ffff74f3ce7c in llvm::orc::BasicIRLayerMaterializationUnit::materialize(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#21 0x0000ffff74eecb60 in llvm::orc::MaterializationTask::run() () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#22 0x0000ffff74ed4534 in void llvm::detail::UniqueFunctionBase<void, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> > >::CallImpl<void (*)(std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >)>(void*, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >&) () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so
#23 0x0000ffff74eecc78 in llvm::orc::ExecutionSession::dispatchOutstandingMUs() () from /home/mose/repo/julia/usr/bin/../lib/libLLVM-15jl.so

and

mmap(NULL, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffff01528000
mprotect(0xffff01528000, 16384, PROT_READ|PROT_WRITE) = 0
mprotect(0xffff0152c000, 16384, PROT_READ|PROT_EXEC) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [INT QUIT USR1 TERM], 8) = 0
rt_sigprocmask(SIG_SETMASK, [INT QUIT USR1 TERM], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [INT QUIT USR1 TERM], 8) = 0
rt_sigprocmask(SIG_SETMASK, [INT QUIT USR1 TERM], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [INT QUIT USR1 TERM], 8) = 0
rt_sigprocmask(SIG_SETMASK, [INT QUIT USR1 TERM], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [INT QUIT USR1 TERM], 8) = 0
rt_sigprocmask(SIG_SETMASK, [INT QUIT USR1 TERM], NULL, 8) = 0
futex(0xaaaba572dd58, FUTEX_WAKE_PRIVATE, 2147483647) = 0
mmap(NULL, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffff01520000
mprotect(0xffff01520000, 16384, PROT_READ|PROT_WRITE) = 0
mprotect(0xffff01524000, 16384, PROT_READ|PROT_EXEC) = -1 ENOMEM (Cannot allocate memory)
write(2, "JIT session error: ", 19JIT session error: )     = 19
write(2, "Cannot allocate memory", 22Cannot allocate memory)  = 22
write(2, "\n", 1
)                       = 1
futex(0xaaaba5823798, FUTEX_WAKE_PRIVATE, 2147483647) = 0
--- SIGBUS {si_signo=SIGBUS, si_code=BUS_ADRALN, si_addr=0xaaaba47bd} ---

giordano · 2023-06-08T21:45:45Z

Reporting here the progress discussed on Slack. @gbaraldi figured out what was the problem: this issue in Java reported a similar ENOMEM error on mprotect, and it was suggested that the the application was creating too many mappings, exceeding the limit in /proc/sys/vm/max_map_count. He also noticed that running the LinearAlgebra tests creates over 71k mappings (wc -l /proc/$$/maps, where $$ is the PID of the process), while in many systems the default limit is 65530. We increased the limit of mappings on the CI machine with sysctl -w vm.max_map_count=262144 (not sure what's the value actually used) and the restarted job is finally passing all the tests 🥳 This problem is not specific to aarch64, also using JITLink on x86_64 results in several thousands of mappings open. As far as I understand this is an upstream bug in LLVM (?) memory manager.

Not sure where to go from here. Document the possibility of having to increase the memory mappings limit and merge this PR? Linux aarch64 is tier 2 and currently suffers from many segmentation faults anyway (although those should be solved by JITLink, enabled with this PR), so situation wouldn't be much worse.

pchintalapudi · 2023-06-08T22:06:48Z

I'm in favor of merging and documenting the memory mapping limit issue+resolution in the release notes. The memory manager issues aren't likely to be fixed in the near term, and resource limits are easier to fix than segmentation faults.

gbaraldi · 2023-06-08T23:40:09Z

Yeah, I'm fine with merging too. The issue is common to all platforms that use JITLink, it seems it just reared it's head here, the solution for the user is pretty easy, but it probably requires quite a bit of LLVM work.

vchuravy · 2023-06-09T00:37:46Z

Agreed! We should add a news entry, and additional an entry in the docs.

Do we have an upstream issue number?

gbaraldi · 2023-06-09T01:10:50Z

There isn't an upstream issue just yet.

giordano · 2023-06-09T19:10:09Z

and additional an entry in the docs.

Do you refer to the explanation of the work around potentially needed to increase the limit of mappings? Where do you suggest to place it specifically?

There isn't an upstream issue just yet.

Is there one now?

vtjnash · 2023-06-09T19:57:37Z

This is not an issue with our existing cgmemmgr, so it seems like this is something that should be fixed upstream

giordano · 2023-06-10T00:15:24Z

This is not an issue with our existing cgmemmgr, so it seems like this is something that should be fixed upstream

Yeah, I was wondering if @gbaraldi opened a ticket upstream in llvm so that we can add a reference to it in the docs.

giordano · 2023-06-11T15:31:04Z

I added the NEWS entry and mentioned the ticket opened by Gabriel in the ARM build devdocs page. Please check if it's all OK.

Anything else to do this for this PR? 🙂

gbaraldi · 2023-06-11T15:34:30Z

LGTM

maleadt · 2023-06-14T08:19:47Z

FWIW, I'm still seeing an occasional segfault on a Jetson (linux-aarch64, NVIDIA ARM chip):

Thread 3 "julia" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffe309a1d0 (LWP 7227)]
0x0000fffff39829a0 in (anonymous namespace)::GetCFGOnlyPasses::passEnumerate(llvm::PassInfo const*) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
(gdb) bt
#0  0x0000fffff39829a0 in (anonymous namespace)::GetCFGOnlyPasses::passEnumerate(llvm::PassInfo const*) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#1  0x0000fffff3991c3c in llvm::PassRegistry::enumerateWith(llvm::PassRegistrationListener*) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#2  0x0000fffff3983c88 in llvm::AnalysisUsage::setPreservesCFG() () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#3  0x0000fffff46b0868 in (anonymous namespace)::ConstantHoistingLegacyPass::getAnalysisUsage(llvm::AnalysisUsage&) const () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#4  0x0000fffff3946368 in llvm::PMTopLevelManager::findAnalysisUsage(llvm::Pass*) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#5  0x0000fffff394b064 in llvm::PMTopLevelManager::schedulePass(llvm::Pass*) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#6  0x0000fffff3e312f4 in llvm::TargetPassConfig::addPass(llvm::Pass*) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#7  0x0000fffff3e34c64 in llvm::TargetPassConfig::addIRPasses() () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#8  0x0000fffff5c9f1e0 in (anonymous namespace)::AArch64PassConfig::addIRPasses() () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#9  0x0000fffff3e35cbc in llvm::TargetPassConfig::addISelPasses() () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#10 0x0000fffff3b6a4a8 in llvm::LLVMTargetMachine::addPassesToEmitMC(llvm::legacy::PassManagerBase&, llvm::MCContext*&, llvm::raw_pwrite_stream&, bool) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#11 0x0000fffff54a5fe8 in llvm::orc::SimpleCompiler::operator()(llvm::Module&) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#12 0x0000fffff72d288c in (anonymous namespace)::CompilerT::operator() (this=0xaaaaaaae49e0, M=...) at /usr/include/c++/9/bits/move.h:99
#13 0x0000fffff55058b4 in llvm::orc::IRCompileLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, llvm::orc::ThreadSafeModule) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#14 0x0000fffff5506078 in llvm::orc::IRTransformLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >, llvm::orc::ThreadSafeModule) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#15 0x0000fffff72d4f78 in JuliaOJIT::OptSelLayerT::emit (this=0xffffbe53e428, R=..., TSM=...) at /usr/include/c++/9/bits/shared_ptr_base.h:1176
#16 0x0000fffff550ee7c in llvm::orc::BasicIRLayerMaterializationUnit::materialize(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility> >) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#17 0x0000fffff54beb60 in llvm::orc::MaterializationTask::run() () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#18 0x0000fffff54a6534 in void llvm::detail::UniqueFunctionBase<void, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> > >::CallImpl<void (*)(std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >)>(void*, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task> >&) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#19 0x0000fffff54bec78 in llvm::orc::ExecutionSession::dispatchOutstandingMUs() () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#20 0x0000fffff54c3030 in llvm::orc::ExecutionSession::OL_completeLookup(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >, std::shared_ptr<llvm::orc::AsynchronousSymbolQuery>, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> >, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> > > > const&)>) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#21 0x0000fffff54c3e2c in llvm::orc::InProgressFullLookupState::complete(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#22 0x0000fffff54b4708 in llvm::orc::ExecutionSession::OL_applyQueryPhase1(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState> >, llvm::Error) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#23 0x0000fffff54bef04 in llvm::orc::ExecutionSession::lookup(llvm::orc::LookupKind, std::vector<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>, std::allocator<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags> > > const&, llvm::orc::SymbolLookupSet, llvm::orc::SymbolState, llvm::unique_function<void (llvm::Expected<llvm::DenseMap<llvm::orc::SymbolStringPtr, llvm::JITEvaluatedSymbol, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, llvm::JITEvaluatedSymbol> > >)>, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> >, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> > > > const&)>) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#24 0x0000fffff54bf0b8 in llvm::orc::ExecutionSession::lookup(std::vector<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>, std::allocator<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags> > > const&, llvm::orc::SymbolLookupSet, llvm::orc::LookupKind, llvm::orc::SymbolState, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> >, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void> > > > const&)>) () from /home/tim/Julia/src/julia/build/usr/bin/../lib/libLLVM-15jl.so
#25 0x0000fffff72d7024 in JuliaOJIT::addModule (this=0xaaaaaab35150, TSM=...) at /usr/include/c++/9/bits/std_function.h:564
#26 0x0000fffff72dba68 in jl_add_to_ee (M=..., NewExports=..., Queued=..., Stack=...) at /usr/include/c++/9/bits/shared_ptr_base.h:1176
#27 0x0000fffff72dc830 in _jl_compile_codeinst (codeinst=<optimized out>, codeinst@entry=0xffff3cc0a070, src=src@entry=0xffff3cc10010, world=world@entry=30687, context=..., is_recompile=false) at /home/tim/Julia/src/julia/src/jitlayers.cpp:243
#28 0x0000fffff72ddf90 in jl_generate_fptr_impl (mi=<optimized out>, world=30687) at /usr/include/c++/9/ext/atomicity.h:96
#29 0x0000fffff74baed4 in jl_compile_method_internal (world=30687, mi=<optimized out>) at /home/tim/Julia/src/julia/src/gf.c:2475
#30 jl_compile_method_internal (mi=<optimized out>, world=30687) at /home/tim/Julia/src/julia/src/gf.c:2364
#31 0x0000fffff74bc008 in _jl_invoke (world=<optimized out>, mfunc=0xffff3cbe41f0, nargs=11, args=0xffffbe53fca0, F=0xffffe8265730 <jl_system_image_data+63448240>) at /home/tim/Julia/src/julia/src/gf.c:2881
#32 ijl_apply_generic (F=0xffffe8265730 <jl_system_image_data+63448240>, args=0xffffbe53fca0, nargs=11) at /home/tim/Julia/src/julia/src/gf.c:3071
#33 0x0000ffffe41ae898 in macro expansion () at logging.jl:365
#34 julia_profile_printing_listener_81356 () at Base.jl:574
#35 0x0000ffffe4425eb8 in julia_#1039_81417 () at Base.jl:598
#36 0x0000ffffe41aec64 in jfptr_YY.1039_81418 () from /home/tim/Julia/src/julia/build/usr/lib/julia/sys.so
#37 0x0000fffff74bbdf0 in _jl_invoke (world=<optimized out>, mfunc=0xffffe61e9b30 <jl_system_image_data+29386928>, nargs=0, args=0xffffedb40238, F=0xffffe61e98c0 <jl_system_image_data+29386304>) at /home/tim/Julia/src/julia/src/gf.c:2869
#38 ijl_apply_generic (/build/gdb-S79MMs/gdb-9.2/gdb/inline-frame.c:159: internal-error: void inline_frame_this_id(frame_info*, void**, frame_id*): Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.

This seems to happen when exiting the process, but haven't been able to debug yet because GDB crashes (see the last line, that assertion is GDB's). So I'm not sure it's related, but it does at least point to the JIT.

gbaraldi · 2023-06-14T10:18:27Z

That segfault is happening everywhere :(

mgood7123 · 2023-10-09T12:38:04Z

I'm in favor of merging and documenting the memory mapping limit issue+resolution in the release notes. The memory manager issues aren't likely to be fixed in the near term, and resource limits are easier to fix than segmentation faults.

tho this is not simple to do on non-rooted android, right?

additionally does windows allow you to increase such limit?

giordano added system:arm ARMv7 and AArch64 compiler:llvm For issues that relate to LLVM labels May 10, 2023

giordano requested review from vchuravy and pchintalapudi May 10, 2023 23:02

giordano commented May 10, 2023

View reviewed changes

giordano mentioned this pull request May 10, 2023

Enable JITLink on aarch64 linux. #45859

Closed

pchintalapudi reviewed May 10, 2023

View reviewed changes

src/jitlayers.h Show resolved Hide resolved

pchintalapudi approved these changes May 11, 2023

View reviewed changes

minecraft2048 mentioned this pull request May 15, 2023

BUG: Segmentation fault when cross-compiling from amd64 to arm64 using qemu emulation #49474

Closed

giordano added the rr trace included label Jun 7, 2023

vchuravy added needs news A NEWS entry is required for this change needs docs Documentation for this change is required labels Jun 9, 2023

This was referenced Jun 10, 2023

Segfault during JIT memory invalidation #41852

Closed

Julia fails on Chrome-OS and NVIDIA Jetson ARM-64 Linux #47399

Closed

gbaraldi mentioned this pull request Jun 10, 2023

orc::InProcessMemoryMapper makes too many memory mappings llvm/llvm-project#63236

Open

sunho and others added 4 commits June 11, 2023 16:29

Enable JITLink in aarch64 linux.

2d31065

Simplify logic to enable JITLink

e7dd7ab

Do not enable JITLink on Aarch64 Linux with LLVM < 15

7da6e16

Add NEWS entry and mention JITLink memory bug in ARM devdocs

e3fd8dc

giordano force-pushed the mg/sunho-jitlinkk branch from ae73ae2 to e3fd8dc Compare June 11, 2023 15:29

giordano removed needs docs Documentation for this change is required needs news A NEWS entry is required for this change labels Jun 11, 2023

pchintalapudi approved these changes Jun 13, 2023

View reviewed changes

giordano merged commit 7cd7a1b into JuliaLang:master Jun 13, 2023

giordano deleted the mg/sunho-jitlinkk branch June 13, 2023 06:54

palday mentioned this pull request Aug 17, 2023

Segfault on M1 when loading Python Package MNE. JuliaPy/PythonCall.jl#315

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable JITLink in aarch64 linux. #49745

Enable JITLink in aarch64 linux. #49745

giordano commented May 10, 2023 •

edited

Loading

giordano May 10, 2023

pchintalapudi May 10, 2023

vchuravy commented May 10, 2023

pchintalapudi commented May 10, 2023

sunho commented May 11, 2023

pchintalapudi commented May 11, 2023

pchintalapudi commented May 11, 2023

oschulz commented May 12, 2023

giordano commented May 12, 2023

oschulz commented May 12, 2023

minecraft2048 commented May 15, 2023

giordano commented May 15, 2023

giordano commented Jun 7, 2023 •

edited

Loading

giordano commented Jun 7, 2023 •

edited

Loading

giordano commented Jun 8, 2023 •

edited

Loading

pchintalapudi commented Jun 8, 2023

gbaraldi commented Jun 8, 2023

vchuravy commented Jun 9, 2023

gbaraldi commented Jun 9, 2023

giordano commented Jun 9, 2023 •

edited

Loading

vtjnash commented Jun 9, 2023

giordano commented Jun 10, 2023

giordano commented Jun 11, 2023

gbaraldi commented Jun 11, 2023

maleadt commented Jun 14, 2023

gbaraldi commented Jun 14, 2023

mgood7123 commented Oct 9, 2023 •

edited

Loading

Enable JITLink in aarch64 linux. #49745

Enable JITLink in aarch64 linux. #49745

Conversation

giordano commented May 10, 2023 • edited Loading

giordano May 10, 2023

Choose a reason for hiding this comment

pchintalapudi May 10, 2023

Choose a reason for hiding this comment

vchuravy commented May 10, 2023

pchintalapudi commented May 10, 2023

sunho commented May 11, 2023

pchintalapudi commented May 11, 2023

pchintalapudi commented May 11, 2023

oschulz commented May 12, 2023

giordano commented May 12, 2023

oschulz commented May 12, 2023

minecraft2048 commented May 15, 2023

giordano commented May 15, 2023

giordano commented Jun 7, 2023 • edited Loading

giordano commented Jun 7, 2023 • edited Loading

giordano commented Jun 8, 2023 • edited Loading

pchintalapudi commented Jun 8, 2023

gbaraldi commented Jun 8, 2023

vchuravy commented Jun 9, 2023

gbaraldi commented Jun 9, 2023

giordano commented Jun 9, 2023 • edited Loading

vtjnash commented Jun 9, 2023

giordano commented Jun 10, 2023

giordano commented Jun 11, 2023

gbaraldi commented Jun 11, 2023

maleadt commented Jun 14, 2023

gbaraldi commented Jun 14, 2023

mgood7123 commented Oct 9, 2023 • edited Loading

giordano commented May 10, 2023 •

edited

Loading

giordano commented Jun 7, 2023 •

edited

Loading

giordano commented Jun 7, 2023 •

edited

Loading

giordano commented Jun 8, 2023 •

edited

Loading

giordano commented Jun 9, 2023 •

edited

Loading

mgood7123 commented Oct 9, 2023 •

edited

Loading