Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Clang crashes with an inline statement in ggml #1928

Closed
RossComputerGuy opened this issue Sep 8, 2023 · 12 comments
Closed

[BUG] Clang crashes with an inline statement in ggml #1928

RossComputerGuy opened this issue Sep 8, 2023 · 12 comments
Labels
bug

Comments

@RossComputerGuy
Copy link

Description

When ggml.c in llama.cpp is compiled, this compiler crash occurs:

  fatal error: error in backend: Cannot select: 0x8da26c0: v4f32 = fmaxnum 0xbeda088, 0xbd15438, src/llama.cpp/ggml.c:3751:117 @[ src/llama.cpp/ggml.c:10520:9 @[ src/llama.cpp/ggml.c:10533:17 @[ src/llama.cpp/ggml.c:15101:17 ] ] ]
    0xbeda088: v4f32,i32,ch = ARMISD::VLD1_UPD<(load (s128) from %ir.544, align 4, !tbaa !985, !alias.scope !38240)> 0xaec05d8, 0xbd10b48, Constant:i32<16>, Constant:i32<1>, src/llama.cpp/ggml.c:3751:118 @[ src/llama.cpp/ggml.c:10520:9 @[ src/llama.cpp/ggml.c:10533:17 @[ src/llama.cpp/ggml.c:15101:17 ] ] ]
      0xbd10b48: i32,ch = CopyFromReg 0xaec05d8, Register:i32 %167, src/llama.cpp/ggml.c:3751:105 @[ src/llama.cpp/ggml.c:10520:9 @[ src/llama.cpp/ggml.c:10533:17 @[ src/llama.cpp/ggml.c:15101:17 ] ] ]
        0xb61c338: i32 = Register %167
      0x8da1fd8: i32 = Constant<16>
      0xbecdfb8: i32 = Constant<1>
    0xbd15438: v4f32 = bitcast 0xb61bcb8, src/llama.cpp/ggml.c:3751:123 @[ src/llama.cpp/ggml.c:10520:9 @[ src/llama.cpp/ggml.c:10533:17 @[ src/llama.cpp/ggml.c:15101:17 ] ] ]
      0xb61bcb8: v4i32 = ARMISD::VMOVIMM TargetConstant:i32<0>, src/llama.cpp/ggml.c:3751:123 @[ src/llama.cpp/ggml.c:10520:9 @[ src/llama.cpp/ggml.c:10533:17 @[ src/llama.cpp/ggml.c:15101:17 ] ] ]
        0x8da2248: i32 = TargetConstant<0>
  In function: ggml_compute_forward_unary
  PLEASE submit a bug report to https://github.com/android-ndk/ndk/issues and include the crash backtrace, preprocessed source, and associated run script.
  Stack dump:
  0.	Program arguments: /usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi23 --sysroot=/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DGGML_USE_K_QUANTS -D_XOPEN_SOURCE=600 -I/home/runner/work/flutter_llama/flutter_llama/src/llama.cpp/. -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=armv7-a -mthumb -Wformat -Werror=format-security -O2 -g -DNDEBUG -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Wno-unused-function -mfpu=neon-fp-armv8 -mno-unaligned-access -funsafe-math-optimizations -pthread -std=gnu11 -MD -MT llama.cpp/CMakeFiles/ggml.dir/ggml.c.o -MF llama.cpp/CMakeFiles/ggml.dir/ggml.c.o.d -o llama.cpp/CMakeFiles/ggml.dir/ggml.c.o -c /home/runner/work/flutter_llama/flutter_llama/src/llama.cpp/ggml.c
  1.	<eof> parser at end of file
  2.	Code generation
  3.	Running pass 'Function Pass Manager' on module '/home/runner/work/flutter_llama/flutter_llama/src/llama.cpp/ggml.c'.
  4.	Running pass 'ARM Instruction Selection' on function '@ggml_compute_forward_unary'
   #0 0x00000000047d91d8 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x47d91d8)
   #1 0x00000000047d8340 llvm::sys::RunSignalHandlers() (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x47d8340)
   #2 0x00000000047a3dc3 (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x47a3dc3)
   #3 0x00000000047a3d7b (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x47a3d7b)
   #4 0x00000000047d7a87 llvm::sys::Process::Exit(int, bool) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x47d7a87)
   #5 0x00000000040dc70a (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x40dc70a)
   #6 0x0000000003083072 llvm::report_fatal_error(llvm::Twine const&, bool) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x3083072)
   #7 0x000000000282b5f5 llvm::SelectionDAGISel::CannotYetSelect(llvm::SDNode*) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x282b5f5)
   #8 0x0000000006cf4e77 (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x6cf4e77)
   #9 0x000000000641f425 (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x641f425)
  #10 0x0000000005e86b63 llvm::SelectionDAGISel::DoInstructionSelection() (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x5e86b63)
  #11 0x0000000005e8710a llvm::SelectionDAGISel::CodeGenAndEmitDAG() (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x5e8710a)
  #12 0x0000000006417d3c llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x6417d3c)
  #13 0x0000000006457ad3 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x6457ad3)
  #14 0x00000000064572df (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x64572df)
  #15 0x0000000005d9faea llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x5d9faea)
  #16 0x0000000005da0113 llvm::FPPassManager::runOnFunction(llvm::Function&) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x5da0113)
  #17 0x0000000005d9fc6f llvm::FPPassManager::runOnModule(llvm::Module&) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x5d9fc6f)
  #18 0x00000000063aa794 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x63aa794)
  #19 0x00000000065d6968 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, std::__1::unique_ptr<llvm::raw_pwrite_stream, std::__1::default_delete<llvm::raw_pwrite_stream> >) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x65d6968)
  #20 0x00000000060524d5 (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x60524d5)
  #21 0x0000000005ea25a9 clang::ParseAST(clang::Sema&, bool, bool) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x5ea25a9)
  #22 0x00000000063c128d clang::FrontendAction::Execute() (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x63c128d)
  #23 0x00000000063c112d clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x63c112d)
  #24 0x00000000063c1541 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x63c1541)
  #25 0x00000000066a9f54 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x66a9f54)
  #26 0x00000000066a6de3 (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x66a6de3)
  #27 0x00000000066a6c92 (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x66a6c92)
  #28 0x00000000066a6c61 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x66a6c61)
  #29 0x00000000066a69f4 clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, bool*) const (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x66a69f4)
  #30 0x00000000066a685f clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&) const (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x66a685f)
  #31 0x00000000066a66f2 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::__1::pair<int, clang::driver::Command const*> >&) (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x66a66f2)
  #32 0x00000000066752ee main (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x66752ee)
  #33 0x00007fdce403cd90 (/lib/x86_64-linux-gnu/libc.so.6+0x29d90)
  #34 0x00007fdce403ce40 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e40)
  #35 0x00000000064cce69 _start (/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin/clang+0x64cce69)
  clang: error: clang frontend command failed with exit code 70 (use -v to see invocation)
  Android (9352603, based on r450784d1) clang version 14.0.7 (https://android.googlesource.com/toolchain/llvm-project 4c603efb0cca074e9238af8b4106c30add4418f6)
  Target: armv7-none-linux-android23
  Thread model: posix
  InstalledDir: /usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin
  clang: note: diagnostic msg: 
  ********************

  PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
  Preprocessed source(s) and associated run script(s) are located at:
  clang: note: diagnostic msg: /tmp/ggml-53c50e.c
  clang: note: diagnostic msg: /tmp/ggml-53c50e.sh
  clang: note: diagnostic msg: 

  ********************
  [4/6] Building CXX object llama.cpp/CMakeFiles/llama.dir/llama.cpp.o
  ninja: build stopped: subcommand failed.

[Origin](https://github.com/MidstallSoftware/flutter_llama/actions/runs/6126297086/job/16630008946]

Affected versions

r25

Canary version

No response

Host OS

Linux

Host OS version

Ubuntu 22.04 (GitHub CI)

Affected ABIs

armeabi-v7a

Build system

CMake

Other build system

Flutter

minSdkVersion

23

Device API level

No response
@RossComputerGuy RossComputerGuy mentioned this issue Sep 8, 2023
Closed
@pirama-arumuga-nainar
Copy link
Collaborator

Does this reproduce with r26? r25 is no longer supported.

If this also crashes with r26, attach reproducers mentioned in the crash message (see below for example) to further investigate.

 PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
  Preprocessed source(s) and associated run script(s) are located at:
  clang: note: diagnostic msg: /tmp/ggml-53c50e.c
  clang: note: diagnostic msg: /tmp/ggml-53c50e.sh
  clang: note: diagnostic msg:

@pirama-arumuga-nainar pirama-arumuga-nainar closed this as not planned Won't fix, can't repro, duplicate, stale Sep 8, 2023
@RossComputerGuy
Copy link
Author

Ok, I'll try with r26. I though r25 was still supported as r26 is in beta according to this.

@DanAlbert
Copy link
Member

I though r25 was still supported as r26 is in beta according to this.

Technically it's still supported until r26 comes out, but r26 will be out before we can possibly turn around an r25 fix for this bug. This is explained here: https://github.com/android/ndk/wiki/NDK-Release-Process#support-windows

@DanAlbert
Copy link
Member

DanAlbert commented Sep 8, 2023 •
edited
Loading

For it to be a candidate for r25 even if that weren't the case, it'd need to be a regression from r24, and we also can't determine that without the repro case. There's pretty much never a time that we can help if there's no repro case attached.

Ah, I see we got part of the repro stuff, yeah, include the stuff clang creates :) That makes things go much faster. Sorry for misunderstanding!

@RossComputerGuy
Copy link
Author

I've so far seen this fail with NDK versions:

  • 23.1.7779620
  • 25.2.9519653
  • 26.0.10636728

The C file: https://termbin.com/09jf
Contents of the shell script:

# Crash reproducer for Android (10087095, +pgo, +bolt, +lto, -mlgo, based on r487747c) clang version 17.0.2 (https://android.googlesource.com/toolchain/llvm-project d9f89f4d16663d5012e5c09495f3b30ece3d2362)
# Driver args: "--target=armv7-none-linux-androideabi23" "--sysroot=/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/sysroot" "-D" "GGML_USE_K_QUANTS" "-D" "_XOPEN_SOURCE=600" "-I" "/home/ross/MidstallSW/flutter_llama/src/llama.cpp/." "-g" "-D" "ANDROID" "-fdata-sections" "-ffunction-sections" "-funwind-tables" "-fstack-protector-strong" "-no-canonical-prefixes" "-D" "_FORTIFY_SOURCE=2" "-march=armv7-a" "-mthumb" "-Wformat" "-Werror=format-security" "-O2" "-g" "-D" "NDEBUG" "-fPIC" "-Wall" "-Wextra" "-Wpedantic" "-Wcast-qual" "-Wdouble-promotion" "-Wshadow" "-Wstrict-prototypes" "-Wpointer-arith" "-Wmissing-prototypes" "-Werror=implicit-int" "-Wno-unused-function" "-mfpu=neon-fp-armv8" "-mno-unaligned-access" "-funsafe-math-optimizations" "-pthread" "-std=gnu11" "-MD" "-MT" "llama.cpp/CMakeFiles/ggml.dir/ggml.c.o" "-MF" "llama.cpp/CMakeFiles/ggml.dir/ggml.c.o.d" "-o" "llama.cpp/CMakeFiles/ggml.dir/ggml.c.o" "-c" "/home/ross/MidstallSW/flutter_llama/src/llama.cpp/ggml.c"
# Original command:  "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/bin/clang" "-cc1" "-triple" "thumbv7-none-linux-android23" "-emit-obj" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "ggml.c" "-mrelocation-model" "pic" "-pic-level" "2" "-fhalf-no-semantic-interposition" "-mframe-pointer=all" "-fapprox-func" "-funsafe-math-optimizations" "-fno-signed-zeros" "-mreassociate" "-freciprocal-math" "-ffp-contract=fast" "-fno-rounding-math" "-mconstructor-aliases" "-funwind-tables=1" "-target-cpu" "generic" "-target-feature" "+soft-float-abi" "-target-feature" "+vfp2" "-target-feature" "+vfp2sp" "-target-feature" "+vfp3" "-target-feature" "+vfp3d16" "-target-feature" "+vfp3d16sp" "-target-feature" "+vfp3sp" "-target-feature" "+fp16" "-target-feature" "+vfp4" "-target-feature" "+vfp4d16" "-target-feature" "+vfp4d16sp" "-target-feature" "+vfp4sp" "-target-feature" "+fp-armv8" "-target-feature" "+fp-armv8d16" "-target-feature" "+fp-armv8d16sp" "-target-feature" "+fp-armv8sp" "-target-feature" "-fullfp16" "-target-feature" "+fp64" "-target-feature" "+d32" "-target-feature" "+neon" "-target-feature" "-sha2" "-target-feature" "-aes" "-target-feature" "-fp16fml" "-target-feature" "+strict-align" "-target-abi" "aapcs-linux" "-mfloat-abi" "soft" "-Wunaligned-access" "-mllvm" "-treat-scalable-fixed-error-as-warning" "-debug-info-kind=constructor" "-dwarf-version=4" "-debugger-tuning=gdb" "-ffunction-sections" "-fdata-sections" "-fcoverage-compilation-dir=/home/ross/MidstallSW/flutter_llama/android/.cxx/RelWithDebInfo/2w2f681z/armeabi-v7a" "-resource-dir" "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/lib/clang/17" "-dependency-file" "llama.cpp/CMakeFiles/ggml.dir/ggml.c.o.d" "-MT" "llama.cpp/CMakeFiles/ggml.dir/ggml.c.o" "-sys-header-deps" "-D" "GGML_USE_K_QUANTS" "-D" "_XOPEN_SOURCE=600" "-I" "/home/ross/MidstallSW/flutter_llama/src/llama.cpp/." "-D" "ANDROID" "-D" "_FORTIFY_SOURCE=2" "-D" "NDEBUG" "-isysroot" "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/sysroot" "-internal-isystem" "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/lib/clang/17/include" "-internal-isystem" "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/local/include" "-internal-externc-isystem" "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/arm-linux-androideabi" "-internal-externc-isystem" "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/sysroot/include" "-internal-externc-isystem" "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include" "-O2" "-Wformat" "-Werror=format-security" "-Wall" "-Wextra" "-Wpedantic" "-Wcast-qual" "-Wdouble-promotion" "-Wshadow" "-Wstrict-prototypes" "-Wpointer-arith" "-Wmissing-prototypes" "-Werror=implicit-int" "-Wno-unused-function" "-std=gnu11" "-fdebug-compilation-dir=/home/ross/MidstallSW/flutter_llama/android/.cxx/RelWithDebInfo/2w2f681z/armeabi-v7a" "-ferror-limit" "19" "-pthread" "-stack-protector" "2" "-fno-signed-char" "-fgnuc-version=4.2.1" "-vectorize-loops" "-vectorize-slp" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o" "llama.cpp/CMakeFiles/ggml.dir/ggml.c.o" "-x" "c" "/home/ross/MidstallSW/flutter_llama/src/llama.cpp/ggml.c"
 "/home/ross/Android/Sdk/ndk/26.0.10636728/toolchains/llvm/prebuilt/linux-x86_64/bin/clang" "-cc1" "-triple" "thumbv7-none-linux-android23" "-emit-obj" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "ggml.c" "-mrelocation-model" "pic" "-pic-level" "2" "-fhalf-no-semantic-interposition" "-mframe-pointer=all" "-fapprox-func" "-funsafe-math-optimizations" "-fno-signed-zeros" "-mreassociate" "-freciprocal-math" "-ffp-contract=fast" "-fno-rounding-math" "-mconstructor-aliases" "-funwind-tables=1" "-target-cpu" "generic" "-target-feature" "+soft-float-abi" "-target-feature" "+vfp2" "-target-feature" "+vfp2sp" "-target-feature" "+vfp3" "-target-feature" "+vfp3d16" "-target-feature" "+vfp3d16sp" "-target-feature" "+vfp3sp" "-target-feature" "+fp16" "-target-feature" "+vfp4" "-target-feature" "+vfp4d16" "-target-feature" "+vfp4d16sp" "-target-feature" "+vfp4sp" "-target-feature" "+fp-armv8" "-target-feature" "+fp-armv8d16" "-target-feature" "+fp-armv8d16sp" "-target-feature" "+fp-armv8sp" "-target-feature" "-fullfp16" "-target-feature" "+fp64" "-target-feature" "+d32" "-target-feature" "+neon" "-target-feature" "-sha2" "-target-feature" "-aes" "-target-feature" "-fp16fml" "-target-feature" "+strict-align" "-target-abi" "aapcs-linux" "-mfloat-abi" "soft" "-Wunaligned-access" "-mllvm" "-treat-scalable-fixed-error-as-warning" "-debug-info-kind=constructor" "-dwarf-version=4" "-debugger-tuning=gdb" "-ffunction-sections" "-fdata-sections" "-fcoverage-compilation-dir=/home/ross/MidstallSW/flutter_llama/android/.cxx/RelWithDebInfo/2w2f681z/armeabi-v7a" "-sys-header-deps" "-D" "GGML_USE_K_QUANTS" "-D" "_XOPEN_SOURCE=600" "-D" "ANDROID" "-D" "_FORTIFY_SOURCE=2" "-D" "NDEBUG" "-O2" "-Wformat" "-Werror=format-security" "-Wall" "-Wextra" "-Wpedantic" "-Wcast-qual" "-Wdouble-promotion" "-Wshadow" "-Wstrict-prototypes" "-Wpointer-arith" "-Wmissing-prototypes" "-Werror=implicit-int" "-Wno-unused-function" "-std=gnu11" "-fdebug-compilation-dir=/home/ross/MidstallSW/flutter_llama/android/.cxx/RelWithDebInfo/2w2f681z/armeabi-v7a" "-ferror-limit" "19" "-pthread" "-stack-protector" "2" "-fno-signed-char" "-fgnuc-version=4.2.1" "-vectorize-loops" "-vectorize-slp" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-x" "c" "ggml-c7bd3c.c"

@DanAlbert
Copy link
Member

I've so far seen this fail with NDK versions:

  • 23.1.7779620
  • 25.2.9519653
  • 26.0.10636728

Thanks! That helps clear things up. Reopening since this is still broken.

@DanAlbert DanAlbert reopened this Sep 8, 2023
@appujee
Copy link
Collaborator

appujee commented Sep 8, 2023

i see this is still a crash in fairly latest (8ba1c38a0d70a8e972f1f011629d6391f8744072) llvm-project.

llvm-project/build/bin/clang --target=armv7-none-linux-androideabi24  -fdata-sect
ions -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=ar
mv7-a -mthumb  -O2 -g  -fPIC  -mfpu=neon-fp-armv8 -mno-unaligned-access -funsafe-math-optimizations -pthread -std=gnu1
1  -o ggml.c.o -c ggml.i
fatal error: error in backend: Cannot select: 0x55848e4be0e0: v4f32 = fmaxnum 0x55848fa99830, 0x55848fc54190, ggml.c:3751:117 @[ ggml.c:10520:9 @[ ggml.c:10533:17 @[ ggml.c:15101:17 ] ] ]
  0x55848fa99830: v4f32,i32,ch = ARMISD::VLD1_UPD<(load (s128) from %ir.471, align 4, !tbaa !2217)> 0x55848cf24680, 0x55848c7af020, Constant:i32<16>, Constant:i32<1>, ggml.c:3751:118 @[ ggml.c:10520:9 @[ ggml.c:10533:17 @[ ggml.c:15101:17 ] ] ]
    0x55848c7af020: i32,ch = CopyFromReg 0x55848cf24680, Register:i32 %147, ggml.c:3751:118 @[ ggml.c:10520:9 @[ ggml.c:10533:17 @[ ggml.c:15101:17 ] ] ]
      0x55848e9bba00: i32 = Register %147
    0x55848e4be1c0: i32 = Constant<16>
    0x55848fa926f0: i32 = Constant<1>
  0x55848fc54190: v4f32 = bitcast 0x55848f35da80, ggml.c:3751:123 @[ ggml.c:10520:9 @[ ggml.c:10533:17 @[ ggml.c:15101:17 ] ] ]
    0x55848f35da80: v4i32 = ARMISD::VMOVIMM TargetConstant:i32<0>, ggml.c:3751:123 @[ ggml.c:10520:9 @[ ggml.c:10533:17 @[ ggml.c:15101:17 ] ] ]
      0x55848fc61f00: i32 = TargetConstant<0>
In function: ggml_compute_forward_unary
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: ../../llvm-project/build/bin/clang --target=armv7-none-linux-androideabi24 -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=armv7-a -mthumb -O2 -g -fPIC -mfpu=neon-fp-armv8 -mno-unaligned-access -funsafe-math-optimizations -pthread -std=gnu11 -o ggml.c.o -c ggml.i
1.      <eof> parser at end of file
2.      Code generation
3.      Running pass 'Function Pass Manager' on module 'ggml.c'.
4.      Running pass 'ARM Instruction Selection' on function '@ggml_compute_forward_unary'
 #0 0x000055848639d037 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (../../llvm-project/build/bin/clang+0x1eec037)
 #1 0x000055848639adbe llvm::sys::RunSignalHandlers() (../../llvm-project/build/bin/clang+0x1ee9dbe)
 #2 0x000055848631c21f (anonymous namespace)::CrashRecoveryContextImpl::HandleCrash(int, unsigned long) CrashRecoveryContext.cpp:0:0
 #3 0x000055848631c1db (../../llvm-project/build/bin/clang+0x1e6b1db)
 #4 0x0000558486397977 llvm::sys::Process::Exit(int, bool) (../../llvm-project/build/bin/clang+0x1ee6977)
 #5 0x0000558485108312 (../../llvm-project/build/bin/clang+0xc57312)
 #6 0x000055848632092b llvm::report_fatal_error(llvm::Twine const&, bool) (../../llvm-project/build/bin/clang+0x1e6f92b)
 #7 0x00005584870cb050 llvm::SelectionDAGISel::isOrEquivalentToAdd(llvm::SDNode const*) const (../../llvm-project/build/bin/clang+0x2c1a050)
 #8 0x00005584870ca53d (../../llvm-project/build/bin/clang+0x2c1953d)
 #9 0x00005584854cb29e (anonymous namespace)::ARMDAGToDAGISel::Select(llvm::SDNode*) ARMISelDAGToDAG.cpp:0:0
#10 0x00005584870c1eef llvm::SelectionDAGISel::DoInstructionSelection() (../../llvm-project/build/bin/clang+0x2c10eef)
#11 0x00005584870c1590 llvm::SelectionDAGISel::CodeGenAndEmitDAG() (../../llvm-project/build/bin/clang+0x2c10590)
#12 0x00005584870c0846 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) (../../llvm-project/build/bin/clang+0x2c0f846)
#13 0x00005584870be632 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) (../../llvm-project/build/bin/clang+0x2c0d632)
#14 0x00005584854c8721 (anonymous namespace)::ARMDAGToDAGISel::runOnMachineFunction(llvm::MachineFunction&) ARMISelDAGToDAG.cpp:0:0
#15 0x0000558485b32369 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (../../llvm-project/build/bin/clang+0x1681369)
#16 0x0000558485f9be62 llvm::FPPassManager::runOnFunction(llvm::Function&) (../../llvm-project/build/bin/clang+0x1aeae62)
#17 0x0000558485fa30f3 llvm::FPPassManager::runOnModule(llvm::Module&) (../../llvm-project/build/bin/clang+0x1af20f3)
#18 0x0000558485f9ca00 llvm::legacy::PassManagerImpl::run(llvm::Module&) (../../llvm-project/build/bin/clang+0x1aeba00)
#19 0x0000558486539ed3 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>) (../../llvm-project/build/bin/clang+0x2088ed3)
#20 0x00005584871c6bae clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (../../llvm-project/build/bin/clang+0x2d15bae)
#21 0x00005584880bae54 clang::ParseAST(clang::Sema&, bool, bool) (../../llvm-project/build/bin/clang+0x3c09e54)
#22 0x0000558486c0c109 clang::FrontendAction::Execute() (../../llvm-project/build/bin/clang+0x275b109)
#23 0x0000558486b812d6 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (../../llvm-project/build/bin/clang+0x26d02d6)
#24 0x0000558486cc906b clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (../../llvm-project/build/bin/clang+0x281806b)
#25 0x0000558485108031 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (../../llvm-project/build/bin/clang+0xc57031)
#26 0x0000558485105b65 ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) driver.cpp:0:0
#27 0x00005584869ef7d9 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const::$_1>(long) Job.cpp:0:0
#28 0x000055848631c1c5 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (../../llvm-project/build/bin/clang+0x1e6b1c5)
#29 0x00005584869ef265 clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (../../llvm-project/build/bin/clang+0x253e265)
#30 0x00005584869b3cb3 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (../../llvm-project/build/bin/clang+0x2502cb3)
#31 0x00005584869b3eee clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (../../llvm-project/build/bin/clang+0x2502eee)
#32 0x00005584869d17e0 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (../../llvm-project/build/bin/clang+0x25207e0)
#33 0x0000558485105397 clang_main(int, char**, llvm::ToolContext const&) (../../llvm-project/build/bin/clang+0xc54397)
#34 0x0000558485112d21 main (../../llvm-project/build/bin/clang+0xc61d21)
#35 0x00007f4c2a4456ca __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#36 0x00007f4c2a445785 call_init ./csu/../csu/libc-start.c:128:20
#37 0x00007f4c2a445785 __libc_start_main ./csu/../csu/libc-start.c:347:5
#38 0x0000558485102111 _start (../../llvm-project/build/bin/clang+0xc51111)
clang: error: clang frontend command failed with exit code 70 (use -v to see invocation)
clang version 18.0.0 (https://github.com/llvm/llvm-project.git c7430a7c6401b86ec5aedb90b9387285dfa04049)
Target: armv7-none-linux-android24
Thread model: posix
InstalledDir: ../../llvm-project/build/bin
clang: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs.

@appujee
Copy link
Collaborator

appujee commented Sep 8, 2023

llvm/llvm-project#65820

@DanAlbert
Copy link
Member

Thanks, @appujee!

@appujee
Copy link
Collaborator

appujee commented Sep 17, 2023 •
edited
Loading

fixed in llvm-project llvm/llvm-project@347b3f1

@pirama-arumuga-nainar
Copy link
Collaborator

Thanks for resolving this in upstream @appujee. http://r.android.com/2753660 and http://r.android.com/2755291 cherry-pick this to AOSP.

@DanAlbert DanAlbert reopened this Feb 1, 2024
@DanAlbert
Copy link
Member

Fix is merged in r26. I think the toolchain we currently have in master is too old for the fix, but it'll be in the r27 toolchain before it goes to beta 1. Leaving this open until then.

Closed
@grendello grendello mentioned this issue Feb 16, 2024
Merged
grendello added a commit to dotnet/android that referenced this issue Feb 19, 2024
@grendello
Bump NDK to r26c (#8732)
7368487 
Changes: https://github.com/android/ndk/wiki/Changelog-r26#r26c

  * Updated LLVM to clang-r487747e. See AndroidVersion.txt and clang_source_info.md 
     in the toolchain directory for version information.
  * [Issue 1928](android/ndk#1928): Fixed Clang crash in 
     instruction selection for 32-bit armv8 floating point.
  * [Issue 1953](android/ndk#1953): armeabi-v7a libc++ 
     libraries are once again built as thumb.

`AndroidVersion.txt` contains:

```
17.0.2
based on r487747e
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
LLVM
Status: Prebuilts submitted
NDK r26c
Status: Merged
NDK r27
Status: Merged
Milestone
No milestone
Development

No branches or pull requests
4 participants
@DanAlbert @RossComputerGuy @pirama-arumuga-nainar @appujee