Crash with illegal instruction: 4 on one Mac

[marcandre]

I have two Macs: a MacPro 5,1 and a MacBook 12", both running with the same OS, XCode & clang, elixir and erlang/OTP.

On my MacBook, tests pass, yet on the MacPro, jaxon crashes systematically with illegal instruction: 4

$ mix test
[...]
==> jaxon
mkdir -p priv
clang -undefined dynamic_lookup -dynamiclib -msse2 -mavx2 -std=c99 -O3 -I/Users/work/.asdf/installs/erlang/24.0.1/erts-12.0.1/include c_src/decoder*.c -o priv/decoder.so
Compiling 11 files (.ex)
Generated jaxon app
Illegal instruction: 4

I've attached the crash report, but the thread that crashes looks like:

Thread 4 Crashed:: 1_scheduler
0   decoder.so                    	0x000000014be7aa05 parse_string + 21
1   decoder.so                    	0x000000014be7aeb1 decode + 177
2   decoder.so                    	0x000000014be7b362 decode_binary + 370
3   beam.smp                      	0x000000010ad45c84 beam_jit_call_nif + 100

I found this issue because my app calls jaxon and also crashes, with the equivalent of:

["{}"] |> Jaxon.Stream.from_enumerable() |> Enum.to_list() 

Note that the stream gets created, it is only when it is to be consumed (to_list) that it crashes.

For that code, the crash log looks like:

Thread 4 Crashed:: 1_scheduler
0   decoder.so                    	0x000000014300e301 decode_binary + 257
1   beam.smp                      	0x0000000101c56c84 beam_jit_call_nif + 100
2   ???                           	0x0000000142f03b90 0 + 5418007440

I first thought Erlang was the issue, so I reinsalled it, and also downgraded it but I also get a crash.

I realize completely how difficult it is to diagnose something one can not test, but is there something I can check / reinstall to see?

MacOS: 10.14.6
XCode: 11.31.1

$ clang --version
Apple clang version 11.0.0 (clang-1100.0.33.17)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Volumes/Media/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ elixir --version
Erlang/OTP 24 [erts-12.0.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit]

Elixir 1.12.1 (compiled with Erlang/OTP 22)

beam.smp_2021-06-14-153412_mal.crash.txt

[marcandre]

Ah, found something. If I remove the -O3 optimization compilation flag, I no longer get a crash!
I do get a failure message multiple times about NifParser:

01:09:08.818 [warn]  The on_load function for module Elixir.Jaxon.Parsers.NifParser returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so, 2): Symbol not found: _timespec_to_ms\n  Referenced from: /Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so\n  Expected in: flat namespace\n in /Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so\''}}


01:09:09.194 [warn]  The on_load function for module Elixir.Jaxon.Parsers.NifParser returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so, 2): Symbol not found: _timespec_to_ms\n  Referenced from: /Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so\n  Expected in: flat namespace\n in /Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so\''}}


01:09:09.209 [warn]  The on_load function for module Elixir.Jaxon.Parsers.NifParser returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so, 2): Symbol not found: _timespec_to_ms\n  Referenced from: /Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so\n  Expected in: flat namespace\n in /Users/work/jaxon/_build/test/lib/jaxon/priv/decoder.so\''}}
  
 1) test parse bench/data/utf-8-unescaped.json (JaxonTest)
     test/jaxon_test.exs:106
     ** (UndefinedFunctionError) function Jaxon.Parsers.NifParser.parse/2 is undefined (module Jaxon.Parsers.NifParser is not available)
     code: assert {:ok, _} = decode(File.read!(unquote(filename)))
[...]

Any idea what is going on?

[marcandre]

False alarm. I removed all code having to deal with timespec, timespec_to_ms and get_current_monotic_time but I'm still getting crashes (decode_binary + 298) :-(

[marcandre]

I'm trying to get a better indication of what code makes it crash.

• Changed compile options to -O0 -g
• running mix test test/parser_test.exs which has a single test
• putting some printf+fflush

I'm not always getting the crash at the same spot 😢 . I just got a crash on double final_exp = pow(10.0, abs_exp); apparently, I just don't see how this could be a problem of any kind.

Thread 5 Crashed:: 2_scheduler
0   decoder.so                    	0x000000001216caa5 parse_number + 1221 (decoder.c:166)
1   decoder.so                    	0x000000001216d8f8 decode + 360 (decoder.c:468)
2   decoder.so                    	0x000000001216e1a6 decode_binary + 966 (decoder_nif.c:198)

[marcandre]

Caused by #28, closing.