[Regression] Rust v1.20.0 leads to segfault with lazy_static Mutex inside a dylib

[archshift]

Smallest sample code I could come up with: https://gist.github.com/d679876d14dd6990494ca65440800dad

When the program reaches main.cpp,L8 (env::args().nth(0).unwrap();), it crashes with a segfault. This did not happen in v1.19.0.

Version:

rustc 1.20.0 (f3d6973f4 2017-08-27)
binary: rustc
commit-hash: f3d6973f41a7d1fb83029c9c0ceaf0f5d4fd7208
commit-date: 2017-08-27
host: x86_64-apple-darwin
release: 1.20.0
LLVM version: 4.0

Backtrace with rust-lldb:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x00000001003f0ca9 libsystem_platform.dylib`OSSpinLockLock + 7
    frame #1: 0x00000001000091ad what`je_arena_dalloc_small [inlined] je_malloc_mutex_lock at mutex.h:83 [opt]
    frame #2: 0x0000000100009195 what`je_arena_dalloc_small [inlined] je_arena_dalloc_bin(arena=&0x1000007feedfacf, chunk=&0x100400000, ptr=&0x100502cc0, pageind=<unavailable>) at arena.c:2760 [opt]
    frame #3: 0x000000010000915a what`je_arena_dalloc_small(tsd=&0x10120c008, arena=&0x1000007feedfacf, chunk=&0x100400000, ptr=&0x100502cc0, pageind=<unavailable>) at arena.c:2777 [opt]
    frame #4: 0x0000000100003783 what`je_sdallocx at arena.h:1428 [opt]
    frame #5: 0x00000001000036a2 what`je_sdallocx [inlined] je_isdalloct at jemalloc_internal.h:1086 [opt]
    frame #6: 0x00000001000036a2 what`je_sdallocx [inlined] je_isqalloc(ptr=<unavailable>) at jemalloc_internal.h:1096 [opt]
    frame #7: 0x0000000100003694 what`je_sdallocx [inlined] isfree(tcache=<unavailable>) at jemalloc.c:1842 [opt]
    frame #8: 0x000000010000368f what`je_sdallocx(ptr=<unavailable>, size=<unavailable>, flags=<unavailable>) at jemalloc.c:2532 [opt]
    frame #9: 0x0000000100002645 what`alloc::heap::{{impl}}::dealloc(self=&0x7fff5fbff690, ptr=&0x100502cc0, layout=Layout {
size: 19,
align: 1
}) at heap.rs:103
    frame #10: 0x00000001000015af what`alloc::raw_vec::{{impl}}::dealloc_buffer<u8,alloc::heap::Heap>(self=&0x7fff5fbff680) at raw_vec.rs:637
    frame #11: 0x0000000100002bcd what`alloc::raw_vec::{{impl}}::drop<u8,alloc::heap::Heap>(self=&0x7fff5fbff680) at raw_vec.rs:645
    frame #12: 0x0000000100001b75 what`core::ptr::drop_in_place<alloc::raw_vec::RawVec<u8, alloc::heap::Heap>>((null)=&0x7fff5fbff680) at ptr.rs:60
    frame #13: 0x0000000100001be0 what`core::ptr::drop_in_place<alloc::vec::Vec<u8>>((null)=&0x7fff5fbff680) at ptr.rs:60
    frame #14: 0x0000000100001ce5 what`core::ptr::drop_in_place<alloc::string::String>((null)=&0x7fff5fbff680) at ptr.rs:60
    frame #15: 0x0000000100002c68 what`what::main at main.rs:8
    frame #16: 0x000000010010630d libfoo.so`__rust_maybe_catch_panic + 29
    frame #17: 0x0000000100100809 libfoo.so`std::rt::lang_start::heed3cc6f59fb65ca + 473
    frame #18: 0x0000000100002cca what`main + 42
    frame #19: 0x00000001002f3235 libdyld.dylib`start + 1

[nikomatsakis]

Huh. That's pretty mysterious! We'll have to investigate and see if we can narrow down what might have caused it.

[nikomatsakis]

An ideal thing would be to try and bisect this using the nightly snapshots, to try and narrow down precisely when the regression occurred.

[nikomatsakis]

triage: P-high

[nikomatsakis]

I will try to do this, though if anyone can help with the bisection, it would be most appreciated. =)

[TimNN]

I can run the bisection tool later today.

[nikomatsakis]

I can reproduce the problem locally, at least.

[nikomatsakis]

(Using ./run.sh from the gist)

[nikomatsakis]

@TimNN that'd be awesome =)

[TimNN]

Here are the results:

• Can reproduce on macOS, cannot reproduce on Linux.
• Current beta and stable are affected
• Current nightly is not affected (Edit: apparently in some environments nightly is affected as well, see below)
• bisect-rust points to rustc: Implement the #[global_allocator] attribute #42727

Since the segfault occurs in jemalloc I tried building with the system allocator by adding

#![feature(alloc_system)]
extern crate alloc_system;

to foo.rs, main.rs and lazy-static/lib.rs, however jemalloc symbols still occur in the backtrace (tested on stable).

[sfackler]

@alexcrichton any PRs jump out to you as having fixed this?

[alexcrichton]

@TimNN what nightly did you test what wasn't affected? For me the current nightly, rustc 1.22.0-nightly (d93036a04 2017-09-07), segfaults like beta/stable. That leads me to believe that this may not be fixed on nightly.

That being said, this script is doing something that's not supported and fundamentally buggy, which is linking a rust dylib crate type natively (not through the compiler) into another rust program. This means that you've got two standard libraries will all the symbol clashes that come with that. What's likely happening here is something's getting allocated by one allocator and free'd by another because the calls are being routed basically randomly through the two standard libraries.

If you use the cdylib crate type then this works on all channels. I'm not sure this is a regression so much as "this was never supposed to work"

[TimNN]

@alexcrichton I'm running rustc 1.22.0-nightly (d93036a04 2017-09-07) as well. I'm also still on El Capitan, which I imagine could cause the different behaviour (if you are running something else).

[nikomatsakis]

@alexcrichton

That being said, this script is doing something that's not supported and fundamentally buggy...

Ah, I missed that. Do we want to close this as "unsupported", or do you think there's another bug here? I'm inclined to think you've correctly diagnosed the problem.

@archshift -- can you say a bit more about how you came upon this use case? Maybe we can help you set things up in a better way?

[alexcrichton]

@TimNN ah ok that may explain it yeah, I was on 10.12.6

@nikomatsakis yeah I'm ok closing as unsupported, but I'm ok waiting to hear back from @archshift

[archshift]

@nikomatsakis @alexcrichton I was linking a rust staticlib into a C dylib, then linking the resulting dylib with another rust project.

[alexcrichton]

@archshift ah yeah unfortunately that's not supported today unless you manually hide symbols from the Rust library. Otherwise allocators are just one instance of a set of symbols that can cause conflicts, but there's other global state in the standard library which can get mixed up and cause conflicts.

[nikomatsakis]

Going to close then. Thanks @archshift for the bug report!