ICE on out of memory error

[amarao]

Hello.

I've played with rust, and when I've tried to push too many things in Vec, it cause Rust to made out of memory error. That was fine. But that it said that there was an 'Ilegal instruction' and that core was dumped.

Final of strace output on the binary:

mmap(NULL, 1073741824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8041200000
madvise(0x7f8081200000, 536870912, MADV_DONTNEED) = 0
mmap(0x7f8081200000, 1073741824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8001200000
munmap(0x7f8001200000, 1073741824)      = 0
mmap(NULL, 2147483648, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
brk(0x557a28000000)                     = 0x5579a7e5f000
write(2, "fatal runtime error: out of memo"..., 35fatal runtime error: out of memory
) = 35
--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPN, si_addr=0x5579a61c11ba} ---
+++ killed by SIGILL (core dumped) +++
Illegal instruction (core dumped)

Final of backtrace in gdb:

#6  0x000055555556d258 in panic_unwind::__rust_maybe_catch_panic (f=0x5555555667f0 <std::panicking::try::do_call<fn(),()>>, data=0x2 <error: Cannot access memory at address 0x2>, data_ptr=0x7fffffffdc90, vtable_ptr=0x7fffffffdc88)
    at /build/rustc-tTSICN/rustc-1.18.0+dfsg1/src/libpanic_unwind/lib.rs:98
#7  0x00005555555667a8 in std::panicking::try<(),fn()> (f=<optimized out>) at /build/rustc-tTSICN/rustc-1.18.0+dfsg1/src/libstd/panicking.rs:433
#8  0x000055555555d2a6 in std::panic::catch_unwind<fn(),()> (f=<optimized out>) at /build/rustc-tTSICN/rustc-1.18.0+dfsg1/src/libstd/panic.rs:361
#9  std::rt::lang_start (main=0x55555555aca0 <ch8::main> "UH\211\345H\203\354`1\300\211Ǹ\004\000", argc=1, argv=0x7fffffffde18) at /build/rustc-tTSICN/rustc-1.18.0+dfsg1/src/libstd/rt.rs:57
#10 0x000055555555ade3 in main ()

gdb's layout asm shows:

   │0x55555555d1ab <std::sys::imp::init::oom_handler+11>    mov    $0x2,%edi                                                                                                                                                                                                             
   │0x55555555d1b0 <std::sys::imp::init::oom_handler+16>    mov    $0x23,%edx                                                                                                                                                                                                            
   │0x55555555d1b5 <std::sys::imp::init::oom_handler+21>    callq  0x5555555599f8                                                                                                                                                                                                        
  >│0x55555555d1ba <std::sys::imp::init::oom_handler+26>    ud2        

Why Rust calls for 'ud2' instruction which is intended to cause invalid opcode exception?

My source code (but I think, that any long chain of push'es would do this).

enum E {
    Signed(i32),
    Unsigned(u32)
}


fn messup(input: &E) -> i32 {
  match *input{
    E::Signed(_) => {return 0;},
    E::Unsigned(_) => {return 1;},
  }
}

fn main() {
  let mut counter = 0;
  let mut example: Vec<E> = vec![];
  while counter < 100000000 {
    example.push(E::Signed(1));
    example.push(E::Unsigned(1));
    //println!("{}", messup(&example[counter]));
    messup(&example[counter]);
    counter = counter + 1;
  }
  //println!("{}, {}", messup(&example[0]), messup(&example[1]))

}

[nagisa]

Why Rust calls for 'ud2' instruction which is intended to cause invalid opcode exception?

This is how intrinsics::abort is implemented. The described behaviour is expected in out-of-memory situations. It is not ideal and you will have to wait until fallible allocations are supported in containers for it to get better: #29802.