rustc stackoverflow (SIGSEGV)

[womeier]

I was trying to run rust code extracted from Coq using the https://github.com/AU-COBRA/coq-rust-extraction plugin resulting in this project: binom-extracted.zip
The original program is main in binom.v, some operations on a binomial queue.

It does compile when setting the RUST_MIN_STACK environment var, as suggested in the seemingly related #122357, so feel free to close.

It may be due to a deep nesting of function calls, __nat_succ (the successor constructor of the naive representation of natural numbers in Coq).

fn  __nat_succ(x: u64) -> u64 {
  x.checked_add(1).unwrap()
 }

...
           __nat_succ(
              __nat_succ(
                __nat_succ(
                   ... // like ~2000 more nested calls
                        0))))

I expected to see this happen: successful compilation

Instead, this happened: rustc stackoverflow

Run command

cargo build with this Cargo.toml:

[package]
name = "binom"
version = "0.1.0"
authors = ["Cobra"]
edition = "2018"

[dependencies]
bumpalo = "3.5.0"

[profile.release]
# Tell `rustc` to optimize for small code size.
opt-level = 3

Meta

rustc --version --verbose:

rustc 1.76.0 (07dca489a 2024-02-04)
binary: rustc
commit-hash: 07dca489ac2d933c78d3c5158e3f43beefeb02ce
commit-date: 2024-02-04
host: x86_64-unknown-linux-gnu
release: 1.76.0
LLVM version: 17.0.6

Backtrace

error: rustc interrupted by SIGSEGV, printing backtrace

/home/wolfgang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-ef0b2e016afc8182.so(+0x2c31aa6)[0x7f2380631aa6]
/nix/store/ksk3rnb0ljx8gngzk19jlmbjyvac4hw6-glibc-2.38-44/lib/libc.so.6(+0x3deb0)[0x7f237d703eb0]
/home/wolfgang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-ef0b2e016afc8182.so(+0x3fbf79f)[0x7f23819bf79f]

### cycle encountered after 3 frames with period 4
/home/wolfgang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-ef0b2e016afc8182.so(+0x3fbc9c6)[0x7f23819bc9c6]
/home/wolfgang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-ef0b2e016afc8182.so(_RNvMs0_NtNtCs5wUc2TDgYSc_11rustc_parse6parser4exprNtB7_6Parser21parse_expr_assoc_with+0x79)[0x7f23819b9683]
/home/wolfgang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-ef0b2e016afc8182.so(+0x3fb5bac)[0x7f23819b5bac]
/home/wolfgang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-ef0b2e016afc8182.so(+0x3fc08af)[0x7f23819c08af]
### recursed 63 times

/home/wolfgang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-ef0b2e016afc8182.so(+0x3fbc9c6)[0x7f23819bc9c6]

note: rustc unexpectedly overflowed its stack! this is a bug
note: maximum backtrace depth reached, frames may have been lost
note: we would appreciate a report at https://github.com/rust-lang/rust
error: could not compile `binom` (bin "binom")

Caused by:
  process didn't exit successfully: `/home/wolfgang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name binom --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=172 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=da589634616a0e94 -C extra-filename=-da589634616a0e94 --out-dir /home/wolfgang/ProjectsUni/Compilerstep/certicoqwasm-rustextraction/benchmarks/rust/binom-extracted/target/debug/deps -C incremental=/home/wolfgang/ProjectsUni/Compilerstep/certicoqwasm-rustextraction/benchmarks/rust/binom-extracted/target/debug/incremental -L dependency=/home/wolfgang/ProjectsUni/Compilerstep/certicoqwasm-rustextraction/benchmarks/rust/binom-extracted/target/debug/deps --extern bumpalo=/home/wolfgang/ProjectsUni/Compilerstep/certicoqwasm-rustextraction/benchmarks/rust/binom-extracted/target/debug/deps/libbumpalo-bea91fa79be86aae.rlib` (signal: 11, SIGSEGV: invalid memory reference)

[workingjubilee]

@womeier Please specify the build command you used. Did you build this with optimizations or what?

[workingjubilee]

Oh, I'm sorry, I misunderstood.

No, #122357 is not quite related because it's in LLVM. This is in rustc.

/home/jubilee/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-af81632212034101.so(_RNvMs0_NtNtCs5HMe7kQiB7A_11rustc_parse6parser4exprNtB7_6Parser21parse_expr_assoc_with+0x7b)[0x7ef914d3c1fb]
### recursed 63 times

Maybe we should be using ensure_sufficient_stack in more places in the Parser?

[womeier]

@womeier Please specify the build command you used. Did you build this with optimizations or what?

Included the Cargo.toml file.

[workingjubilee]

Pulled out a debuginfo'd backtrace from gdb and quickly found this very regular period:

#0  0x00007da0db8844fe in <rustc_parse::parser::Parser>::parse_outer_attributes () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#1  0x00007da0db8975f8 in <rustc_parse::parser::Parser>::parse_expr_prefix () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#2  0x00007da0db8939e7 in <rustc_parse::parser::Parser>::parse_expr_assoc_with () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#3  0x00007da0db8937f3 in <rustc_parse::parser::Parser>::parse_expr_assoc () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#4  0x00007da0db954e31 in <rustc_parse::parser::Parser>::with_res::<core::result::Result<rustc_ast::ptr::P<rustc_ast::ast::Expr>, rustc_errors::diagnostic::Diag>, <rustc_parse::parser::Parser>::parse_expr_res::{closure#0}> ()
   from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#5  0x00007da0db893623 in <rustc_parse::parser::Parser>::parse_expr_catch_underscore () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#6  0x00007da0db93cc21 in <rustc_parse::parser::Parser>::parse_paren_comma_seq::<rustc_ast::ptr::P<rustc_ast::ast::Expr>, <rustc_parse::parser::Parser>::parse_expr_paren_seq::{closure#0}> () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#7  0x00007da0db899cec in <rustc_parse::parser::Parser>::parse_expr_dot_or_call_with_ () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#8  0x00007da0db8984fe in <rustc_parse::parser::Parser>::parse_expr_dot_or_call_with () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#9  0x00007da0db92c746 in <rustc_parse::parser::Parser>::collect_tokens_for_expr::<<rustc_parse::parser::Parser>::parse_expr_dot_or_call::{closure#0}>::{closure#0} () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#10 0x00007da0db92a3ee in <rustc_parse::parser::Parser>::collect_tokens_for_expr::<<rustc_parse::parser::Parser>::parse_expr_dot_or_call::{closure#0}> () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#11 0x00007da0db89787a in <rustc_parse::parser::Parser>::parse_expr_prefix () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#12 0x00007da0db8939e7 in <rustc_parse::parser::Parser>::parse_expr_assoc_with () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#13 0x00007da0db8937f3 in <rustc_parse::parser::Parser>::parse_expr_assoc () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#14 0x00007da0db954e31 in <rustc_parse::parser::Parser>::with_res::<core::result::Result<rustc_ast::ptr::P<rustc_ast::ast::Expr>, rustc_errors::diagnostic::Diag>, <rustc_parse::parser::Parser>::parse_expr_res::{closure#0}> ()
   from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#15 0x00007da0db893623 in <rustc_parse::parser::Parser>::parse_expr_catch_underscore () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#16 0x00007da0db93cc21 in <rustc_parse::parser::Parser>::parse_paren_comma_seq::<rustc_ast::ptr::P<rustc_ast::ast::Expr>, <rustc_parse::parser::Parser>::parse_expr_paren_seq::{closure#0}> () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#17 0x00007da0db899cec in <rustc_parse::parser::Parser>::parse_expr_dot_or_call_with_ () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#18 0x00007da0db8984fe in <rustc_parse::parser::Parser>::parse_expr_dot_or_call_with () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#19 0x00007da0db92c746 in <rustc_parse::parser::Parser>::collect_tokens_for_expr::<<rustc_parse::parser::Parser>::parse_expr_dot_or_call::{closure#0}>::{closure#0} () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#20 0x00007da0db92a3ee in <rustc_parse::parser::Parser>::collect_tokens_for_expr::<<rustc_parse::parser::Parser>::parse_expr_dot_or_call::{closure#0}> () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so
#21 0x00007da0db89787a in <rustc_parse::parser::Parser>::parse_expr_prefix () from /home/jubilee/rust/rustc/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-decf49367d89a1c2.so

At first glance I thought that parse_expr_assoc_with was the problem, while recursing into the rhs, but it seems that was only an artifact of the visible symbols in the backtrace. Funnily enough, the parse_outer_attributes is period-breaking, but we've run out of stack by the time we hit it!

[womeier]

Pretty cool to get this fixed in a day, thanks @workingjubilee.