Stack overflow? in parser in some code with syntax errors #80447

ijackson · 2020-12-28T18:34:43Z

Steps to reproduce

git clone https://salsa.debian.org/iwj/otter.git -b wip.ice
cd otter
cargo build -p otter-webdriver-tests

I'm afraid I haven't minimised this test case.

This doesn't seem to be a very serious problem as the compiler has correctly pointed out syntax errors; fixing those syntax errors makes the ICE go away. But since it suggested filing a bug I repro'd with today's nightly and have done so.

Error output

   Compiling otter-webdriver-tests v0.0.1 (/home/rustcargo/d/otter/wdriver)
error: expected one of `)`, `+`, `,`, or `::`, found `->`
   --> wdriver/../wdriver.rs:156:38
[more reasonable looking error messages]
    | non-item starts here
    | item list ends here

thread 'rustc' panicked at 'Stack should be empty: final_buf=FrameData { open: wdriver/../wdriver.rs:156:11: 156:12 (#0), inner: [(Token(Token { kind: BinOp(And), span: wdriver/../wdriver.rs:156:12: 156:13 (#0) }), Alone), (Token(Token { kind: Ident("self", false), span: wdriver/../wdriver.rs:156:13: 156:17 (#0) }), Joint), (Token(Token { kind: Comma, span: wdriver/../wdriver.rs:156:17: 156:18 (#0) }), Alone), (Token(Token { kind: Ident("s", false), span: wdriver/../wdriver.rs:156:19: 156:20 (#0) }), Joint), (Token(Token { kind: Colon, span: wdriver/../wdriver.rs:156:20: 156:21 (#0) }), Alone), (Token(Token { kind: BinOp(And), span: wdriver/../wdriver.rs:156:22: 156:23 (#0) }), Alone), (Token(Token { kind: Ident("dyn", false), span: wdriver/../wdriver.rs:156:23
[a very large quantity more like this]
wdriver.rs:177:19: 177:20 (#0) }), Alone)])), Alone)])), Alone)] } stack=[FrameData { open: wdriver/../wdriver.rs:1:1: 1:1 (#0), inner: [(Token(Token { kind: Ident("fn", false), span: wdriver/../wdriver.rs:156:3: 156:5 (#0) }), Alone), (Token(Token { kind: Ident("subst", false), span: wdriver/../wdriver.rs:156:6: 156:11 (#0) }), Alone)] }]', compiler/rustc_parse/src/parser/mod.rs:1409:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

Backtrace

stack backtrace:
   0:     0x7fd92e0877e7 - std::backtrace_rs::backtrace::libunwind::trace::h746c3e9529d524bc
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:     0x7fd92e0877e7 - std::backtrace_rs::backtrace::trace_unsynchronized::h84373278bfb39e0c
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7fd92e0877e7 - std::sys_common::backtrace::_print_fmt::h517324efde750597
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x7fd92e0877e7 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hf594ab77fac89284
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x7fd92e0f806c - core::fmt::write::h3868db8542c90941
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/core/src/fmt/mod.rs:1078:17
   5:     0x7fd92e079722 - std::io::Write::write_fmt::h3f6656f045fa877f
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/io/mod.rs:1519:15
   6:     0x7fd92e08b4b5 - std::sys_common::backtrace::_print::hda7655c057c24dcc
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x7fd92e08b4b5 - std::sys_common::backtrace::print::h546a6c8431d46287
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x7fd92e08b4b5 - std::panicking::default_hook::{{closure}}::h006dd083853faf51
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/panicking.rs:208:50
   9:     0x7fd92e08b013 - std::panicking::default_hook::hf0f9afb1017317fc
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/panicking.rs:225:9
  10:     0x7fd92e8eeb8b - rustc_driver::report_ice::h1a7ab7fc7b8fd641
  11:     0x7fd91d4912d6 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hf0349c093b90eab1
                               at /volatile/rustcargo/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1342:9
  12:     0x7fd91d4927eb - proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::enter::{{closure}}::{{closure}}::h6c733668da23e5b0
                               at /volatile/rustcargo/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/proc_macro/src/bridge/client.rs:320:21
  13:     0x7fd92e08bdb6 - std::panicking::rust_panic_with_hook::hb7a19826c029b1d6
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/panicking.rs:595:17
  14:     0x7fd92e08b8d7 - std::panicking::begin_panic_handler::{{closure}}::hde71edcd925d0c5e
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/panicking.rs:497:13
  15:     0x7fd92e087cac - std::sys_common::backtrace::__rust_end_short_backtrace::h8a3c7d6cea578919
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/sys_common/backtrace.rs:141:18
  16:     0x7fd92e08b839 - rust_begin_unwind
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/panicking.rs:493:5
  17:     0x7fd92e08b7eb - std::panicking::begin_panic_fmt::hee67ce14b77d0396
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/panicking.rs:435:5
  18:     0x7fd931688252 - <rustc_parse::parser::Parser::collect_tokens::LazyTokenStreamImpl as rustc_ast::tokenstream::CreateTokenStream>::create_token_stream::h841bbbdeec5aad6b
  19:     0x7fd9316a2d2e - rustc_parse::prepend_attrs::h63fa2d717369e1c0
  20:     0x7fd9316a0446 - rustc_parse::nt_to_tokenstream::he85c9b61a269d1e9
  21:     0x7fd92f81c201 - rustc_expand::base::Annotatable::into_tokens::hb578d9bf826452e8
  22:     0x7fd9309d503a - rustc_expand::expand::MacroExpander::fully_expand_fragment::he26fff039b78329f
  23:     0x7fd93161b6c3 - rustc_expand::expand::MacroExpander::expand_crate::hd5614e1080b391fe
  24:     0x7fd930e072dc - rustc_session::utils::<impl rustc_session::session::Session>::time::h77ea5bb11cbba8f9
  25:     0x7fd930e188fd - rustc_interface::passes::configure_and_expand_inner::had4f1edaa9e5dcc1
  26:     0x7fd930e031d5 - rustc_interface::passes::configure_and_expand::{{closure}}::h2a8de7b2e7685621
  27:     0x7fd930dfbd5d - rustc_data_structures::box_region::PinnedGenerator<I,A,R>::new::h2350cf4304a58173
  28:     0x7fd930e17f7b - rustc_interface::passes::configure_and_expand::hc2f6a0c4b008d57b
  29:     0x7fd930e2c38d - rustc_interface::queries::Queries::expansion::h541f7763ec81d5df
  30:     0x7fd930dbb420 - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter::h27c18adf15a8e2d0
  31:     0x7fd930db0ad4 - rustc_span::with_source_map::ha7a34f5cae8f35d5
  32:     0x7fd930dbc4ba - rustc_interface::interface::create_compiler_and_run::hadd076a85b89b30e
  33:     0x7fd930db1768 - std::sys_common::backtrace::__rust_begin_short_backtrace::h64a4b3a8ddd28b5c
  34:     0x7fd930d943ca - core::ops::function::FnOnce::call_once{{vtable.shim}}::h151269a6222c3e41
  35:     0x7fd92e09bfda - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hea1090dbdcecbf5a
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/alloc/src/boxed.rs:1328:9
  36:     0x7fd92e09bfda - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h8d5723d3912bd325
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/alloc/src/boxed.rs:1328:9
  37:     0x7fd92e09bfda - std::sys::unix::thread::Thread::new::thread_start::hc17a425ca2995724
                               at /rustc/257becbfe4987d1f7b12af5a8dd5ed96697cd2e8/library/std/src/sys/unix/thread.rs:71:17
  38:     0x7fd92df85fa3 - start_thread
  39:     0x7fd92dea54cf - clone
  40:                0x0 - <unknown>

The panic message in full

thread 'rustc' panicked at 'Stack should be empty: final_buf=FrameData { open: wdriver/../wdriver.rs:156:11: 156:12 (#0), inner: [(Token(Token { kind: BinOp(And), span: wdriver/../wdriver.rs:156:12: 156:13 (#0) }), Alone), (Token(Token { kind: Ident("self", false), span: wdriver/../wdriver.rs:156:13: 156:17 (#0) }), Joint), (Token(Token { kind: Comma, span: wdriver/../wdriver.rs:156:17: 156:18 (#0) }), Alone), (Token(Token { kind: Ident("s", false), span: wdriver/../wdriver.rs:156:19: 156:20 (#0) }), Joint), (Token(Token { kind: Colon, span: wdriver/../wdriver.rs:156:20: 156:21 (#0) }), Alone), (Token(Token { kind: BinOp(And), span: wdriver/../wdriver.rs:156:22: 156:23 (#0) }), Alone), (Token(Token { kind: Ident("dyn", false), span: wdriver/../wdriver.rs:156:23: 156:26 (#0) }), Alone), (Token(Token { kind: Ident("AsRef", false), span: wdriver/../wdriver.rs:156:27: 156:32 (#0) }), Joint), (Token(Token { kind: Lt, span: wdriver/../wdriver.rs:156:32: 156:33 (#0) }), Alone), (Token(Token { kind: Ident("str", false), span: wdriver/../wdriver.rs:156:33: 156:36 (#0) }), Joint), (Token(Token { kind: Gt, span: wdriver/../wdriver.rs:156:36: 156:37 (#0) }), Alone), (Token(Token { kind: RArrow, span: wdriver/../wdriver.rs:156:38: 156:40 (#0) }), Alone), (Token(Token { kind: Ident("String", false), span: wdriver/../wdriver.rs:156:41: 156:47 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:156:48: 156:49 (#0), close: wdriver/../wdriver.rs:178:3: 178:4 (#0) }, Brace, TokenStream([(Token(Token { kind: Pound, span: wdriver/../wdriver.rs:157:5: 157:6 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:157:6: 157:7 (#0), close: wdriver/../wdriver.rs:157:17: 157:18 (#0) }, Bracket, TokenStream([(Token(Token { kind: Ident("throws", false), span: wdriver/../wdriver.rs:157:7: 157:13 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:157:13: 157:14 (#0), close: wdriver/../wdriver.rs:157:16: 157:17 (#0) }, Paren, TokenStream([(Token(Token { kind: Ident("AE", false), span: wdriver/../wdriver.rs:157:14: 157:16 (#0) }), Alone)])), Alone)])), Alone), (Token(Token { kind: Ident("fn", false), span: wdriver/../wdriver.rs:158:5: 158:7 (#0) }), Alone), (Token(Token { kind: Ident("inner", false), span: wdriver/../wdriver.rs:158:8: 158:13 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:158:13: 158:14 (#0), close: wdriver/../wdriver.rs:178:3: 178:4 (#0) }, Paren, TokenStream([(Token(Token { kind: Ident("self_", false), span: wdriver/../wdriver.rs:158:14: 158:19 (#0) }), Joint), (Token(Token { kind: Colon, span: wdriver/../wdriver.rs:158:19: 158:20 (#0) }), Alone), (Token(Token { kind: BinOp(And), span: wdriver/../wdriver.rs:158:21: 158:22 (#0) }), Alone), (Token(Token { kind: Ident("Self", false), span: wdriver/../wdriver.rs:158:22: 158:26 (#0) }), Joint), (Token(Token { kind: Comma, span: wdriver/../wdriver.rs:158:26: 158:27 (#0) }), Alone), (Token(Token { kind: Ident("s", false), span: wdriver/../wdriver.rs:158:28: 158:29 (#0) }), Joint), (Token(Token { kind: Colon, span: wdriver/../wdriver.rs:158:29: 158:30 (#0) }), Alone), (Token(Token { kind: BinOp(And), span: wdriver/../wdriver.rs:158:31: 158:32 (#0) }), Alone), (Token(Token { kind: Ident("dyn", false), span: wdriver/../wdriver.rs:158:32: 158:35 (#0) }), Alone), (Token(Token { kind: Ident("AsRef", false), span: wdriver/../wdriver.rs:158:36: 158:41 (#0) }), Joint), (Token(Token { kind: Lt, span: wdriver/../wdriver.rs:158:41: 158:42 (#0) }), Alone), (Token(Token { kind: Ident("str", false), span: wdriver/../wdriver.rs:158:42: 158:45 (#0) }), Joint), (Token(Token { kind: Gt, span: wdriver/../wdriver.rs:158:45: 158:46 (#0) }), Alone), (Token(Token { kind: RArrow, span: wdriver/../wdriver.rs:158:47: 158:49 (#0) }), Alone), (Token(Token { kind: Ident("String", false), span: wdriver/../wdriver.rs:158:50: 158:56 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:158:57: 158:58 (#0), close: wdriver/../wdriver.rs:176:5: 176:6 (#0) }, Brace, TokenStream([(Token(Token { kind: Ident("let", false), span: wdriver/../wdriver.rs:159:7: 159:10 (#0) }), Alone), (Token(Token { kind: Ident("s", false), span: wdriver/../wdriver.rs:159:11: 159:12 (#0) }), Alone), (Token(Token { kind: Eq, span: wdriver/../wdriver.rs:159:13: 159:14 (#0) }), Alone), (Token(Token { kind: Ident("s", false), span: wdriver/../wdriver.rs:159:15: 159:16 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:159:16: 159:17 (#0) }), Alone), (Token(Token { kind: Ident("as_ref", false), span: wdriver/../wdriver.rs:159:17: 159:23 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:159:23: 159:24 (#0), close: wdriver/../wdriver.rs:159:24: 159:25 (#0) }, Paren, TokenStream([])), Alone), (Token(Token { kind: Semi, span: wdriver/../wdriver.rs:159:25: 159:26 (#0) }), Alone), (Token(Token { kind: Ident("let", false), span: wdriver/../wdriver.rs:160:7: 160:10 (#0) }), Alone), (Token(Token { kind: Ident("re", false), span: wdriver/../wdriver.rs:160:11: 160:13 (#0) }), Alone), (Token(Token { kind: Eq, span: wdriver/../wdriver.rs:160:14: 160:15 (#0) }), Alone), (Token(Token { kind: Ident("Regex", false), span: wdriver/../wdriver.rs:160:16: 160:21 (#0) }), Joint), (Token(Token { kind: ModSep, span: wdriver/../wdriver.rs:160:21: 160:23 (#0) }), Alone), (Token(Token { kind: Ident("new", false), span: wdriver/../wdriver.rs:160:23: 160:26 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:160:26: 160:27 (#0), close: wdriver/../wdriver.rs:160:37: 160:38 (#0) }, Paren, TokenStream([(Token(Token { kind: Literal(Lit { kind: StrRaw(0), symbol: "@(\\w+)@", suffix: None }), span: wdriver/../wdriver.rs:160:27: 160:37 (#0) }), Alone)])), Alone), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:160:38: 160:39 (#0) }), Alone), (Token(Token { kind: Ident("expect", false), span: wdriver/../wdriver.rs:160:39: 160:45 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:160:45: 160:46 (#0), close: wdriver/../wdriver.rs:160:55: 160:56 (#0) }, Paren, TokenStream([(Token(Token { kind: Literal(Lit { kind: Str, symbol: "bad re!", suffix: None }), span: wdriver/../wdriver.rs:160:46: 160:55 (#0) }), Alone)])), Alone), (Token(Token { kind: Semi, span: wdriver/../wdriver.rs:160:56: 160:57 (#0) }), Alone), (Token(Token { kind: Ident("let", false), span: wdriver/../wdriver.rs:161:7: 161:10 (#0) }), Alone), (Token(Token { kind: Ident("mut", false), span: wdriver/../wdriver.rs:161:11: 161:14 (#0) }), Alone), (Token(Token { kind: Ident("errs", false), span: wdriver/../wdriver.rs:161:15: 161:19 (#0) }), Alone), (Token(Token { kind: Eq, span: wdriver/../wdriver.rs:161:20: 161:21 (#0) }), Alone), (Token(Token { kind: Ident("vec", false), span: wdriver/../wdriver.rs:161:22: 161:25 (#0) }), Joint), (Token(Token { kind: Not, span: wdriver/../wdriver.rs:161:25: 161:26 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:161:26: 161:27 (#0), close: wdriver/../wdriver.rs:161:27: 161:28 (#0) }, Bracket, TokenStream([])), Alone), (Token(Token { kind: Semi, span: wdriver/../wdriver.rs:161:28: 161:29 (#0) }), Alone), (Token(Token { kind: Ident("let", false), span: wdriver/../wdriver.rs:162:7: 162:10 (#0) }), Alone), (Token(Token { kind: Ident("out", false), span: wdriver/../wdriver.rs:162:11: 162:14 (#0) }), Alone), (Token(Token { kind: Eq, span: wdriver/../wdriver.rs:162:15: 162:16 (#0) }), Alone), (Token(Token { kind: Ident("re", false), span: wdriver/../wdriver.rs:162:17: 162:19 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:162:19: 162:20 (#0) }), Alone), (Token(Token { kind: Ident("replace_all", false), span: wdriver/../wdriver.rs:162:20: 162:31 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:162:31: 162:32 (#0), close: wdriver/../wdriver.rs:170:8: 170:9 (#0) }, Paren, TokenStream([(Token(Token { kind: Ident("s", false), span: wdriver/../wdriver.rs:162:32: 162:33 (#0) }), Joint), (Token(Token { kind: Comma, span: wdriver/../wdriver.rs:162:33: 162:34 (#0) }), Alone), (Token(Token { kind: BinOp(Or), span: wdriver/../wdriver.rs:162:35: 162:36 (#0) }), Alone), (Token(Token { kind: Ident("caps", false), span: wdriver/../wdriver.rs:162:36: 162:40 (#0) }), Joint), (Token(Token { kind: Colon, span: wdriver/../wdriver.rs:162:40: 162:41 (#0) }), Alone), (Token(Token { kind: BinOp(And), span: wdriver/../wdriver.rs:162:42: 162:43 (#0) }), Alone), (Token(Token { kind: Ident("regex", false), span: wdriver/../wdriver.rs:162:43: 162:48 (#0) }), Joint), (Token(Token { kind: ModSep, span: wdriver/../wdriver.rs:162:48: 162:50 (#0) }), Alone), (Token(Token { kind: Ident("Captures", false), span: wdriver/../wdriver.rs:162:50: 162:58 (#0) }), Joint), (Token(Token { kind: BinOp(Or), span: wdriver/../wdriver.rs:162:58: 162:59 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:162:60: 162:61 (#0), close: wdriver/../wdriver.rs:170:7: 170:8 (#0) }, Brace, TokenStream([(Token(Token { kind: Ident("let", false), span: wdriver/../wdriver.rs:163:9: 163:12 (#0) }), Alone), (Token(Token { kind: Ident("kw", false), span: wdriver/../wdriver.rs:163:13: 163:15 (#0) }), Alone), (Token(Token { kind: Eq, span: wdriver/../wdriver.rs:163:16: 163:17 (#0) }), Alone), (Token(Token { kind: Ident("caps", false), span: wdriver/../wdriver.rs:163:18: 163:22 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:163:22: 163:23 (#0) }), Alone), (Token(Token { kind: Ident("get", false), span: wdriver/../wdriver.rs:163:23: 163:26 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:163:26: 163:27 (#0), close: wdriver/../wdriver.rs:163:28: 163:29 (#0) }, Paren, TokenStream([(Token(Token { kind: Literal(Lit { kind: Integer, symbol: "1", suffix: None }), span: wdriver/../wdriver.rs:163:27: 163:28 (#0) }), Alone)])), Alone), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:163:29: 163:30 (#0) }), Alone), (Token(Token { kind: Ident("expect", false), span: wdriver/../wdriver.rs:163:30: 163:36 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:163:36: 163:37 (#0), close: wdriver/../wdriver.rs:163:50: 163:51 (#0) }, Paren, TokenStream([(Token(Token { kind: Literal(Lit { kind: Str, symbol: "$1 missing!", suffix: None }), span: wdriver/../wdriver.rs:163:37: 163:50 (#0) }), Alone)])), Alone), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:163:51: 163:52 (#0) }), Alone), (Token(Token { kind: Ident("as_str", false), span: wdriver/../wdriver.rs:163:52: 163:58 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:163:58: 163:59 (#0), close: wdriver/../wdriver.rs:163:59: 163:60 (#0) }, Paren, TokenStream([])), Alone), (Token(Token { kind: Semi, span: wdriver/../wdriver.rs:163:60: 163:61 (#0) }), Alone), (Token(Token { kind: Ident("if", false), span: wdriver/../wdriver.rs:164:9: 164:11 (#0) }), Alone), (Token(Token { kind: Ident("kw", false), span: wdriver/../wdriver.rs:164:12: 164:14 (#0) }), Alone), (Token(Token { kind: EqEq, span: wdriver/../wdriver.rs:164:15: 164:17 (#0) }), Alone), (Token(Token { kind: Literal(Lit { kind: Str, symbol: "", suffix: None }), span: wdriver/../wdriver.rs:164:18: 164:20 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:164:21: 164:22 (#0), close: wdriver/../wdriver.rs:164:44: 164:45 (#0) }, Brace, TokenStream([(Token(Token { kind: Ident("return", false), span: wdriver/../wdriver.rs:164:23: 164:29 (#0) }), Alone), (Token(Token { kind: Literal(Lit { kind: Str, symbol: "", suffix: None }), span: wdriver/../wdriver.rs:164:30: 164:32 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:164:32: 164:33 (#0) }), Alone), (Token(Token { kind: Ident("to_owned", false), span: wdriver/../wdriver.rs:164:33: 164:41 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:164:41: 164:42 (#0), close: wdriver/../wdriver.rs:164:42: 164:43 (#0) }, Paren, TokenStream([])), Alone)])), Alone), (Token(Token { kind: Ident("let", false), span: wdriver/../wdriver.rs:165:9: 165:12 (#0) }), Alone), (Token(Token { kind: Ident("v", false), span: wdriver/../wdriver.rs:165:13: 165:14 (#0) }), Alone), (Token(Token { kind: Eq, span: wdriver/../wdriver.rs:165:15: 165:16 (#0) }), Alone), (Token(Token { kind: Ident("self", false), span: wdriver/../wdriver.rs:165:17: 165:21 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:165:21: 165:22 (#0) }), Alone), (Token(Token { kind: Ident("get", false), span: wdriver/../wdriver.rs:165:22: 165:25 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:165:25: 165:26 (#0), close: wdriver/../wdriver.rs:165:28: 165:29 (#0) }, Paren, TokenStream([(Token(Token { kind: Ident("kw", false), span: wdriver/../wdriver.rs:165:26: 165:28 (#0) }), Alone)])), Alone), (Token(Token { kind: Semi, span: wdriver/../wdriver.rs:165:29: 165:30 (#0) }), Alone), (Token(Token { kind: Ident("v", false), span: wdriver/../wdriver.rs:166:9: 166:10 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:166:10: 166:11 (#0) }), Alone), (Token(Token { kind: Ident("unwrap_or_else", false), span: wdriver/../wdriver.rs:166:11: 166:25 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:166:25: 166:26 (#0), close: wdriver/../wdriver.rs:169:10: 169:11 (#0) }, Paren, TokenStream([(Token(Token { kind: OrOr, span: wdriver/../wdriver.rs:166:26: 166:28 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:166:28: 166:29 (#0), close: wdriver/../wdriver.rs:169:9: 169:10 (#0) }, Brace, TokenStream([(Token(Token { kind: Ident("errs", false), span: wdriver/../wdriver.rs:167:11: 167:15 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:167:15: 167:16 (#0) }), Alone), (Token(Token { kind: Ident("push", false), span: wdriver/../wdriver.rs:167:16: 167:20 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:167:20: 167:21 (#0), close: wdriver/../wdriver.rs:167:34: 167:35 (#0) }, Paren, TokenStream([(Token(Token { kind: Ident("kw", false), span: wdriver/../wdriver.rs:167:21: 167:23 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:167:23: 167:24 (#0) }), Alone), (Token(Token { kind: Ident("to_owned", false), span: wdriver/../wdriver.rs:167:24: 167:32 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:167:32: 167:33 (#0), close: wdriver/../wdriver.rs:167:33: 167:34 (#0) }, Paren, TokenStream([])), Alone)])), Alone), (Token(Token { kind: Semi, span: wdriver/../wdriver.rs:167:35: 167:36 (#0) }), Alone), (Token(Token { kind: Literal(Lit { kind: Str, symbol: "", suffix: None }), span: wdriver/../wdriver.rs:168:11: 168:13 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:168:13: 168:14 (#0) }), Alone), (Token(Token { kind: Ident("to_owned", false), span: wdriver/../wdriver.rs:168:14: 168:22 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:168:22: 168:23 (#0), close: wdriver/../wdriver.rs:168:23: 168:24 (#0) }, Paren, TokenStream([])), Alone)])), Alone)])), Alone)])), Alone)])), Alone), (Token(Token { kind: Semi, span: wdriver/../wdriver.rs:170:9: 170:10 (#0) }), Alone), (Token(Token { kind: Ident("if", false), span: wdriver/../wdriver.rs:171:7: 171:9 (#0) }), Alone), (Token(Token { kind: Not, span: wdriver/../wdriver.rs:171:10: 171:11 (#0) }), Alone), (Token(Token { kind: Ident("errs", false), span: wdriver/../wdriver.rs:171:12: 171:16 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:171:16: 171:17 (#0) }), Alone), (Token(Token { kind: Ident("is_empty", false), span: wdriver/../wdriver.rs:171:17: 171:25 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:171:25: 171:26 (#0), close: wdriver/../wdriver.rs:171:26: 171:27 (#0) }, Paren, TokenStream([])), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:171:28: 171:29 (#0), close: wdriver/../wdriver.rs:174:7: 174:8 (#0) }, Brace, TokenStream([(Token(Token { kind: Ident("throw", false), span: wdriver/../wdriver.rs:172:9: 172:14 (#0) }), Joint), (Token(Token { kind: Not, span: wdriver/../wdriver.rs:172:14: 172:15 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:172:15: 172:16 (#0), close: wdriver/../wdriver.rs:173:33: 173:34 (#0) }, Paren, TokenStream([(Token(Token { kind: Ident("anyhow", false), span: wdriver/../wdriver.rs:172:16: 172:22 (#0) }), Joint), (Token(Token { kind: Not, span: wdriver/../wdriver.rs:172:22: 172:23 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:172:23: 172:24 (#0), close: wdriver/../wdriver.rs:173:32: 173:33 (#0) }, Paren, TokenStream([(Token(Token { kind: Literal(Lit { kind: Str, symbol: "bad substitution(s) {:?} in {:?}", suffix: None }), span: wdriver/../wdriver.rs:172:24: 172:58 (#0) }), Joint), (Token(Token { kind: Comma, span: wdriver/../wdriver.rs:172:58: 172:59 (#0) }), Alone), (Token(Token { kind: BinOp(And), span: wdriver/../wdriver.rs:173:24: 173:25 (#0) }), Alone), (Token(Token { kind: Ident("errs", false), span: wdriver/../wdriver.rs:173:25: 173:29 (#0) }), Joint), (Token(Token { kind: Comma, span: wdriver/../wdriver.rs:173:29: 173:30 (#0) }), Alone), (Token(Token { kind: Ident("s", false), span: wdriver/../wdriver.rs:173:31: 173:32 (#0) }), Alone)])), Alone)])), Alone), (Token(Token { kind: Semi, span: wdriver/../wdriver.rs:173:34: 173:35 (#0) }), Alone)])), Alone), (Token(Token { kind: Ident("out", false), span: wdriver/../wdriver.rs:175:7: 175:10 (#0) }), Joint), (Token(Token { kind: Dot, span: wdriver/../wdriver.rs:175:10: 175:11 (#0) }), Alone), (Token(Token { kind: Ident("into", false), span: wdriver/../wdriver.rs:175:11: 175:15 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:175:15: 175:16 (#0), close: wdriver/../wdriver.rs:175:16: 175:17 (#0) }, Paren, TokenStream([])), Alone)])), Alone), (Token(Token { kind: Ident("inner", false), span: wdriver/../wdriver.rs:177:5: 177:10 (#0) }), Alone), (Delimited(DelimSpan { open: wdriver/../wdriver.rs:177:10: 177:11 (#0), close: wdriver/../wdriver.rs:177:18: 177:19 (#0) }, Paren, TokenStream([(Token(Token { kind: Ident("self", false), span: wdriver/../wdriver.rs:177:11: 177:15 (#0) }), Joint), (Token(Token { kind: Comma, span: wdriver/../wdriver.rs:177:15: 177:16 (#0) }), Alone), (Token(Token { kind: Ident("s", false), span: wdriver/../wdriver.rs:177:17: 177:18 (#0) }), Alone)])), Alone), (Token(Token { kind: Question, span: wdriver/../wdriver.rs:177:19: 177:20 (#0) }), Alone)])), Alone)])), Alone)] } stack=[FrameData { open: wdriver/../wdriver.rs:1:1: 1:1 (#0), inner: [(Token(Token { kind: Ident("fn", false), span: wdriver/../wdriver.rs:156:3: 156:5 (#0) }), Alone), (Token(Token { kind: Ident("subst", false), span: wdriver/../wdriver.rs:156:6: 156:11 (#0) }), Alone)] }]', compiler/rustc_parse/src/parser/mod.rs:1409:5

The text was updated successfully, but these errors were encountered:

durka · 2020-12-29T02:23:24Z

Minimized:

proc macro

extern crate proc_macro;
use proc_macro::TokenStream;

#[proc_macro_attribute]
pub fn mac(_attrs: TokenStream, input: TokenStream) -> TokenStream {
    input
}

lib.rs

pub trait Crash {
    #[mac::mac]
    fn one(s: () {
    }

    fn two();
}

ijackson · 2020-12-29T02:38:37Z

@rustbot modify labels -E-needs-mcve +A-proc-macros

Issue: rust-lang/rust#80447

durka · 2020-12-29T04:18:49Z

Is the solution possibly to just remove the offending assertion? The token stream will be nonsense but some other error reporting should catch the mismatched delimiters, I'd assume?

osa1 · 2021-01-27T12:24:46Z

I just quickly tried the suggestion above (removing the assertion) and this is the error I got without it:

error: expected one of `)` or `,`, found `{`
 --> src/lib.rs:3:18
  |
3 |     fn one(s: () {
  |           -     -^
  |           |     |
  |           |     help: `)` may belong here
  |           unclosed delimiter

error: non-item in item list
 --> src/lib.rs:5:1
  |
1 | pub trait Crash {
  |                 - item list starts here
...
5 | }
  | ^
  | |
  | non-item starts here
  | item list ends here

error: expected one of `!` or `::`, found `:`
 --> src/lib.rs:2:5
  |
2 |     #[proc_macro::mac]
  |     ^^^^^^^^^^^^^^^^^^

error: aborting due to 3 previous errors

error: could not compile `issue`

To learn more, run the command again with --verbose.

Note that I tried a slightly simpler version of the small repro above:

pub trait Crash {
    #[proc_macro::mac]
    fn one(s: () {
    }
}

This still fails with ICE with the assertion.

leftwo · 2021-09-15T04:04:26Z

I think I'm seeing the same issue with code containing a similar syntax error:

use tracing::instrument;

struct al;

impl al {
    #[instrument]
    fn ox(&mut self {
    }
}

The error is the ox fn does not have the closing paren.

I see this on both release and debug builds:
stable: 1.55.0
beta: 1.56.0-beta.2
Nightly: 1.57.0-nightly (c3c0f80d6 2021-09-14)

Here is the error and stack from Rust Playground:

   Compiling playground v0.0.1 (/playground)
error: expected one of `)` or `,`, found `{`
 --> src/lib.rs:7:10
  |
7 |     fn ox(&mut self {
  |          ^         -^
  |          |         |
  |          |         help: `)` may belong here
  |          unclosed delimiter

error: non-item in item list
 --> src/lib.rs:9:1
  |
5 | impl al {
  |         - item list starts here
...
9 | }
  | ^
  | |
  | non-item starts here
  | item list ends here

thread 'rustc' panicked at 'Stack should be empty: final_buf=FrameData { open: src/lib.rs:7:10: 7:11 (#0), open_delim: Paren, inner: [(Token(Token { kind: BinOp(And), span: src/lib.rs:7:11: 7:12 (#0) }), Alone), (Token(Token { kind: Ident("mut", false), span: src/lib.rs:7:12: 7:15 (#0) }), Alone), (Token(Token { kind: Ident("self", false), span: src/lib.rs:7:16: 7:20 (#0) }), Alone), (Delimited(DelimSpan { open: src/lib.rs:7:21: 7:22 (#0), close: src/lib.rs:8:5: 8:6 (#0) }, Brace, AttrAnnotatedTokenStream([])), Alone)] } stack=[FrameData { open: no-location (#0), open_delim: NoDelim, inner: [(Token(Token { kind: Ident("fn", false), span: src/lib.rs:7:5: 7:7 (#0) }), Alone), (Token(Token { kind: Ident("ox", false), span: src/lib.rs:7:8: 7:10 (#0) }), Alone)] }]', compiler/rustc_parse/src/parser/attr_wrapper.rs:501:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/c3c0f80d6081092faff801542dd82f0e2420152b/library/std/src/panicking.rs:517:5
   1: std::panicking::begin_panic_fmt
             at /rustc/c3c0f80d6081092faff801542dd82f0e2420152b/library/std/src/panicking.rs:460:5
   2: <rustc_parse::parser::attr_wrapper::LazyTokenStreamImpl as rustc_ast::tokenstream::CreateTokenStream>::create_token_stream
   3: rustc_parse::prepend_attrs
   4: rustc_parse::nt_to_tokenstream
   5: rustc_expand::base::Annotatable::into_tokens
   6: rustc_expand::expand::MacroExpander::fully_expand_fragment
   7: rustc_expand::expand::MacroExpander::expand_crate
   8: rustc_session::utils::<impl rustc_session::session::Session>::time
   9: rustc_interface::passes::configure_and_expand
  10: rustc_interface::queries::Queries::expansion
  11: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter
  12: rustc_span::with_source_map
  13: scoped_tls::ScopedKey<T>::set
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.57.0-nightly (c3c0f80d6 2021-09-14) running on x86_64-unknown-linux-gnu

note: compiler flags: -C opt-level=3 -C embed-bitcode=no -C codegen-units=1 --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack

…aron1011 Manually cleanup token stream when macro expansion aborts. In case of syntax error in macro expansion, the expansion code can decide to stop processing anything. In that case, the token stream is malformed. This makes downstream users, like derive macros, ICE. In this case, this PR manually cleans up the token stream by closing all currently open delimiters. Fixes rust-lang#96818. Fixes rust-lang#80447. Fixes rust-lang#81920. Fixes rust-lang#91023.

Manually cleanup token stream when macro expansion aborts. In case of syntax error in macro expansion, the expansion code can decide to stop processing anything. In that case, the token stream is malformed. This makes downstream users, like derive macros, ICE. In this case, this PR manually cleans up the token stream by closing all currently open delimiters. Fixes rust-lang/rust#96818. Fixes rust-lang/rust#80447. Fixes rust-lang/rust#81920. Fixes rust-lang/rust#91023.

ijackson added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Dec 28, 2020

jonas-schievink added A-parser Area: The parsing of Rust source code to an AST E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example labels Dec 28, 2020

rustbot added A-proc-macros Area: Procedural macros and removed E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example labels Dec 29, 2020

fanninpm added a commit to fanninpm/glacier that referenced this issue Dec 29, 2020

Add 80447

973f222

Issue: rust-lang/rust#80447

rust-lang-glacier-bot added the glacier ICE tracked in rust-lang/glacier. label Dec 31, 2020

leftwo mentioned this issue Sep 14, 2021

ICE with instrument tokio-rs/tracing#1472

Open

cjgillot mentioned this issue Aug 7, 2022

Manually cleanup token stream when macro expansion aborts. #100250

Merged

bors closed this as completed in 14b27cf Sep 20, 2022

github-actions bot mentioned this issue Sep 21, 2022

ices/80447.sh: fixed with errors rust-lang/glacier#1421

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stack overflow? in parser in some code with syntax errors #80447

Stack overflow? in parser in some code with syntax errors #80447

ijackson commented Dec 28, 2020

durka commented Dec 29, 2020 •

edited

Loading

ijackson commented Dec 29, 2020

durka commented Dec 29, 2020

osa1 commented Jan 27, 2021

leftwo commented Sep 15, 2021

Stack overflow? in parser in some code with syntax errors #80447

Stack overflow? in parser in some code with syntax errors #80447

Comments

ijackson commented Dec 28, 2020

Steps to reproduce

Meta

Error output

The panic message in full

durka commented Dec 29, 2020 • edited Loading

ijackson commented Dec 29, 2020

durka commented Dec 29, 2020

osa1 commented Jan 27, 2021

leftwo commented Sep 15, 2021

durka commented Dec 29, 2020 •

edited

Loading