Trying to expand macro in extern block causes excess memory usage

[WarpspeedSCP]

rustc --version --verbose output:

rustc 1.30.0-nightly (3bc2ca7e4 2018-09-20)
binary: rustc
commit-hash: 3bc2ca7e4f8507f82a4c357ee19300166bcd8099
commit-date: 2018-09-20
host: x86_64-unknown-linux-gnu
release: 1.30.0-nightly
LLVM version: 8.0

While trying to implement Lua in Rust, I wanted to write a macro to make writing function signatures easier. When I tested the macro within an extern "C" block, and ran rustc --pretty expanded -Z unstable-options, my computer froze due to all (8 gigs) of my RAM being used up by rustc . An MCVE-

#![feature(macros_in_extern)]
#![feature(concat_idents)]

macro_rules! lua_func {
    ($name: ident, $ret: ty, $var: ident, $type: ty) => {
        let fn_name = concat_idents!(lua_, $name);
        pub fn fn_name (L: *mut lua_State, $var: $type) -> $ret
    };
}

extern "C" {
    lua_func!(toboolean, bool, idx, int);
}

[dtolnay]

Minimized:

#![feature(macros_in_extern)]

macro_rules! m {
    () => {
        let
    };
}

extern "C" {
    m!();
}

[abonander]

This might well be my fault, so I'll try to look at it sometime this week.

[abonander]

I've been able to repro the issue but I haven't run it through an allocation profiler yet*. The latest master doesn't like to build with my local Rust install so I spent a lot of time last night trying to figure out why but it finally worked when I gave up and just let it download a snapshot for stage 0.

[abonander]

So far it looks like the leak involves Parser::parse_foreign_item() calling Parser::check_keyword() repeatedly (which pushes to Parser::expected_keywords every time), because parse_foreign_item() isn't erroring and isn't advancing the parser state and thus this loop runs forever.

Simple fix is to check at the end of parse_foreign_item() what the current token is and error if it's not a closing brace or Eof.

[abonander]

Parser::parse_foreign_item() is a bit of an enimga in that it returns Result<Option<ForeignItem>> whereas the analogues for TraitItem and ImplItem are required to return an item or error, and the caller is responsible for detecting EOF. I think that caused some confusion in ext/expand.rs as a result. parse_foreign_item() should be refactored to have the same obligatory return.