Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce TtParser #95159

Merged
merged 7 commits into from
Mar 23, 2022
Merged

Introduce TtParser #95159

merged 7 commits into from
Mar 23, 2022

Commits on Mar 18, 2022

  1. Introduce TtParser.

    It currently has no state, just the three methods `parse_tt`,
    `parse_tt_inner`, and `bb_items_ambiguity_error`.
    
    This commit is large but trivial, and mostly consists of changes to the
    indentation of those methods. Subsequent commits will do more.
    nnethercote committed Mar 18, 2022
    Configuration menu
    Copy the full SHA
    d21b4f3 View commit details
    Browse the repository at this point in the history
  2. Rename bb_items_ambiguity_error as ambiguity_error.

    Because it involves `next_items` as well as `bb_items`.
    nnethercote committed Mar 18, 2022
    Configuration menu
    Copy the full SHA
    354bd10 View commit details
    Browse the repository at this point in the history
  3. Add TtParser::macro_name.

    Instead of passing it into `parse_tt`.
    nnethercote committed Mar 18, 2022
    Configuration menu
    Copy the full SHA
    39810a8 View commit details
    Browse the repository at this point in the history
  4. Remove an impossible code path.

    Doc comments cannot appear in a matcher.
    nnethercote committed Mar 18, 2022
    Configuration menu
    Copy the full SHA
    10644e0 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2022

  1. Remove MatcherPosHandle.

    This type was a small performance win for `html5ever`, which uses a
    macro with hundreds of very simple rules that don't contain any
    metavariables. But this type is complicated (extra lifetimes) and
    perf-neutral for macros that do have metavariables.
    
    This commit removes `MatcherPosHandle`, simplifying things a lot. This
    increases the allocation rate for `html5ever` and similar cases a bit,
    but makes things easier for follow-up changes that will improve
    performance more than what we lost here.
    nnethercote committed Mar 20, 2022
    Configuration menu
    Copy the full SHA
    cedb787 View commit details
    Browse the repository at this point in the history
  2. Move items into TtParser as Vecs.

    By putting them in `TtParser`, we can reuse them for every rule in a
    macro. With that done, they can be `SmallVec` instead of `Vec`, and this
    is a performance win because these vectors are hot and `SmallVec`
    operations are a bit slower due to always needing an "inline or heap?"
    check.
    nnethercote committed Mar 20, 2022
    Configuration menu
    Copy the full SHA
    754dc8e View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2022

  1. Eliminate TokenTreeOrTokenTreeSlice.

    As its name suggests, `TokenTreeOrTokenTreeSlice` is either a single
    `TokenTree` or a slice of them. It has methods `len` and `get_tt` that
    let it be treated much like an ordinary slice. The reason it isn't an
    ordinary slice is that for `TokenTree::Delimited` the open and close
    delimiters are represented implicitly, and when they are needed they are
    constructed on the fly with `Delimited::{open,close}_tt`, rather than
    being present in memory.
    
    This commit changes `Delimited` so the open and close delimiters are
    represented explicitly. As a result, `TokenTreeOrTokenTreeSlice` is no
    longer needed and `MatcherPos` and `MatcherTtFrame` can just use an
    ordinary slice. `TokenTree::{len,get_tt}` are also removed, because they
    were only needed to support `TokenTreeOrTokenTreeSlice`.
    
    The change makes the code shorter and a little bit faster on benchmarks
    that use macro expansion heavily, partly because `MatcherPos` is a lot
    smaller (less data to `memcpy`) and partly because ordinary slice
    operations are faster than `TokenTreeOrTokenTreeSlice::{len,get_tt}`.
    nnethercote committed Mar 22, 2022
    Configuration menu
    Copy the full SHA
    31df680 View commit details
    Browse the repository at this point in the history