Some parser cleanups #127233
Merged
Some parser cleanups #127233
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It can contain an owned value instead of a reference.
I.e. change the return type from `TokenStream` to `Vec<TokenTree>`. Most of the callsites require a `TokenStream`, but the recursive call used to create `target_tokens` requires a `Vec<TokenTree>`. It's easy to convert a `Vec<TokenTree>` to a `TokenStream` (just call `TokenStream::new`) but it's harder to convert a `TokenStream` to a `Vec<TokenTree>` (either iterate/clone/collect, or use `Lrc::into_inner` if appropriate). So this commit changes the return value to simplify that `target_tokens` call site.
`tts` is a better name than `streams` for a `Vec<TokenTree>`.
Currently it uses a mixture of functional style (`flat_map`) and imperative style (`push`), which is a bit hard to read. This commit converts it to fully imperative, which is more concise and avoids the need for `smallvec`.
Both the indenting, and the missing `)`.
To put the simple case first.
So that the `capturing` state is adjusted immediately before and after the call to `f`.
To make things a little clearer, and to avoid some `mut` variables.
rustbot
added
S-waiting-on-review
The number of source code bytes can't exceed a `u32`'s range, so a token position also can't. This reduces the size of `Parser` and `LazyAttrTokenStreamImpl` by eight bytes each.
nnethercote
force-pushed
the
parser-cleanups
branch
from
9ba9511 to
3d750e2
Compare
And update the comment. Clearly the return type of this function was changed at some point in the past, but its name and comment weren't updated to match.
petrochenkov
approved these changes
Jul 2, 2024
|
@bors r+ |
|
🌲 The tree is currently closed for pull requests below priority 100. This pull request will be tested once the tree is reopened. |
bors
added
S-waiting-on-bors
petrochenkov
reviewed
Jul 2, 2024
|
@bors r=petrochenkov |
|
🌲 The tree is currently closed for pull requests below priority 100. This pull request will be tested once the tree is reopened. |
matthiaskrgr
added a commit
to matthiaskrgr/rust
that referenced
this pull request
Jul 2, 2024
…rochenkov Some parser cleanups Cleanups I made while looking closely at this code. r? `@petrochenkov`
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Jul 3, 2024
…iaskrgr Rollup of 9 pull requests Successful merges: - rust-lang#126403 (Actually report normalization-based type errors correctly for alias-relate obligations in new solver) - rust-lang#126803 (Change `asm-comments` to `verbose-asm`, always emit user comments) - rust-lang#126917 (Disable rmake test `inaccessible-temp-dir` on riscv64) - rust-lang#127050 (Make mtime of reproducible tarballs dependent on git commit) - rust-lang#127145 (Add `as_lang_item` to `LanguageItems`, new trait solver) - rust-lang#127184 (More refactorings to rustc_interface) - rust-lang#127202 (Remove global error count checks from typeck) - rust-lang#127233 (Some parser cleanups) - rust-lang#127245 (Add a test for `generic_const_exprs`) r? `@ghost` `@rustbot` modify labels: rollup
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Jul 3, 2024
…iaskrgr Rollup of 9 pull requests Successful merges: - rust-lang#126403 (Actually report normalization-based type errors correctly for alias-relate obligations in new solver) - rust-lang#126803 (Change `asm-comments` to `verbose-asm`, always emit user comments) - rust-lang#126917 (Disable rmake test `inaccessible-temp-dir` on riscv64) - rust-lang#127050 (Make mtime of reproducible tarballs dependent on git commit) - rust-lang#127145 (Add `as_lang_item` to `LanguageItems`, new trait solver) - rust-lang#127184 (More refactorings to rustc_interface) - rust-lang#127202 (Remove global error count checks from typeck) - rust-lang#127233 (Some parser cleanups) - rust-lang#127245 (Add a test for `generic_const_exprs`) r? `@ghost` `@rustbot` modify labels: rollup
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Jul 3, 2024
…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#126803 (Change `asm-comments` to `verbose-asm`, always emit user comments) - rust-lang#127050 (Make mtime of reproducible tarballs dependent on git commit) - rust-lang#127145 (Add `as_lang_item` to `LanguageItems`, new trait solver) - rust-lang#127202 (Remove global error count checks from typeck) - rust-lang#127233 (Some parser cleanups) - rust-lang#127248 (Add parse fail test using safe trait/impl trait) - rust-lang#127264 (Small `run-make-support` API improvements) - rust-lang#127270 (bootstrap: pass correct struct size to winapi) r? `@ghost` `@rustbot` modify labels: rollup
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Jul 3, 2024
Rollup merge of rust-lang#127233 - nnethercote:parser-cleanups, r=petrochenkov Some parser cleanups Cleanups I made while looking closely at this code. r? ``@petrochenkov``
nnethercote
added a commit
to nnethercote/rust
that referenced
this pull request
Jul 4, 2024
These should have been removed in rust-lang#127233 when the positions were changed from `usize` to `u32`.
nnethercote
added a commit
to nnethercote/rust
that referenced
this pull request
Jul 4, 2024
These should have been removed in rust-lang#127233 when the positions were changed from `usize` to `u32`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cleanups I made while looking closely at this code.
r? @petrochenkov