Factor TokenTrees into their own file in preparation for TokenStreams. #34385
Conversation
|
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @nrc (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
|
cc @Manishearth (breaking batch) |
|
r=me |
| use print::pprust; | ||
| use ptr::P; | ||
| use tokenstream::{TokenTree}; |
There was a problem hiding this comment.
I'll make the same comment as last time: is it worth changing this use to a pub use in order to avoid the breakage?
There was a problem hiding this comment.
IMO, no. Token streams will eventually be added to tokenstream.rs, and hopefully places that currently use tokens trees will generally move to using token streams. Looking forward, procedural macros will shift to work over token streams, and the breakage here is (at least partially) intended to ensure that tokentree actions / procedural macros no longer need to rely on ast.rs in any way.
|
travis fails |
…reams. Modified tests to point to the new file now.
|
@Manishearth Fixed the travis failure. |
syntax-[breaking-change] cc rust-lang#31645 (Only breaking because ast::TokenTree is now tokenstream::TokenTree.) This pull request refactors TokenTrees into their own file as src/libsyntax/tokenstream.rs, moving them out of src/libsyntax/ast.rs, in order to prepare for an accompanying TokenStream implementation (per RFC 1566).
Batch up libsyntax breaking changes Batch of the following syntax-[breaking-change] changes: - #34213: Add a variant `Macro` to `TraitItemKind` - #34368: Merge the variant `QPath` of `PatKind` into the variant `PatKind::Path` - #34385: Move `syntax::ast::TokenTree` into a new module `syntax::tokenstream` - #33943: - Remove the type parameter from `visit::Visitor` - Remove `attr::WithAttrs` -- use `attr::HasAttrs` instead. - Change `fold_tt`/`fold_tts` to take token trees by value and avoid wrapping token trees in `Rc`. - Remove the field `ctxt` of `ast::Mac_` - Remove inherent method `attrs()` of types -- use the method `attrs` of `HasAttrs` instead. - #34316: - Remove `ast::Decl`/`ast::DeclKind` and add variants `Local` and `Item` to `StmtKind`. - Move the node id for statements from the `StmtKind` variants to a field of `Stmt` (making `Stmt` a struct instead of an alias for `Spanned<StmtKind>`) - Rename `ast::ExprKind::Again` to `Continue`. - #34339: Generalize and abstract `ThinAttributes` to `ThinVec<Attribute>` - Use `.into()` in convert between `Vec<Attribute>` and `ThinVec<Attribute>` - Use autoderef instead of `.as_attr_slice()` - #34436: Remove the optional expression from `ast::Block` and instead use a `StmtKind::Expr` at the end of the statement list. - #34403: Move errors into a separate crate (unlikely to cause breakage)
Batch up libsyntax breaking changes Batch of the following syntax-[breaking-change] changes: - #34213: Add a variant `Macro` to `TraitItemKind` - #34368: Merge the variant `QPath` of `PatKind` into the variant `PatKind::Path` - #34385: Move `syntax::ast::TokenTree` into a new module `syntax::tokenstream` - #33943: - Remove the type parameter from `visit::Visitor` - Remove `attr::WithAttrs` -- use `attr::HasAttrs` instead. - Change `fold_tt`/`fold_tts` to take token trees by value and avoid wrapping token trees in `Rc`. - Remove the field `ctxt` of `ast::Mac_` - Remove inherent method `attrs()` of types -- use the method `attrs` of `HasAttrs` instead. - #34316: - Remove `ast::Decl`/`ast::DeclKind` and add variants `Local` and `Item` to `StmtKind`. - Move the node id for statements from the `StmtKind` variants to a field of `Stmt` (making `Stmt` a struct instead of an alias for `Spanned<StmtKind>`) - Rename `ast::ExprKind::Again` to `Continue`. - #34339: Generalize and abstract `ThinAttributes` to `ThinVec<Attribute>` - Use `.into()` in convert between `Vec<Attribute>` and `ThinVec<Attribute>` - Use autoderef instead of `.as_attr_slice()` - #34436: Remove the optional expression from `ast::Block` and instead use a `StmtKind::Expr` at the end of the statement list. - #34403: Move errors into a separate crate (unlikely to cause breakage)
syntax-[breaking-change] cc #31645
(Only breaking because ast::TokenTree is now tokenstream::TokenTree.)
This pull request refactors TokenTrees into their own file as src/libsyntax/tokenstream.rs, moving them out of src/libsyntax/ast.rs, in order to prepare for an accompanying TokenStream implementation (per RFC 1566).
r? @nrc
(Note: duplicate pull request because I destroyed the previous branch last time on accident.)