Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Procedural macros 1.1 #1681

Merged
merged 8 commits into from
Aug 22, 2016
Merged

Conversation

alexcrichton
Copy link
Member

Extract a very small sliver of today's procedural macro system in the compiler,
just enough to get basic features like custom derive working, to have an
eventually stable API. Ensure that these features will not pose a maintenance
burden on the compiler but also don't try to provide enough features for the
"perfect macro system" at the same time. Overall, this should be considered an
incremental step towards an official "plugins 2.0".

Rendered

Extract a very small sliver of today's procedural macro system in the compiler,
just enough to get basic features like custom derive working, to have an
eventually stable API. Ensure that these features will not pose a maintenance
burden on the compiler but also don't try to provide enough features for the
"perfect macro system" at the same time. Overall, this should be considered an
incremental step towards an official "plugins 2.0".
@alexcrichton alexcrichton added T-compiler Relevant to the compiler team, which will review and decide on the RFC. T-lang Relevant to the language team, which will review and decide on the RFC. labels Jul 18, 2016
@alexcrichton alexcrichton self-assigned this Jul 18, 2016
@aturon
Copy link
Member

aturon commented Jul 18, 2016

Suggestion: this might want to be titled Plugins 1.1, as it doesn't really impact the "macro" system (if you think of that as macro_rules, which I think many do).

@aturon
Copy link
Member

aturon commented Jul 18, 2016

cc @jimmycuadra

@eddyb
Copy link
Member

eddyb commented Jul 18, 2016

@aturon I believe this is because @nrc wanted to avoid "plugins" for "procedural macros".

@alexcrichton
Copy link
Member Author

@aturon yeah it's true that this only really affects what we call plugins today, but I believe that @nrc wanted to start avoiding the term "plugin" as everything will eventually be a macro in the "macros 2.0" world.

* Sharing an API of the entire compiler
* Frozen interface between the compiler and macros

### `librustc_macro`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name is unfortunate, couldn't we use libmacro?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is currently done because macro is a keyword, and this doesn't propose changing that just yet. I was thinking that we could perhaps wholesale rename the crate one day as well if macro becomes an available name.

Copy link

@MovingtoMars MovingtoMars Jul 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about libmacros? This follows the approach that Go takes, ie the strings package.

@aturon
Copy link
Member

aturon commented Jul 18, 2016

Thanks @alexcrichton for the awesome writeup! We've been working for a while on some way enable usage of Serde and similar libraries on stable in the near term, without massive stabilization. For example, there's a code generation proposal (which sits entirely on the side from the macro/plugin system), as well as attempts to find a very small increment toward "macros 2.0". But this is the first proposal that can be implemented immediately, has a clear path toward speedy stabilization, and imposes virtually no maintenance burden. I'm incredibly excited to be so close to a solution for the biggest cause of nightly usage today.

One thing that wasn't totally obvious to me, given that the macros 2.0 plan is a bit out of cache: what, if any, parts of what you propose to stabilize here would end up being deprecated once macros 2.0 has fully landed?

(To be clear, I have zero problem stabilizing the small slice of functionality you've outlined here in the near term even if in the long term we plan to deprecate it.)

such as:

```rust
extern crate double;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would've expected at least #[macro_use] to be necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @nrc, I think that importing custom derive annotations was a somewhat unspecified extension of "macros 2.0" right now, so to me it seems somewhat up in the air about whether we'd require that or not.

I don't mind one way or another, but I'm curious what @nrc thinks as well.

Eventually of course custom-derive will be imported through the normal module system I believe, so this is purely just a temporary stopgap.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that if you do anything by default now, you'll need to opt out to be able to use the normal module system.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha, an excellent point! You've convinced me, I'll add it.

@nrc nrc changed the title RFC: Macros 1.1 RFC: Procedural macros 1.1 Jul 18, 2016
@alexcrichton
Copy link
Member Author

@aturon

what, if any, parts of what you propose to stabilize here would end up being deprecated once macros 2.0 has fully landed?

This is a good question! Of the pieces here, what I would imagine is:

  • If we decide to rename anything, we'd accumulate deprecations for old names.
  • The behavior of "go to a string, parse, then go back to a token stream" will likely be deprecated in favor of directly manipulating the token stream.
  • The definition form of custom #[derive] will likely change and the proposed form here would be deprecated. We likely eventually want one that's imported via use (somehow).

Other than that though I imagine these pieces to survive stabilization:

  • The concept of a "macro" crate type
  • The concept of a "macro crate" provided in the standard distribution, providing all details necessary to talk to the compiler.
  • The ability to inform Cargo of crates which are macro crates
  • Cargo's behavior of compiling macro crates for the host and passing them to the compiler for the next unit of compilation.
  • Transformers like #[derive] operating over token streams and allowing customization via attributes that are stripped during the expansion process

@strega-nil
Copy link

strega-nil commented Jul 18, 2016

First, I don't think this RFC has gone through enough design, and fast-tracking it just to get people off of nightly is not a good idea, IMO. However, I am in favor of eventually (soon-ish) getting this sliver.

Some of my specific issues:

First: librustc_macro is a terrible stable name! My vote's in for libmacro. It's not a rustc-specific thing, it's an all-of-Rust thing!

Second: Context seems like an unnecessary implementation detail, and, if they're used the same way as in the original proposal "Procedural macros should report errors, warnings, etc. via the MacroContext [Context]."... why. We already have an existing error reporting mechanism, with Results. This new way seems very un-Rustic.

Third: I don't like how macros are defined. I'd rather they be defined with something like macro instead, especially as we eventually are going to allow proc macros and normal stuff in the same library.

@eddyb
Copy link
Member

eddyb commented Jul 18, 2016

@ubsan I don't like Context either, but Result won't cut it, macros 2.0 will eventually want a full-blown diagnostics API, where you can emit multiple warnings and non-fatal errors.
If the necessary information is placed in TLS, the API would work without a Context type.

@sfackler
Copy link
Member

@ubsan Result is an all-or-nothing thing. In the compilation process, it is very common to be creating many diagnostics during one compilation pass. A compilation error is not necessarily fatal - you (within reason) want to continue as long as possible to avoid forcing the user to go through a recompilation cycle for every error when many could all be reported at once. In addition, warnings, notes, etc are certainly not fatal.

@arielb1
Copy link
Contributor

arielb1 commented Jul 18, 2016

I think librust_macro could be a fine name.

On the integration level: plugins imply some form of host ABI. Stable plugins mean a stable HOSTRUSTC. I think we can just leave HOSTRUSTC as an implementation-defined thing, but we must be careful not to change its definition on existing platforms.

@strega-nil
Copy link

@sfackler makes sense. I still don't think Context is the best, though.

@sfackler
Copy link
Member

Do you have any suggestions for a better replacement?

@strega-nil
Copy link

@sfackler No, but I think @eddyb does :P

@arielb1
Copy link
Contributor

arielb1 commented Jul 18, 2016

On naming: I see librust_macro as an ordinary plugin library, and rustc_macro_derive as an exposed proc macro that (unstable) compiler plugins should eventually be able to reproduce (BTW, what is the nomenclature wrt. proc macro vs. plugin vs. compiler extension?)

rustc_macro_derive is a bad name, but we still don't have modular macros. I think pseudo-modular #[WHATEVER_CRATE_NAME_WE_PICK_derive] would work.

As for the API: maybe have #[CRATE_derive_v0] for the Context-free version?

@sfackler
Copy link
Member

How does shoving Context into TLS hide implementation details?

@eddyb
Copy link
Member

eddyb commented Jul 18, 2016

@sfackler The TLS slot wouldn't be exposed, the APIs would just not take any context.

@sfackler
Copy link
Member

@ubsan's concern with Context seemed to be that it was an implementation detail, and I'm confused as to how moving all of the methods on Context to free functions hides any details of the implementation. It seems totally isomorphic in that regard.

@arielb1
Copy link
Contributor

arielb1 commented Jul 18, 2016

Yeah. A Context parameter is easy to ignore and hard to add later.

@nrc
Copy link
Member

nrc commented Aug 18, 2016

I'm in favour of this. I'm not sure how I feel about the namespacing issue, I'd like to see how this looks when implemented.

@nikomatsakis
Copy link
Contributor

Huzzah! The @rust-lang/lang team has decided to accept this RFC. We decided to ask @alexcrichton to move some of the detailed matters (e.g., prefixes, etc) into unresolved questions to be addressed during stabilization. Thanks everyone!

@nikomatsakis nikomatsakis merged commit ac22573 into rust-lang:master Aug 22, 2016
@nikomatsakis
Copy link
Contributor

Tracking issue: rust-lang/rust#35900

If you'd like to keep following the development of this feature, please subscribe to that issue, thanks! :)

nikomatsakis added a commit that referenced this pull request Aug 22, 2016
@llogiq
Copy link
Contributor

llogiq commented Aug 22, 2016

I still miss my comment about the inferior traceability of code through macros 1.1 as opposed to the current systems. Could we at least add it to the drawbacks section?

@nikomatsakis
Copy link
Contributor

@llogiq I pushed some text about spans

@llogiq
Copy link
Contributor

llogiq commented Aug 22, 2016

Ok then. I'll keep an eye on the implementation. 😉

untitaker added a commit to untitaker/rfcs that referenced this pull request Sep 11, 2016
nrc added a commit that referenced this pull request Sep 11, 2016
hauleth added a commit to rust-num/num that referenced this pull request Sep 18, 2016
Current solution follow syntax proposed in rust-lang/rfcs#1681.

Tracking issue rust-lang/rust#35900

Close #227
hauleth added a commit to rust-num/num that referenced this pull request Sep 18, 2016
Current solution follow syntax proposed in rust-lang/rfcs#1681.

Tracking issue rust-lang/rust#35900

Close #227
hadronized pushed a commit to hadronized/rfcs that referenced this pull request Feb 10, 2017
hadronized pushed a commit to hadronized/rfcs that referenced this pull request Feb 10, 2017
@Centril Centril added A-macros Macro related proposals and issues A-proc-macros Proc macro related proposals & ideas labels Nov 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-macros Macro related proposals and issues A-proc-macros Proc macro related proposals & ideas final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. T-compiler Relevant to the compiler team, which will review and decide on the RFC. T-lang Relevant to the language team, which will review and decide on the RFC.
Projects
None yet
Development

Successfully merging this pull request may close these issues.