Some rustc-ap dependencies of rustfmt are missing on crates.io

[topecongiro]

The recent refactorings have split libsyntax and its friends into multiple crates. Currently, rustfmt is trying to catch up with those changes. However, after the refactorings, some crates seem to be unavailable on crates.io.

The following is a list of crates that need to be published as rustc-ap-* on crates.io so that rustfmt can update to the latest syntax (as far as I am aware of):

• https://github.com/rust-lang/rust/tree/master/src/librustc_parse
• https://github.com/rust-lang/rust/tree/master/src/librustc_index

[topecongiro]

@alexcrichton Would you mind having a look at this?

Also, it seems that at some point, the publish rate of rustc-ap-* crates has changed to weekly from daily. I am ok with this frequency in general, but in this case, I would like to have the above crates available as soon as possible.

Could you please make a release before the regular schedule when publishing the version with the problem crates?

[est31]

cc @Centril who made the splitting.

@topecongiro rustc_index is published now: https://crates.io/crates/rustc-ap-rustc_index

rustc_parse is a different question. Currently, the publish script publishes all dependencies of libsyntax. Now it also has to publish rustc_parse as well as its dependencies.

[alexcrichton]

I haven't personally followed the changes much to what's going on with refactorings in rustc itself, but this script is auto-run from this repository and should be tweakable by anyone! I'd be happy to take a PR to publish anything new which needs to be published.

[calebcartwright]

Thanks @alexcrichton! I'll work on a PR with the updates shortly (assuming you aren't working on it already @topecongiro)

[topecongiro]

@calebcartwright Thank you 🚀

@alexcrichton Is it possible to add @rust-lang/wg-rustfmt as a collaborator of this repo and an owner of rustc-ap-* crates so that if there is any problem we could fix it and publish them ourselves?

[est31]

@topecongiro you won't need to be crates.io owners as the auto-publishing happens from a script in this very repo. Ability to merge PRs would be enough. This also means a problem though, as the repo contains @alexcrichton 's crates.io token as a secret and with it, the repo's code publishes the crates.

If he'd just flip switches to give you access, anyone in that wg could merge code changes that expose @alexcrichton 's crates.io token before he can stop that person. With that token, the person could publish malicious updates to any of @alexcrichton 's large set of widely used crates.

Thus from a security standpoint, I think that IF you are given access, measures to scope access to rustc-ap-* crates should be applied, e.g. following the steps I describe here rust-lang/crates.io#849 (comment).

[alexcrichton]

@topecongiro as these crates are more heavily depended on by rustfmt and break more often I'm all for sharing the load with maintainership here, I don't have a ton of time to manage it myself! The crates.io token is indeed the thing I worry about, so adding more users with access is something I'd want to do carefully. That being said I try to always be responsive to PRs, this repo just doesn't get a whole lot of PRs :)

Perhaps we could start with PRs to the repo, and if I'm too slow we can figure out a better way of publication?

[topecongiro]

@alexcrichton @est31 Thank you for the explanation. I was not aware of the security concern; thank you for pointing that out.

My concern is that with the current method, we cannot fix broken rustfmt or update its syntax unless going through the process of sending PRs to this or the rustc repository and waiting for the new rustc-ap-* crates publish. If we are unlucky, the process could take one week or longer.

I am considering to create a forked version of rustc that consists of its frontend libraries and replace the usage of rustc-ap-* crate in rustfmt with them. Creating the fork will impose the additional maintenance cost to the rustfmt-wg, though that should be relatively low with a tool similar to this crate. And I find it more natural to maintain syntax libraries that rustfmt depends on by rustfmt-wg.

If the above approach is viable, then this and rustc-ap-* crates could be abandoned, at least from the rustfmt point of view.

@alexcrichton Either way, thank you for creating and maintaining this tool!

[alexcrichton]

@topecongiro FWIW although this is published on a weekly cadence I need only hit one button to publish things, so if you need publication sooner feel free to ping me