Skip to content

Commit

Permalink
Rollup merge of rust-lang#45098 - sunjay:breakingrustfmtrls, r=alexcr…
Browse files Browse the repository at this point in the history
…ichton

Documenting the process for when rustfmt/rls break

**DO NOT MERGE YET**

I'm documenting what to do when rustfmt or rls break because of your changes. I'm currently going through this and will keep adding more as I figure out what all the steps are. This first commit is based on @alexcrichton's [comment on my original PR](rust-lang#44766 (comment)).

[Rendered](https://github.com/sunjay/rust/blob/breakingrustfmtrls/CONTRIBUTING.md#breaking-tools-built-with-the-compiler)

Reviews are welcome, but as I mentioned, I will be revising this as I go.
  • Loading branch information
kennytm authored Oct 18, 2017
2 parents 9dfb210 + 790604a commit 578016b
Showing 1 changed file with 110 additions and 0 deletions.
110 changes: 110 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,116 @@ In order to prepare your PR, you can run the build locally by doing
there, you may wish to set `submodules = false` in the `config.toml`
to prevent `x.py` from resetting to the original branch.

#### Breaking Tools Built With The Compiler
[breaking-tools-built-with-the-compiler]: #breaking-tools-built-with-the-compiler

Rust's build system builds a number of tools that make use of the
internals of the compiler. This includes clippy,
[RLS](https://github.com/rust-lang-nursery/rls) and
[rustfmt](https://github.com/rust-lang-nursery/rustfmt). If these tools
break because of your changes, you may run into a sort of "chicken and egg"
problem. These tools rely on the latest compiler to be built so you can't update
them to reflect your changes to the compiler until those changes are merged into
the compiler. At the same time, you can't get your changes merged into the compiler
because the rust-lang/rust build won't pass until those tools build and pass their
tests.

That means that, in the default state, you can't update the compiler without first
fixing rustfmt, rls and the other tools that the compiler builds.

Luckily, a feature was [added to Rust's build](https://github.com/rust-lang/rust/pull/45243)
to make all of this easy to handle. The idea is that you mark the tools as "broken",
so that the rust-lang/rust build passes without trying to build them, then land the change
in the compiler, wait for a nightly, and go update the tools that you broke. Once you're done
and the tools are working again, you go back in the compiler and change the tools back
from "broken".

This should avoid a bunch of synchronization dances and is also much easier on contributors as
there's no need to block on rls/rustfmt/other tools changes going upstream.

Here are those same steps in detail:

1. (optional) First, if it doesn't exist already, create a `config.toml` by copying
`config.toml.example` in the root directory of the Rust repository.
Set `submodules = false` in the `[build]` section. This will prevent `x.py`
from resetting to the original branch after you make your changes. If you
need to [update any submodules to their latest versions][updating-submodules],
see the section of this file about that for more information.
2. (optional) Run `./x.py test src/tools/rustfmt` (substituting the submodule
that broke for `rustfmt`). Fix any errors in the submodule (and possibly others).
3. (optional) Make commits for your changes and send them to upstream repositories as a PR.
4. (optional) Maintainers of these submodules will **not** merge the PR. The PR can't be
merged because CI will be broken. You'll want to write a message on the PR referencing
your change, and how the PR should be merged once your change makes it into a nightly.
5. Update `src/tools/toolstate.toml` to indicate that the tool in question is "broken",
that will disable building it on CI. See the documentation in that file for the exact
configuration values you can use.
6. Commit the changes to `src/tools/toolstate.toml`, **do not update submodules in your commit**,
and then update the PR you have for rust-lang/rust.
7. Wait for your PR to merge.
8. Wait for a nightly
9. (optional) Help land your PR on the upstream repository now that your changes are in nightly.
10. (optional) Send a PR to rust-lang/rust updating the submodule, reverting `src/tools/toolstate.toml` back to a "building" or "testing" state.

#### Updating submodules
[updating-submodules]: #updating-submodules

These instructions are specific to updating `rustfmt`, however they may apply
to the other submodules as well. Please help by improving these instructions
if you find any discrepencies or special cases that need to be addressed.

To update the `rustfmt` submodule, start by running the appropriate
[`git submodule` command](https://git-scm.com/book/en/v2/Git-Tools-Submodules).
For example, to update to the latest commit on the remote master branch,
you may want to run:
```
git submodule update --remote src/tools/rustfmt
```
If you run `./x.py build` now, and you are lucky, it may just work. If you see
an error message about patches that did not resolve to any crates, you will need
to complete a few more steps which are outlined with their rationale below.

*(This error may change in the future to include more information.)*
```
error: failed to resolve patches for `https://github.com/rust-lang-nursery/rustfmt`
Caused by:
patch for `rustfmt-nightly` in `https://github.com/rust-lang-nursery/rustfmt` did not resolve to any crates
failed to run: ~/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path ~/rust/src/bootstrap/Cargo.toml
```

If you haven't used the `[patch]`
section of `Cargo.toml` before, there is [some relevant documentation about it
in the cargo docs](http://doc.crates.io/manifest.html#the-patch-section). In
addition to that, you should read the
[Overriding dependencies](http://doc.crates.io/specifying-dependencies.html#overriding-dependencies)
section of the documentation as well.

Specifically, the following [section in Overriding dependencies](http://doc.crates.io/specifying-dependencies.html#testing-a-bugfix) reveals what the problem is:

> Next up we need to ensure that our lock file is updated to use this new version of uuid so our project uses the locally checked out copy instead of one from crates.io. The way [patch] works is that it'll load the dependency at ../path/to/uuid and then whenever crates.io is queried for versions of uuid it'll also return the local version.
>
> This means that the version number of the local checkout is significant and will affect whether the patch is used. Our manifest declared uuid = "1.0" which means we'll only resolve to >= 1.0.0, < 2.0.0, and Cargo's greedy resolution algorithm also means that we'll resolve to the maximum version within that range. Typically this doesn't matter as the version of the git repository will already be greater or match the maximum version published on crates.io, but it's important to keep this in mind!
This says that when we updated the submodule, the version number in our
`src/tools/rustfmt/Cargo.toml` changed. The new version is different from
the version in `Cargo.lock`, so the build can no longer continue.

To resolve this, we need to update `Cargo.lock`. Luckily, cargo provides a
command to do this easily.

First, go into the `src/` directory since that is where `Cargo.toml` is in
the rust repository. Then run, `cargo update -p rustfmt-nightly` to solve
the problem.

```
$ cd src
$ cargo update -p rustfmt-nightly
```

This should change the version listed in `src/Cargo.lock` to the new version you updated
the submodule to. Running `./x.py build` should work now.

## Writing Documentation
[writing-documentation]: #writing-documentation

Expand Down

0 comments on commit 578016b

Please sign in to comment.