Move the reference into a submodule

[steveklabnik]

Part of #39588

This moves the reference out into its own submodule. Right now, it's under my own account; I would assume that it should go under rust-lang, but I guess there's an argument for the nursery too? I can fix that up before we commit it.

r? @alexcrichton @brson

[steveklabnik]

Oh, and once this ships, gonna do the exact same with the nomicon.

[petrochenkov]

Why?!

First splitting a single page into small pieces, then breaking table of contents, now moving the reference into a separate repo.

Now you can't just modify the reference when adding, or changing, or stabilizing some language feature, you have to go and clone @steveklabnik's repo and submit a PR there and wait until it's merged, and then return to rust-lang/rust and update submodules. Do you think this will affect reference quality and up-to-date-ness positively?

EDIT: Removed that one sentence, sorry @steveklabnik. I probably need to get a habit of waiting for a day before answering to anything controversial.

[steveklabnik]

@petrochenkov this was all discussed as part of rust-lang/rfcs#1828

Many people have wanted docs out of the repo for a long time; and yes, overall, we do.

Stabilizing a new language feature is the only time there needs to be inter-repo coordination, and that happens relatively rarely.

[nagisa]

Do you think this will affect reference quality and up-to-date-ness positively?

Not like anybody thinks of updating reference for any change anyway. /me shrugss

[aturon]

@petrochenkov

Why every time the docs team touches the reference infrastructure, things become worse?

I understand and care about your frustration, and we should work to improve matters, but I want once more to make clear that this kind of belittling statement is not ok. It's important to keep a couple things in mind:

• We all care about Rust, its quality, and the quality of the infrastructure. People make mistakes, but we're on the same team here.

• Comments like these, which may seem harmless in isolation, really add up over time. One thing about working in open source is that everything you do and say is exposed, constantly. Criticism gets multiplied by the size of the community, and can happen at any time -- and there's no skin that's thick enough to withstand it at scale. In aggregate it can become really demoralizing and burn people out. And that doesn't do anyone any good.

This is why empathy is vital, and we have to do everything we can to address real technical problems without demeaning people. Even if this is just your personal style, it is counterproductive for addressing your technical concerns, and it is not sustainable.

[petrochenkov]

I've read the RFC and see at least several people mentioning that it's better kept in tree (rust-lang/rfcs#1828 (comment), rust-lang/rfcs#1828 (comment), rust-lang/rfcs#1828 (comment), rust-lang/rfcs#1828 (comment)) and @steveklabnik giving an answer (rust-lang/rfcs#1828 (comment)) that the reference is going to rarely modified since it should not contain unstable stuff, which is a pretty strange statement, since the reference needs to describe full language and not just its restricted stable subset.

I understand the motivation to keep all the docs together and convert them in a single mdbook style, but in practice all the changes to the reference done so far turned out to be disimprovements, for both readers and writers.
This issue alone is not super important (previously I bothered to update the reference when working on the compiler, but not very often, now I'm not sure), but they just continue summing up.

[steveklabnik]

that the reference is going to rarely modified since it should not contain unstable stuff, which is a pretty strange statement, since the reference needs to describe full language and not just its restricted stable subset.

I think this is a big disconnect here; the reference is supposed to describe the language, but unstable things are not in the language yet. This is one of the reasons why the Unstable Book was created; to be able to have a home for things that don't yet exist.

Furthermore, please understand that all of this is in transition, that is, the TOC issue will be fixed, for example. But before we can make significant changes to the reference and how it's structured, these steps need to be done; part of the whole idea of moving things out of tree is that those who care about improving the reference have one place to track issues, make improvements, etc. Right now, modifying the reference's text goes through the homu queue, slowing turnaround times on PRs and (moderately) clogging up the queue for non-doc related PRs.

For another example of a pain point this solves, currently, modifying the reference requires a full x.py doc, which involves a full build of rust, which involves a build of LLVM... this overall decreases the amount of work it takes to improve the reference, which can only help improve it.

[petrochenkov]

unstable things are not in the language yet

This is even more strange statement, but I guess we have a fundamental disagreement here.

For another example of a pain point this solves, currently, modifying the reference requires a full x.py doc...

Something something rustbuild something something.
If this is the reason why people usually contributing to other docs payed less attention to the reference so far, then separating it may indeed be a good idea. On the other hand, library docs continue to be in-source and they are actively updated despite the build issue + historically reference was updated by people working on the compiler (less often than everyone would like though) and they have to do a full build anyway.
Ok, let's see what will happen in practice when the reference is moved.

[alexcrichton]

On the technical side of things we currently prefer submodules to reside in rust-lang-nursery or rust-lang to hopefully improve our changes of keeping this commit building until the end of time. I think for now we can start out in the nursery and then eventually move to rust-lang once we feel that it's ready? (or do we feel this is rust-lang ready?)

Also, just to reiterate to make sure we're on the same page, our current policy with submodules is that any pointer we check into tree should be long-lasting and preserved forever (in theory). Basically that means that any commit checked in should always be reachable from some branch in the remote repo. If that means we only ever point to the master branch and force-pushes are disabled, that's totally fine. LLVM, for example, we maintain branches for each time we update and then we just leave old branches lying around.

(just procedural comments, otherwise this technically lgtm)

[steveklabnik]

(or do we feel this is rust-lang ready?)

@alexcrichton yeah so to me, the reference has been a part of Rust for a very long time, and is not likely to be straight-up removed, even if it is going to eventually undergo a lot of evolution. That is, putting it in the nursery, to me, says "we may throw this away, or we may keep it", with going to rust-lang-deprecated being the former and rust-lang being the latter. I can't imagine that it's going to be thrown away, so I prefer rust-lang. I'm fundamentally okay with wherever.

Also, just to reiterate to make sure we're on the same page, our current policy with submodules is that any pointer we check into tree should be long-lasting and preserved forever (in theory).

👍

[est31]

Putting the reference into a separate repository will make contributing easier for people who do repeated contributions to the reference, and make it harder for people, like me, and @petrochenkov I presume who do occasional contributions to it. Instead of having to fork and clone one rust-lang/rust repository, you now have to fork and clone two repositories. So for me personally, this change will make it less likely to contribute to the reference, but then again I've only done a single commit to it, so it probably won't mean much if future contributions from me are lost.

However, on a positive note, people like me don't have to make contributions, as I think there are enough people more than eager to help out with filing PRs to the various repositories that contain documentation to get a feature stabilized. And especially the possible gains of contributors outweigh the contributors lost.

Question: its already now pretty complicated to get a change merged onto the beta branch: first the change needs to be applied to master, then only it can be considered to be backported. Now its going to be even more complicated. When you want to change the beta reference, e.g. to document a stabilisation on the beta branch (those do occur) you'll have to file a PR to the master reference, then after it was merged another one to the beta reference, then first one to the main rust master branch, and then another one to the beta branch, to update the submodules. Can that be made simpler and easier somehow?

About rust-lang vs rust-lang-nursery: I think if something has previously lived in the main rust tree, then it can be considered stable enough to be put into rust-lang proper. EDIT: obviously wasn't done with the num crate, so idk.

Also, if this change turns out to be bad, then it can be reversed easily, so its not a big problem to get this PR merged.

[steveklabnik]

updated, now that the reference is in rust-lang-nursery.

@est31 that's a good question; I'm happy to go with whatever strategy @alexcrichton wants here.

[alexcrichton]

@bors: r+

[bors]

📌 Commit 52f3dc1 has been approved by alexcrichton

[bors]

☔ The latest upstream changes (presumably #39917) made this pull request unmergeable. Please resolve the merge conflicts.

[steveklabnik]

@bors: r=alexcrichton

[bors]

📌 Commit 846f59f has been approved by alexcrichton

[bors]

⌛ Testing commit 846f59f with merge 83cbaf5...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 83cbaf5 to master...