Distribute json doc #101799

LukeMathWalker · 2022-09-14T12:31:45Z

Overview

We add a new component, rust-json-docs, to distribute the JSON version of rustdoc's output for public compiler crates (i.e. std, alloc, proc_macro, core and test).
As discussed in #101383, we do not bundle this up as part of the existing rust-docs component since rustdoc's JSON format is still unstable.

Open questions / Doubts

I tried my best, but I never touched this codebase and I couldn't find much documentation on how dist works - I pattern-matched existing code, which might have led to some non-sensical choices in the eyes of people more familiar with the codebase. In particular, I am not sure if my choice of adding a new config flag is appropriate or if the decision to build/not build the JSON docs is more appropriately gated by one of the existing flags.
Any suggestion is more than welcome.

Closes #101383

rust-highfive · 2022-09-14T12:31:49Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jyn514 (or someone else) soon.

Please see the contribution instructions for more information.

GuillaumeGomez · 2022-09-14T13:16:09Z

cc @Mark-Simulacrum

aDotInTheVoid · 2022-09-14T14:56:15Z

@rustbot modify labels: +A-rustdoc-json

jyn514

Thanks! This looks roughly right, but I think we can simplify it quite a bit.

Once you've made the change that builds the JSON docs whenever the normal docs are built, I can run @bors try to generate the rust-json-docs component and verify that it has the output you expect. You can also run x dist rust-json-docs locally and look at the tarball in build/dist.

src/bootstrap/dist.rs

src/bootstrap/doc.rs

config.toml.example

src/bootstrap/config.rs

src/bootstrap/doc.rs

jyn514 · 2022-09-14T22:15:33Z

@rustbot author

src/bootstrap/dist.rs

LukeMathWalker · 2022-09-15T12:40:29Z

@rustbot label -S-waiting-on-author +S-waiting-on-review

jyn514

This looks in a lot better shape, thanks :) just a few small comments.

src/bootstrap/doc.rs

jyn514 · 2022-09-15T14:28:47Z

src/bootstrap/doc.rs

+        if requested_crates.iter().any(|p| p == krate) {
+            // No need to document more of the libraries if we have the one we want.


this looks extremely suspicious, but it was already here before so no need to fix it here.

Yeah, this felt broken to me as well - especially considering that we have a vector of request crates but we abort as soon as one of those is built (instead of making sure that all of them are built).

src/bootstrap/doc.rs

jyn514 · 2022-09-15T18:38:28Z

r=me with that last comment addressed and the commits squashed :) thanks for sticking with this!

LukeMathWalker · 2022-09-16T09:13:35Z

r? @jyn514

LukeMathWalker · 2022-09-16T09:14:09Z

Thank you for guiding me through it @jyn514!

jyn514 · 2022-09-16T14:30:40Z

@bors r+

bors · 2022-09-16T14:30:41Z

📌 Commit ccd4383 has been approved by jyn514

It is now in the queue for this repository.

LukeMathWalker · 2022-09-17T18:38:55Z

I've added a check before attempting the deletion (alternatively I can add try_remove_dir method and handle the error that is returned).
@rustbot label -S-waiting-on-author +S-waiting-on-review

jyn514 · 2022-09-17T18:55:32Z

I noticed that after this change we'll print documenting std twice. Can you fix that while you're at it?

…ed documentation for std crates in nightly toolchains. We also add a new flag to `x doc`, `--json`, to render the JSON-formatted version alongside the HTML-formatted one.

LukeMathWalker · 2022-09-17T19:59:28Z

Fixed it! @jyn514

jyn514 · 2022-09-17T20:37:24Z

@bors r+

bors · 2022-09-17T20:37:25Z

📌 Commit 235dcce has been approved by jyn514

It is now in the queue for this repository.

bors · 2022-09-18T06:01:10Z

⌛ Testing commit 235dcce with merge 8b5ebc275c2ef02347a6a6c1496ea49b411259bd...

bors · 2022-09-18T06:03:16Z

💔 Test failed - checks-actions

rust-log-analyzer · 2022-09-18T06:04:22Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

jyn514 · 2022-09-18T14:21:36Z

curl: (56) OpenSSL SSL_read: Connection reset by peer, errno 54

@bors retry

bors · 2022-09-19T02:13:57Z

⌛ Testing commit 235dcce with merge c8e12cc...

bors · 2022-09-19T04:56:08Z

☀️ Test successful - checks-actions
Approved by: jyn514
Pushing c8e12cc to master...

rust-timer · 2022-09-19T06:14:45Z

Finished benchmarking commit (c8e12cc): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean¹	range	count²
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.6%	[1.9%, 3.4%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.7%	[-2.8%, -2.5%]	2
All ❌✅ (primary)	-	-	0

Cycles

This benchmark run did not return any relevant results for this metric.

the arithmetic mean of the percent change ↩
number of relevant changes ↩

rustbot added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Sep 14, 2022

rust-highfive assigned jyn514 Sep 14, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 14, 2022

This comment has been minimized.

Sign in to view

rustbot added the A-rustdoc-json Area: Rustdoc JSON backend label Sep 14, 2022

jyn514 reviewed Sep 14, 2022

View reviewed changes

src/bootstrap/dist.rs Outdated Show resolved Hide resolved

src/bootstrap/doc.rs Outdated Show resolved Hide resolved

config.toml.example Outdated Show resolved Hide resolved

src/bootstrap/config.rs Outdated Show resolved Hide resolved

src/bootstrap/doc.rs Outdated Show resolved Hide resolved

jyn514 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 14, 2022

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 14, 2022

jyn514 reviewed Sep 14, 2022

View reviewed changes

src/bootstrap/dist.rs Show resolved Hide resolved

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 15, 2022

This comment has been minimized.

Sign in to view

jyn514 approved these changes Sep 15, 2022

View reviewed changes

jyn514 reviewed Sep 15, 2022

View reviewed changes

src/bootstrap/doc.rs Outdated Show resolved Hide resolved

src/bootstrap/doc.rs Outdated Show resolved Hide resolved

src/bootstrap/doc.rs Show resolved Hide resolved

jyn514 reviewed Sep 15, 2022

View reviewed changes

src/bootstrap/doc.rs Outdated Show resolved Hide resolved

LukeMathWalker force-pushed the distribute-json-doc branch from 37d0444 to ccd4383 Compare September 16, 2022 09:12

bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 16, 2022

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 17, 2022

Add a new component, rust-json-docs, to distribute the JSON-formatt…

235dcce

…ed documentation for std crates in nightly toolchains. We also add a new flag to `x doc`, `--json`, to render the JSON-formatted version alongside the HTML-formatted one.

LukeMathWalker force-pushed the distribute-json-doc branch from e244549 to 235dcce Compare September 17, 2022 19:58

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 17, 2022

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 18, 2022

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 18, 2022

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 19, 2022

bors merged commit c8e12cc into rust-lang:master Sep 19, 2022

rustbot added this to the 1.66.0 milestone Sep 19, 2022

bors mentioned this pull request Sep 19, 2022

Add version switcher script for std docs #101855

Closed

ehuss mentioned this pull request Sep 20, 2022

Distribute rust-docs-json via rustup. #102042

Merged

jyn514 mentioned this pull request Oct 31, 2022

x doc std unconditionally recompiles the standard library each time #103785

Closed

LukeMathWalker deleted the distribute-json-doc branch August 3, 2024 16:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Distribute json doc #101799

Distribute json doc #101799

LukeMathWalker commented Sep 14, 2022

rust-highfive commented Sep 14, 2022

This comment has been minimized.

GuillaumeGomez commented Sep 14, 2022

aDotInTheVoid commented Sep 14, 2022

jyn514 left a comment

jyn514 commented Sep 14, 2022

This comment has been minimized.

LukeMathWalker commented Sep 15, 2022

This comment has been minimized.

jyn514 left a comment

jyn514 Sep 15, 2022

LukeMathWalker Sep 15, 2022 •

edited

Loading

jyn514 commented Sep 15, 2022

LukeMathWalker commented Sep 16, 2022

LukeMathWalker commented Sep 16, 2022

jyn514 commented Sep 16, 2022

bors commented Sep 16, 2022

LukeMathWalker commented Sep 17, 2022

jyn514 commented Sep 17, 2022

LukeMathWalker commented Sep 17, 2022

jyn514 commented Sep 17, 2022

bors commented Sep 17, 2022

bors commented Sep 18, 2022

bors commented Sep 18, 2022

rust-log-analyzer commented Sep 18, 2022

jyn514 commented Sep 18, 2022

bors commented Sep 19, 2022

bors commented Sep 19, 2022

rust-timer commented Sep 19, 2022

		if requested_crates.iter().any(\|p\| p == krate) {
		// No need to document more of the libraries if we have the one we want.

Distribute json doc #101799

Distribute json doc #101799

Conversation

LukeMathWalker commented Sep 14, 2022

Overview

Open questions / Doubts

rust-highfive commented Sep 14, 2022

This comment has been minimized.

GuillaumeGomez commented Sep 14, 2022

aDotInTheVoid commented Sep 14, 2022

jyn514 left a comment

Choose a reason for hiding this comment

jyn514 commented Sep 14, 2022

This comment has been minimized.

LukeMathWalker commented Sep 15, 2022

This comment has been minimized.

jyn514 left a comment

Choose a reason for hiding this comment

jyn514 Sep 15, 2022

Choose a reason for hiding this comment

LukeMathWalker Sep 15, 2022 • edited Loading

Choose a reason for hiding this comment

jyn514 commented Sep 15, 2022

LukeMathWalker commented Sep 16, 2022

LukeMathWalker commented Sep 16, 2022

jyn514 commented Sep 16, 2022

bors commented Sep 16, 2022

LukeMathWalker commented Sep 17, 2022

jyn514 commented Sep 17, 2022

LukeMathWalker commented Sep 17, 2022

jyn514 commented Sep 17, 2022

bors commented Sep 17, 2022

bors commented Sep 18, 2022

bors commented Sep 18, 2022

rust-log-analyzer commented Sep 18, 2022

jyn514 commented Sep 18, 2022

bors commented Sep 19, 2022

bors commented Sep 19, 2022

rust-timer commented Sep 19, 2022

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Footnotes

LukeMathWalker Sep 15, 2022 •

edited

Loading