Add index page argument #54543

GuillaumeGomez · 2018-09-24T23:10:53Z

@Mark-Simulacrum: I might need some help from you: in bootstrap, I want to add an argument (a new flag added into rustdoc) in order to generate the current index directly when rustdoc is documenting the std lib. However, my change in bootstrap didn't do it and I assume it must be moved inside the Std struct. But there, I don't see how to pass it to rustdoc through cargo. Did I miss anything?

r? @QuietMisdreavus

rust-highfive · 2018-09-24T23:58:13Z

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:44:23] travis_fold:end:stage2-rustdoc

[00:44:23] travis_time:end:stage2-rustdoc:start=1537833488550055619,finish=1537833488826169518,duration=276113899

[00:44:23] error: option `--index-page` argument must be a file
[00:44:23] 
[00:44:23] 
[00:44:23] 
[00:44:23] command did not execute successfully: "/checkout/obj/build/bootstrap/debug/rustdoc" "--html-after-content" "/checkout/src/doc/footer.inc" "--html-before-content" "/checkout/obj/build/x86_64-unknown-linux-gnu/doc/version_info.html" "--html-in-header" "/checkout/src/doc/favicon.inc" "--index-page" "src/doc/index.md" "--markdown-playground-url" "https://play.rust-lang.org/" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/doc" "/checkout/src/doc/rust.md" "--markdown-css" "rust.css"
[00:44:23] 
[00:44:23] 
[00:44:23] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap doc
[00:44:23] Build completed unsuccessfully in 0:04:09
[00:44:23] Build completed unsuccessfully in 0:04:09
[00:44:23] make: *** [all] Error 1
[00:44:23] Makefile:28: recipe for target 'all' failed

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0c2763fc
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
149112 ./src/llvm-emscripten/test
148852 ./obj/build/x86_64-unknown-linux-gnu/stage1-rustc
148580 ./obj/build/bootstrap/debug/incremental
134140 ./obj/build/bootstrap/debug/incremental/bootstrap-2zc4gzhr0d54q
134136 ./obj/build/bootstrap/debug/incremental/bootstrap-2zc4gzhr0d54q/s-f545tnz7cs-y11ee0-3093t8bfyy2gb

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

GuillaumeGomez · 2018-09-25T21:22:19Z

@Mark-Simulacrum Need help when you have a bit of time. I tried to pass the index-page flag to cargo but I had to replace doc with rustdoc, however it doesn't work at all like this. Did I miss anything obvious?

Mark-Simulacrum · 2018-09-25T21:51:55Z

Yes, I plan to take a look at this tomorrow I hope, probably won't get a chance to tonight.

rust-highfive · 2018-09-25T22:25:35Z

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:48:53]     Checking compiler_builtins v0.0.0 (file:///checkout/src/rustc/compiler_builtins_shim)
[00:48:55]  Documenting alloc v0.0.0 (file:///checkout/src/liballoc)
[00:48:55] thread 'main' panicked at '
[00:48:55] 
[00:48:55] failed to run "/path/to/nowhere/rustdoc/not/required" "--crate-name" "alloc" "liballoc/lib.rs" "--target" "x86_64-unknown-linux-gnu" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/doc" "index-page" "/checkout/src/doc/index.md" "-L" "dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps" "-L" "dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/release/deps" "--extern" "compiler_builtins=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libcompiler_builtins-66b9d646ca55a4f6.rmeta" "--extern" "core=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libcore-dfbcb38dfdb6370a.rmeta" "--cfg" "stage1" "--cfg" "dox" "--sysroot" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1" "-Z" "force-unstable-if-unmarked" "-Z" "unstable-options" "--crate-version" "1.30.0-dev": No such file or directory (os error 2)
[00:48:55] ', bootstrap/bin/rustdoc.rs:79:19
[00:48:55] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:48:55] error: Could not document `alloc`.
[00:48:55] 
[00:48:55] 
[00:48:55] Caused by:
[00:48:55]   process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustdoc --crate-name alloc liballoc/lib.rs --target x86_64-unknown-linux-gnu -o /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/doc index-page /checkout/src/doc/index.md -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/release/deps --extern compiler_builtins=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libcompiler_builtins-66b9d646ca55a4f6.rmeta --extern core=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libcore-dfbcb38dfdb6370a.rmeta` (exit code: 101)
[00:48:55] 
[00:48:55] 
[00:48:55] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "rustdoc" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" "panic-unwind jemalloc backtrace" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "-Z" "unstable-options" "-p" "alloc" "--" "index-page" "/checkout/src/doc/index.md"
[00:48:55] 
[00:48:55] 
[00:48:55] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap doc
[00:48:55] Build completed unsuccessfully in 0:05:22
[00:48:55] Build completed unsuccessfully in 0:05:22
[00:48:55] Makefile:28: recipe for target 'all' failed
[00:48:55] make: *** [all] Error 1

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0df531d4
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_time:end:020afff3:start=1537914330481632417,finish=1537914330487494212,duration=5861795
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0f542d87
$ ln -s . checkout && for CORE in obj/cores/

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

src/bootstrap/doc.rs

GuillaumeGomez · 2018-09-29T14:43:29Z

It's now ready!

QuietMisdreavus

Do you have a screenshot of what the index page looks like if you don't provide the --index-page argument?

Can you update the Rustdoc Book with the new CLI flags you added?

src/librustdoc/html/render.rs

GuillaumeGomez · 2018-10-01T20:14:04Z

Yes and yes.

GuillaumeGomez · 2018-10-06T14:10:57Z

@QuietMisdreavus What remains to be done here btw? The books are generated with the index page directly (since it's not *really* running rustdoc, seems weird to use this new flag for them).

QuietMisdreavus · 2018-10-10T15:28:52Z

Do you have a screenshot of what the index page looks like if you don't provide the --index-page argument?

Can you update the Rustdoc Book with the new CLI flags you added?

I don't see an update to either of these?

EDIT: When i say "update the Rustdoc Book", i mean "write new docs for the book talking about the new flags".

GuillaumeGomez · 2018-10-10T21:34:41Z

EDIT: When i say "update the Rustdoc Book", i mean "write new docs for the book talking about the new flags".

Now I get it!

What remains to be done (reminder for me):

Add entry about this feature in the rustdoc book
Add a screenshot of the default index page

GuillaumeGomez · 2018-10-13T12:32:57Z

So here is the default index page generated (yes I generated a lot of crates locally :-.):

GuillaumeGomez · 2018-10-13T12:46:13Z

Updated.

GuillaumeGomez · 2018-10-16T23:10:09Z

@QuietMisdreavus Anything else?

QuietMisdreavus

Why is the CLI flag enable-index-page, but the Context field disable_index_page? It creates confusion when looking through the code if the CLI flag is the opposite of how the code looks at it.

(GitHub just released the ability for us to suggest code changes as a potential patch, so i'm going to test it out here, haha!)

QuietMisdreavus · 2018-10-17T21:12:41Z

src/librustdoc/html/render.rs

@@ -106,6 +109,8 @@ struct Context {
    /// The map used to ensure all generated 'id=' attributes are unique.
    id_map: Rc<RefCell<IdMap>>,
    pub shared: Arc<SharedContext>,
+    pub disable_index_page: bool,


Suggested change

pub disable_index_page: bool,

pub enable_index_page: bool,

QuietMisdreavus · 2018-10-17T21:12:55Z

src/librustdoc/html/render.rs

@@ -501,7 +506,12 @@ pub fn run(mut krate: clean::Crate,
           sort_modules_alphabetically: bool,
           themes: Vec<PathBuf>,
           enable_minification: bool,
-           id_map: IdMap) -> Result<(), Error> {
+           id_map: IdMap,
+           disable_index_page: bool,


Suggested change

disable_index_page: bool,

enable_index_page: bool,

QuietMisdreavus · 2018-10-17T21:13:43Z

src/librustdoc/html/render.rs

@@ -572,6 +582,8 @@ pub fn run(mut krate: clean::Crate,
        codes: ErrorCodes::from(UnstableFeatures::from_environment().is_nightly_build()),
        id_map: Rc::new(RefCell::new(id_map)),
        shared: Arc::new(scx),
+        disable_index_page,


Suggested change

disable_index_page,

enable_index_page,

QuietMisdreavus · 2018-10-17T21:14:13Z

src/librustdoc/html/render.rs

@@ -969,6 +990,42 @@ themePicker.onblur = handleThemeButtonsBlur;
    }
    try_err!(writeln!(&mut w, "initSearch(searchIndex);"), &dst);

+    if cx.disable_index_page == false {


Suggested change

if cx.disable_index_page == false {

if cx.enable_index_page {

QuietMisdreavus · 2018-10-17T21:14:49Z

src/librustdoc/lib.rs

@@ -580,7 +599,10 @@ fn main_args(args: &[String]) -> isize {
                                  renderinfo,
                                  sort_modules_alphabetically,
                                  themes,
-                                  enable_minification, id_map)
+                                  enable_minification, id_map,
+                                  !enable_index_page, index_page,


Suggested change

!enable_index_page, index_page,

enable_index_page, index_page,

QuietMisdreavus · 2018-10-17T22:36:48Z

Oh one more thing: Can you add a test? It should just work with the regular rustdoc tests if you add an auxiliary crate to the build.

GuillaumeGomez · 2018-10-20T23:22:42Z

Updated.

QuietMisdreavus · 2018-10-24T22:05:21Z

src/doc/rustdoc/src/unstable-features.md

@@ -389,3 +389,16 @@ pub struct BigX;

 Then, when looking for it through the `rustdoc` search, if you enter "x" or
 "big", search will show the `BigX` struct first.
+
+### `index-page` feature


Since these are CLI flags, they should follow the convention of the other flags' headings:

### `--index-page`: provide a top-level landing page for docs ### `--enable-index-page`: generate a default index page for docs

(The section about doc_alias is also in the wrong place, but i failed to catch that when you wrote it in the first place... >_>)

Ah. I'll fix all that at once then.

GuillaumeGomez · 2018-11-01T23:05:38Z

Updated.

QuietMisdreavus

Thanks so much!

QuietMisdreavus · 2018-11-02T14:34:58Z

@bors r+

bors · 2018-11-02T14:35:00Z

📌 Commit 2b64605 has been approved by QuietMisdreavus

bors · 2018-11-02T15:39:32Z

⌛ Testing commit 2b64605 with merge e53a5ff...

@Mark-Simulacrum

Add index page argument @Mark-Simulacrum: I might need some help from you: in bootstrap, I want to add an argument (a new flag added into `rustdoc`) in order to generate the current index directly when `rustdoc` is documenting the `std` lib. However, my change in `bootstrap` didn't do it and I assume it must be moved inside the `Std` struct. But there, I don't see how to pass it to `rustdoc` through `cargo`. Did I miss anything? r? @QuietMisdreavus

bors · 2018-11-02T18:28:44Z

☀️ Test successful - status-appveyor, status-travis
Approved by: QuietMisdreavus
Pushing e53a5ff to master...

…acrum Use local links in the alloc docs. Links to other crates (like core) from the alloc crate were incorrectly using the `https://doc.rust-lang.org/nightly/` absolute (remote) links, instead of relative (local) links. For example, the link to `Result` at https://doc.rust-lang.org/1.44.1/alloc/vec/struct.Vec.html#method.try_reserve goes to /nightly/. This is because alloc was being documented before core, and rustdoc relies on the existence of the local directory to know if it should use a local or remote link. There was code that tried to compensate for this (`create_dir_all`), but in rust-lang#54543 it was broken because instead of running `cargo doc` once for all the crates, it was changed to run `cargo rustdoc` for each crate individually. This means that `create_dir_all` was no longer doing what it was supposed to be doing (creating all the directories before starting). The solution here is to just build in the correct order (from the dependency leaves towards the root). An alternate solution would be to switch back to running `cargo doc` once (and use RUSTDOCFLAGS for passing in flags). Another alternate solution would be to iterate over the list twice, creating the directories during the first pass. I also did a little cleanup to remove the "crate-docs" directory. This was added in the past because different crates were built in different directories. Over time, things have been unified (and rustc docs no longer include std), so it is no longer necessary.

rust-highfive assigned QuietMisdreavus Sep 24, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 24, 2018

GuillaumeGomez force-pushed the top-level-index branch from 741050c to e585b25 Compare September 25, 2018 21:21

Mark-Simulacrum reviewed Sep 26, 2018

View reviewed changes

src/bootstrap/doc.rs Show resolved Hide resolved

GuillaumeGomez changed the title ~~[WIP] Add index page argument~~ Add index page argument Sep 29, 2018

GuillaumeGomez force-pushed the top-level-index branch from f3d07a9 to f30b6ea Compare September 29, 2018 14:42

QuietMisdreavus reviewed Oct 1, 2018

View reviewed changes

src/librustdoc/html/render.rs Show resolved Hide resolved

GuillaumeGomez force-pushed the top-level-index branch from 28c6582 to a80d613 Compare October 13, 2018 12:45

QuietMisdreavus reviewed Oct 17, 2018

View reviewed changes

GuillaumeGomez added 3 commits October 21, 2018 01:15

Add index page

fb2813b

new version for bootstrap

f7c8258

Use markdown::render instead of using pulldown_cmark directly

a40b758

GuillaumeGomez force-pushed the top-level-index branch from a80d613 to 91bba19 Compare October 20, 2018 23:22

QuietMisdreavus reviewed Oct 24, 2018

View reviewed changes

GuillaumeGomez added 2 commits November 2, 2018 00:01

Add documentation for index-page features

1244a85

Add test for index-page

7f7d8fb

Move doc_alias doc

2b64605

GuillaumeGomez force-pushed the top-level-index branch from 91bba19 to 2b64605 Compare November 1, 2018 23:05

QuietMisdreavus approved these changes Nov 2, 2018

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 2, 2018

QuietMisdreavus mentioned this pull request Nov 2, 2018

Rustdoc could (have an option to) create a top-level index page #16103

Closed

bors merged commit 2b64605 into rust-lang:master Nov 2, 2018

bors mentioned this pull request Nov 2, 2018

rustdoc: refactor: centralize all command-line argument parsing #55515

Merged

GuillaumeGomez deleted the top-level-index branch November 2, 2018 22:25

ptersilie mentioned this pull request Nov 27, 2018

Build a book for each tag version (including master) softdevteam/grmtools#53

Merged

ehuss mentioned this pull request Jul 15, 2020

Use local links in the alloc docs. #74352

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add index page argument #54543

Add index page argument #54543

GuillaumeGomez commented Sep 24, 2018

rust-highfive commented Sep 24, 2018

GuillaumeGomez commented Sep 25, 2018

Mark-Simulacrum commented Sep 25, 2018

rust-highfive commented Sep 25, 2018

GuillaumeGomez commented Sep 29, 2018

QuietMisdreavus left a comment

GuillaumeGomez commented Oct 1, 2018

GuillaumeGomez commented Oct 6, 2018

QuietMisdreavus commented Oct 10, 2018 •

edited

Loading

GuillaumeGomez commented Oct 10, 2018

GuillaumeGomez commented Oct 13, 2018

GuillaumeGomez commented Oct 13, 2018

GuillaumeGomez commented Oct 16, 2018

QuietMisdreavus left a comment

QuietMisdreavus Oct 17, 2018

QuietMisdreavus Oct 17, 2018

QuietMisdreavus Oct 17, 2018

QuietMisdreavus Oct 17, 2018

QuietMisdreavus Oct 17, 2018

QuietMisdreavus commented Oct 17, 2018

GuillaumeGomez commented Oct 20, 2018

QuietMisdreavus Oct 24, 2018

GuillaumeGomez Oct 25, 2018

GuillaumeGomez commented Nov 1, 2018

QuietMisdreavus left a comment

QuietMisdreavus commented Nov 2, 2018

bors commented Nov 2, 2018

bors commented Nov 2, 2018

bors commented Nov 2, 2018

	if cx.disable_index_page == false {
	if cx.enable_index_page {

	!enable_index_page, index_page,
	enable_index_page, index_page,

Add index page argument #54543

Add index page argument #54543

Conversation

GuillaumeGomez commented Sep 24, 2018

rust-highfive commented Sep 24, 2018

GuillaumeGomez commented Sep 25, 2018

Mark-Simulacrum commented Sep 25, 2018

rust-highfive commented Sep 25, 2018

GuillaumeGomez commented Sep 29, 2018

QuietMisdreavus left a comment

Choose a reason for hiding this comment

GuillaumeGomez commented Oct 1, 2018

GuillaumeGomez commented Oct 6, 2018

QuietMisdreavus commented Oct 10, 2018 • edited Loading

GuillaumeGomez commented Oct 10, 2018

GuillaumeGomez commented Oct 13, 2018

GuillaumeGomez commented Oct 13, 2018

GuillaumeGomez commented Oct 16, 2018

QuietMisdreavus left a comment

Choose a reason for hiding this comment

QuietMisdreavus Oct 17, 2018

Choose a reason for hiding this comment

QuietMisdreavus Oct 17, 2018

Choose a reason for hiding this comment

QuietMisdreavus Oct 17, 2018

Choose a reason for hiding this comment

QuietMisdreavus Oct 17, 2018

Choose a reason for hiding this comment

QuietMisdreavus Oct 17, 2018

Choose a reason for hiding this comment

QuietMisdreavus commented Oct 17, 2018

GuillaumeGomez commented Oct 20, 2018

QuietMisdreavus Oct 24, 2018

Choose a reason for hiding this comment

GuillaumeGomez Oct 25, 2018

Choose a reason for hiding this comment

GuillaumeGomez commented Nov 1, 2018

QuietMisdreavus left a comment

Choose a reason for hiding this comment

QuietMisdreavus commented Nov 2, 2018

bors commented Nov 2, 2018

bors commented Nov 2, 2018

bors commented Nov 2, 2018

QuietMisdreavus commented Oct 10, 2018 •

edited

Loading