Save crate filtering on rustdoc #62941

GuillaumeGomez · 2019-07-24T15:59:28Z

I added a hashmap and a hash encoding for the current crate list in case you have multiple crates handling on a same website (who talked about docs.rs?!). Like that, for each context, you have the filter crate selected.

r? @QuietMisdreavus

GuillaumeGomez · 2019-07-27T19:36:52Z

@bors: try

bors · 2019-07-27T19:37:04Z

⌛ Trying commit ef31242846ccd0b20945216a5982d5ea0d5d087e with merge eef66491f53a32b2c37f3188a8b4cf6f481488db...

bors · 2019-07-27T22:17:37Z

☀️ Try build successful - checks-azure
Build commit: eef66491f53a32b2c37f3188a8b4cf6f481488db

GuillaumeGomez · 2019-07-27T23:14:22Z

@highfive: run-doc-ui eef66491f53a32b2c37f3188a8b4cf6f481488db

rust-highfive · 2019-07-27T23:14:38Z

Rustdoc-UI starting test...

bors · 2019-07-28T01:13:54Z

☀️ Try build successful - checks-azure
Build commit: eef66491f53a32b2c37f3188a8b4cf6f481488db

bors · 2019-07-28T01:14:33Z

☀️ Try build successful - checks-azure
Build commit: eef66491f53a32b2c37f3188a8b4cf6f481488db

GuillaumeGomez · 2019-07-28T10:47:40Z

@highfive: run-doc-ui eef66491f53a32b2c37f3188a8b4cf6f481488db

rust-highfive · 2019-07-28T10:47:41Z

Rustdoc-UI starting test...

GuillaumeGomez · 2019-07-28T11:38:17Z

@highfive: run-doc-ui eef66491f53a32b2c37f3188a8b4cf6f481488db

rust-highfive · 2019-07-28T11:38:18Z

Rustdoc-UI starting test...

GuillaumeGomez · 2019-07-28T11:40:51Z

@highfive: run-doc-ui eef66491f53a32b2c37f3188a8b4cf6f481488db

rust-highfive · 2019-07-28T11:40:52Z

Rustdoc-UI starting test...

GuillaumeGomez · 2019-07-28T12:08:56Z

@highfive: run-doc-ui eef66491f53a32b2c37f3188a8b4cf6f481488db

rust-highfive · 2019-07-28T12:08:58Z

Rustdoc-UI starting test...

rust-highfive · 2019-07-28T12:09:11Z

Rustdoc-UI tests failed "successfully"!

Click to expand the log.

=> Starting doc-ui tests...

basic-code... FAILED (images "basic-code-eef66491f53a32b2c37f3188a8b4cf6f481488db.png" and "basic-code.png" are different)
basic... FAILED (images "basic-eef66491f53a32b2c37f3188a8b4cf6f481488db.png" and "basic.png" are different)
code-sidebar-toggle-mobile... FAILED (images "code-sidebar-toggle-mobile-eef66491f53a32b2c37f3188a8b4cf6f481488db.png" and "code-sidebar-toggle-mobile.png" are different)
code-sidebar-toggle... FAILED (images "code-sidebar-toggle-eef66491f53a32b2c37f3188a8b4cf6f481488db.png" and "code-sidebar-toggle.png" are different)
list_code_block... FAILED (images "list_code_block-eef66491f53a32b2c37f3188a8b4cf6f481488db.png" and "list_code_block.png" are different)
module-mobile... FAILED (images "module-mobile-eef66491f53a32b2c37f3188a8b4cf6f481488db.png" and "module-mobile.png" are different)
theme-change... FAILED (images "theme-change-eef66491f53a32b2c37f3188a8b4cf6f481488db.png" and "theme-change.png" are different)
toggle-docs... FAILED (images "toggle-docs-eef66491f53a32b2c37f3188a8b4cf6f481488db.png" and "toggle-docs.png" are different)

<= doc-ui tests done: 0 succeeded, 0 ignored, 8 failed

GuillaumeGomez · 2019-07-28T12:19:42Z

@highfive: run-doc-ui eef66491f53a32b2c37f3188a8b4cf6f481488db

Mark-Simulacrum

If I'm following this patch correctly, it'll ignore the stored preference if the crate list has changed since the last time the page was loaded. That sounds not entirely helpful, at least in most scenarios, where you might have N crates (including dependencies) and one of your dependencies changing shouldn't really lead to the selector here getting reset.

With that all in mind, maybe this PR can be simplified quite a bit by just storing the single name of the crate most recently selected in local storage?

Mark-Simulacrum · 2019-08-09T19:31:50Z

src/librustdoc/html/static/main.js

+    }
+
+    function getSavedCrates(savedCrates) {
+        if (typeof savedCrates === "undefined") {


Why is this checking for undefined here? Why can't call sites just pass in getCurrentValue(...) as needed?

It looks like there's two call sites to this function: one passes getCurrentValue after a !== null check, the other calls this function with no arguments. The discrepancy there is a bit odd.

This is a good point, this check is completely useless. Good catch!

Mark-Simulacrum · 2019-08-09T19:36:17Z

src/librustdoc/html/static/main.js

+            var elems = getSavedCrates(savedCrates);
+            if (elems.hasOwnProperty(getCratesHash())) {
+                onEach(window.crates, function(e) {
+                    if (e === elems[window.hashCode]) {


Can we avoid storing the crates hash in a global variable and simply have it in a variable here? It looks like this function should presumably be called only once so presumably that's equivalent performance wise?

Mark-Simulacrum · 2019-08-09T19:38:05Z

src/librustdoc/html/static/main.js

+                                    });
+                window.crates.sort();
+            }
+            window.hashCode = window.crates.join(",").hashCode();


The whole list of crates isn't that long in most cases, and we store it in memory here anyway, so why do we need to compute it's hash? Can't the crate list simply be stored as a string?

For the storage, not just as is.

Mark-Simulacrum · 2019-08-09T19:41:17Z

src/librustdoc/html/static/main.js

+        if (window.hashCode === undefined || window.hashCode === 0) {
+            if (window.crates === undefined || window.crates.length === 0) {
+                window.crates = cloneArray(document.getElementById("crate-search")
+                                                   .getElementsByTagName("option"), 1)


AFAICT, we wouldn't actually break anything if we didn't skip the first element in the options list here (it's always the same, sure, but that doesn't hurt). Can we do that to simplify the code here a little? It should also allow dropping the skip arg from cloneArray which is pretty irrelevant to the main purpose of the function.

It adds a useless element into the hash computation and slows it down. If we remove the hash system, then this skip will become useless indeed but for the moment it's important to keep it. :)

The cost of hashing this string is likely to be near-zero compared to all the other JS and such that we're running. I would rather pay that cost and have simpler to read code; as-is we at the very least should have a comment to the effect of strips out default 'all crates' at the beginning.

Mark-Simulacrum · 2019-08-09T19:42:57Z

src/librustdoc/html/static/main.js

            if (selectCrate) {
                selectCrate.onchange = function() {
+                    var savedCrates = getSavedCrates();
+                    const value = selectCrate.value;
+                    savedCrates[getCratesHash()] = value === "All crates" ? undefined : value;


As above, it seems reasonable that we can store "All crates" in local storage as well since it'll simplify the code here a little and ultimately doesn't affect anything presumably?

It's just to avoid keeping unused values. Otherwise, every time you go visit a crate, it'll be added to the local storage, which doesn't sound that nice...

It affect local storage space (which is surprisingly limited). Like this, we don't store the defaults.

Does this work if there's only one crate? It seems like we presumably just don't run this code then? Can we get comments about:

the 'no select crate' dialog case

the reasoning for why we want to not store anything for the all crates case

GuillaumeGomez · 2019-08-09T20:15:50Z

With that all in mind, maybe this PR can be simplified quite a bit by just storing the single name of the crate most recently selected in local storage?

If I do this, it'll get applied to all documentations (on docs.rs for example) and then you'll get some conflicts. Which is why I used this system. If you update dependencies, it'll be reset but it sounded better than having conflicts between crates.

Mark-Simulacrum · 2019-08-09T20:18:28Z

Hm, sure, but that seems like an orthogonal problem. On docs.rs it doesn't matter anyway I think because all crates are built on their own, there's no crate list anyway.

GuillaumeGomez · 2019-08-09T20:24:20Z

Yes but then you'll save this only for one crate. That doesn't seem very useful if done this way... If you provide a website which provides documentation for multiple crates and their dependencies, you'll need it (like doc.rust-lang.org for example which provides std and rustc-internals).

Mark-Simulacrum · 2019-08-09T21:24:29Z

I think it being reset is fine for now, since it simplifies the code quite a bit. The only notable site for which it'll matter I think is doc.rust-lang.org, and even there it's only if you're both a) using the selection dialog and b) navigate to both std docs and rustc internals docs. I'd guess the combination there is pretty rare, so solving that use case in this PR seems unnecessary.

If we were to solve that use case, I would prefer to solve it (not in this PR!) by having rustdoc itself, not the JS, embed some sort of identifier into pages of "build unit" style that we could then use. There's a lot of complexity here (hash function, etc.) that seems like it could be avoided for relatively low impact on most users, and I think we should do that.

GuillaumeGomez · 2019-08-09T22:40:29Z

I think you're right. If I just store current selected crate name, is it good for you? (the PR will become much smaller indeed!)

Mark-Simulacrum · 2019-08-09T23:15:42Z

Yes, I think this should store the selected crate name, and if it is not found, fall back to all crates (but, importantly, not remove the stored preference). I think that gives us the behavior most users want (at least on one "build") and it doesn't erase preferences unnecessarily.

GuillaumeGomez · 2019-08-10T09:40:12Z

Ok, I'll rewrite the PR then. Thanks a lot for your review @Mark-Simulacrum!

Mark-Simulacrum · 2019-08-10T11:53:25Z

src/librustdoc/html/render.rs

@@ -1157,7 +1157,7 @@ themePicker.onblur = handleThemeButtonsBlur;
            &format!("{}\n{}", variables.join(""), all_indexes.join("\n")),
            options.enable_minification),
            &dst);
-        try_err!(write!(&mut v, "initSearch(searchIndex);addSearchOptions(searchIndex);"), &dst);
+        try_err!(write!(&mut v, "addSearchOptions(searchIndex);initSearch(searchIndex);"), &dst);


Oh, I forgot to note during initial review -- can you explain why this has changed?

A global variable story. Not sure if it'll be needed anymore once I apply the new changes you suggested.

This should now be not needed since I believe we've eliminated global variables entirely.

joelpalmer · 2019-08-19T12:29:08Z

Ping from Triage: Hi @GuillaumeGomez - any update on this?

GuillaumeGomez · 2019-08-19T20:34:33Z

Updated.

totsteps · 2019-08-27T22:00:30Z

Ping from triage, @Mark-Simulacrum it seems GuillaumeGomez have made required changes. Can you please review this PR once again.

Thanks.

Mark-Simulacrum · 2019-08-27T22:17:09Z

src/librustdoc/html/static/main.js

@@ -445,6 +445,18 @@ if (!DOMTokenList.prototype.remove) {
        var OUTPUT_DATA = 1;
        var params = getQueryStringParams();

+        // Loading the crate filter if any.


Suggested change

// Loading the crate filter if any.

// Set the crate filter from saved storage, if the current page has the saved crate filter.

// If not, ignore the crate filter -- we want to support filtering for crates on sites like doc.rust-lang.org

// where the crates may differ from page to page while on the same domain.

Mark-Simulacrum · 2019-08-27T22:17:49Z

src/librustdoc/html/static/storage.js

@@ -57,7 +57,7 @@ function onEachLazy(lazyArray, func, reversed) {

 function usableLocalStorage() {
    // Check if the browser supports localStorage at all:
-    if (typeof(Storage) === "undefined") {
+    if (typeof Storage === "undefined") {


Unrelated change, I hope?

Ah, syntax fix. Didn't even think about it...

Mark-Simulacrum · 2019-08-27T22:18:20Z

src/librustdoc/html/render.rs

@@ -1157,7 +1157,7 @@ themePicker.onblur = handleThemeButtonsBlur;
            &format!("{}\n{}", variables.join(""), all_indexes.join("\n")),
            options.enable_minification),
            &dst);
-        try_err!(write!(&mut v, "initSearch(searchIndex);addSearchOptions(searchIndex);"), &dst);
+        try_err!(write!(&mut v, "addSearchOptions(searchIndex);initSearch(searchIndex);"), &dst);


This should now be not needed since I believe we've eliminated global variables entirely.

GuillaumeGomez · 2019-08-28T09:56:36Z

Updated.

Mark-Simulacrum · 2019-08-28T19:23:10Z

@bors r+

Okay, this looks good to me. Thanks of bearing with me on reviews!

bors · 2019-08-28T19:23:12Z

📌 Commit 06228d3 has been approved by Mark-Simulacrum

GuillaumeGomez · 2019-08-28T21:00:42Z

Thanks to you for handling the reviews!

bors · 2019-08-28T21:42:59Z

⌛ Testing commit 06228d3 with merge 0414dfa...

@QuietMisdreavus

…acrum Save crate filtering on rustdoc Fixes #62929. I added a hashmap and a hash encoding for the current crate list in case you have multiple crates handling on a same website (who talked about docs.rs?!). Like that, for each context, you have the filter crate selected. r? @QuietMisdreavus

bors · 2019-08-29T01:22:03Z

☀️ Test successful - checks-azure
Approved by: Mark-Simulacrum
Pushing 0414dfa to master...

rust-highfive assigned QuietMisdreavus Jul 24, 2019

GuillaumeGomez force-pushed the save-crate-filter branch from ad00098 to ef31242 Compare July 24, 2019 16:01

jonas-schievink added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Jul 26, 2019

rust-lang deleted a comment from rust-highfive Jul 27, 2019

Mark-Simulacrum reviewed Aug 9, 2019

View reviewed changes

Mark-Simulacrum reviewed Aug 10, 2019

View reviewed changes

Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 10, 2019

GuillaumeGomez force-pushed the save-crate-filter branch from ef31242 to ff5d1ba Compare August 19, 2019 18:53

Mark-Simulacrum reviewed Aug 27, 2019

View reviewed changes

Save crate filtering on rustdoc

06228d3

GuillaumeGomez force-pushed the save-crate-filter branch from ff5d1ba to 06228d3 Compare August 28, 2019 09:56

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 28, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 29, 2019

bors merged commit 06228d3 into rust-lang:master Aug 29, 2019

GuillaumeGomez deleted the save-crate-filter branch August 29, 2019 12:53

-        // Loading the crate filter if any.
+        // Set the crate filter from saved storage, if the current page has the saved crate filter.
+        // If not, ignore the crate filter -- we want to support filtering for crates on sites like doc.rust-lang.org
+        // where the crates may differ from page to page while on the same domain.

Save crate filtering on rustdoc #62941

Save crate filtering on rustdoc #62941

Conversation

GuillaumeGomez commented Jul 24, 2019

GuillaumeGomez commented Jul 27, 2019

bors commented Jul 27, 2019

bors commented Jul 27, 2019

GuillaumeGomez commented Jul 27, 2019

rust-highfive commented Jul 27, 2019

bors commented Jul 28, 2019

bors commented Jul 28, 2019

GuillaumeGomez commented Jul 28, 2019

rust-highfive commented Jul 28, 2019

GuillaumeGomez commented Jul 28, 2019

rust-highfive commented Jul 28, 2019

GuillaumeGomez commented Jul 28, 2019

rust-highfive commented Jul 28, 2019

GuillaumeGomez commented Jul 28, 2019

rust-highfive commented Jul 28, 2019

rust-highfive commented Jul 28, 2019

GuillaumeGomez commented Jul 28, 2019

Mark-Simulacrum left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mark-Simulacrum Aug 9, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GuillaumeGomez commented Aug 9, 2019

Mark-Simulacrum commented Aug 9, 2019

GuillaumeGomez commented Aug 9, 2019

Mark-Simulacrum commented Aug 9, 2019

GuillaumeGomez commented Aug 9, 2019

Mark-Simulacrum commented Aug 9, 2019

GuillaumeGomez commented Aug 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joelpalmer commented Aug 19, 2019

GuillaumeGomez commented Aug 19, 2019

totsteps commented Aug 27, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GuillaumeGomez commented Aug 28, 2019

Mark-Simulacrum commented Aug 28, 2019

bors commented Aug 28, 2019

GuillaumeGomez commented Aug 28, 2019

bors commented Aug 28, 2019

bors commented Aug 29, 2019

Mark-Simulacrum Aug 9, 2019 •

edited

Loading