lldb-preview and rust-mingw are always missing

[RalfJung]

The lldb-preview and rust-mingw components have been reported as missing since... well, as long as I can remember. Is there still any point in keeping them in that table? It is quite alarming at first to see things reporting as missing, until one realized that for these two that is probably expected.

[Mark-Simulacrum]

rust-mingw is just not on the default x86_64-unknown-linux-gnu target -- x86_64-pc-windows-gnu has it pretty much always.

We stopped shipping lldb-preview entirely I think a while back (1 year ish?) -- I'm not sure why it's still being rendered at all.

[RalfJung]

Hm, so it doesn't know which tools are "expected" for which target?

[RalfJung]

lldb-preview is still handled in dist.rs here:

https://github.com/rust-lang/rust/blob/34d6b446e56d90ae34048a2e31eb1e39dbda2a1b/src/bootstrap/dist.rs#L2462

[Mark-Simulacrum]

Yeah, looks like that code exits here probably: https://github.com/rust-lang/rust/blob/34d6b446e56d90ae34048a2e31eb1e39dbda2a1b/src/bootstrap/dist.rs#L2380 since we don't actually put lldb in the bindir these days (since it's not built to my knowledge)

[RalfJung]

So we could likely just remove that entire dist step, and then it would also disappear here?

[Mark-Simulacrum]

Yes, and we probably should. I don't know what we should do with rust-mingw though.

[RalfJung]

rust-lang/rust#72058 landed 3 days ago and the lldb-preview row is still shown. Will it just take some more time or is some further action required to remove that row?

[Mark-Simulacrum]

I believe we currently have a window of 7 days into the past that we look -- presumably in around 5 days we'll no longer have that row.

[RalfJung]

11 days later, the lldb-preview row still exists.

[Mark-Simulacrum]

hm, looks like I misinterpreted some configuration or so -- we fetch back to April 30th in the last CI build. So I guess until that moves up to past May 14th (when the relevant PR landed) or perhaps May 15th this'll stick around.

Latest manifest is for 2020-05-28
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-27/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-26/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-25/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-24/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-23/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-22/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-21/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-20/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-19/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-18/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-17/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-16/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-15/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-13/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-12/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-11/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-10/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-09/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-08/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-07/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-06/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-05/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-04/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-03/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-02/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-05-01/channel-rust-nightly.toml
Fetching a manifest from https://static.rust-lang.org/dist/2020-04-30/channel-rust-nightly.toml

[RalfJung]

@Mark-Simulacrum looking at the manifest file, when rls is missing things look somewhat like this:

[pkg.rls-preview]
version = ""
[pkg.rls-preview.target.aarch64-unknown-linux-gnu]
available = false

[pkg.rls-preview.target.arm-unknown-linux-gnueabi]
available = false

[...]

However, for rust-mingw, they look like this:

[pkg.rust-mingw]
version = "1.47.0-nightly (7e6d6e5f5 2020-08-16)"
git_commit_hash = "7e6d6e5f535321c2223f044caba16f97b825009c"
[pkg.rust-mingw.target.i686-pc-windows-gnu]
available = true
url = "https://static.rust-lang.org/dist/2020-08-17/rust-mingw-nightly-i686-pc-windows-gnu.tar.gz"
hash = "0f34b7798d77cf506c8fad037fa547b337546bb29c2a6cdbed688039e6d60252"
xz_url = "https://static.rust-lang.org/dist/2020-08-17/rust-mingw-nightly-i686-pc-windows-gnu.tar.xz"
xz_hash = "026f203f57b5d7987f15fb338084ddead7945fd7274feb089ca5c05c0352eaa6"

[pkg.rust-mingw.target.x86_64-pc-windows-gnu]
available = true
url = "https://static.rust-lang.org/dist/2020-08-17/rust-mingw-nightly-x86_64-pc-windows-gnu.tar.gz"
hash = "c4976261e85324035f9bf6a0e1d372c540f9189b9b8bae49d52ec7d55ca5e15f"
xz_url = "https://static.rust-lang.org/dist/2020-08-17/rust-mingw-nightly-x86_64-pc-windows-gnu.tar.xz"
xz_hash = "ca6d18622ed7d369329bd765443812b299a344d22b0755e96417c683578452fa"

In particular, there is no available = false. So maybe that would be the clue that could be used to figure out if a package should be labelled as "missing" or as just not existing at all for this platform?

[Mark-Simulacrum]

Indeed! It looks like the code was previously adding all components regardless of target. I've adjusted it such that we skip adding rows where components are missing for that target for the last ~30 days (or however far back we're downloading).

[RalfJung]

Thanks!

What does "missing" mean here? When it was available = false for 30 days it seems fine to still report it as missing, this indicates a package that should exist but is unavailable.

But a package that's not even listed for a given target, IMO also shouldn't trigger a row in the table.

Your comment says

This does mean that newly added components will not be visible until they are
first available, but that seems fine.

which is confusing because I expected things would be shown immediately once they show up with available = false.

[Mark-Simulacrum]

The current code doesn't differentiate between available = false and just missing. I think it would be pretty annoying to make that change -- it looks like it'll be a pretty invasive change.

I think it's quite rare that we have a component that we do intend to ship missing for 30 days -- that's almost a full cycle -- so hopefully we won't notice that so much.

[RalfJung]

The current code doesn't differentiate between available = false and just missing.

(Maybe "absent" would be a better term than "missing" for when the package just does not exist in the manifest, not even as unavailable.)
Ah I see. Making that distinction is what I suggested, so I was a bit confused when you said "indeed" but then it didn't sound like you were actually agreeing with what I said. ;)

[Mark-Simulacrum]

I am in agreement -- I want things to look that way. It's just not the current implementation :)