Bump bootstrap compiler to 1.68 #107297

Mark-Simulacrum · 2023-01-25T15:15:09Z

This also changes our stage0.json to include the rustc component for the rustfmt pinned nightly toolchain, which is currently necessary due to rustfmt dynamically linking to that toolchain's librustc_driver and libstd.

r? @pietroalbini

rustbot · 2023-01-25T15:15:20Z

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

Stabilizing library features
Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
Changing public documentation in ways that create new stability guarantees
Changing observable runtime behavior of library APIs

Mark-Simulacrum · 2023-01-25T16:12:25Z

It looks like the rustfmt binary now dynamically links librustc_driver and libstd, which is causing breakage here. I'm not sure yet what introduced that bug, need to do a bisection to figure out.

library/core/src/mem/mod.rs

bjorn3 · 2023-01-25T21:01:07Z

Rustfmt now links librustc_driver.so because of my PR #105609 to stop shipping rlibs in rustc-dev to reduce it's size. This means the only copy of the compiled code of rustc libraries is in librustc_driver.so and as such everything using them needs to link librustc_driver.so. I guess it would be possible to keep rustfmt statically link it by copying the respective rlibs to the sysroot only when building rustfmt, but not packaging it in rustc-dev. That may break out-of-tree rustfmt development though.

jyn514 · 2023-01-25T21:04:02Z

Setting the LD_LIBRARY_PATH for rustfmt at runtime shouldn't be too hard, no? That seems better than special-casing rustfmt in bootstrap.

bjorn3 · 2023-01-25T21:07:39Z

For building LD_LIBRARY_PATH doesn't change a thing, we need the -L flag of rustc instead. -L would probably work though.

Mark-Simulacrum · 2023-01-25T21:14:07Z

Setting the LD_LIBRARY_PATH for rustfmt at runtime shouldn't be too hard, no? That seems better than special-casing rustfmt in bootstrap.

The problem is that at least today, we're not downloading the full nightly toolchain as part of bootstrap, just the rustfmt binary/package -- so there's nothing to point LD_LIBRARY_PATH to.

jyn514 · 2023-01-25T21:19:56Z

Ok, can we change that? It seems reasonable to download rustc-dev for rustfmt's toolchain too.

As a long term thing, it's never been clear to me why we don't just use beta rustfmt ... we can't use new language features until they're supported by the beta compiler anyway. I guess it matters for std, one more thing https://jyn.dev/2023/01/12/Bootstrapping-Rust-in-2023.html would help with.

cuviper · 2023-01-25T21:35:48Z

It seems reasonable to download rustc-dev for rustfmt's toolchain too.

It shouldn't need rustc-dev, but rather the rustc and rust-std components.

it's never been clear to me why we don't just use beta rustfmt

It would indeed be simpler if we got rustfmt matching the bootstrap compiler.

Mark-Simulacrum · 2023-01-25T23:18:11Z

Ok, can we change that? It seems reasonable to download rustc-dev for rustfmt's toolchain too.

That would ~double the amount you need to download to work on rustc, which feels quite annoying. It may be the easiest short term fix though.

As a long term thing, it's never been clear to me why we don't just use beta rustfmt

Rustfmt doesn't support RUSTC_BOOTSTRAP and we have unstable features via rustfmt.toml. The rustfmt team was reluctant to add support historically.

Mark-Simulacrum · 2023-01-25T23:19:24Z

I just had a thought: I will just skip bumping rustfmt, there's no hard need to do so.

Mark-Simulacrum · 2023-01-25T23:57:49Z

Or... not. I guess we're already using new syntax in std which the older rustfmt doesn't support.

Mark-Simulacrum · 2023-01-26T00:00:02Z

cc rust-lang/rustfmt#4884 for recent thread on RUSTC_BOOTSTRAP in rustfmt

Mark-Simulacrum · 2023-01-26T00:10:45Z

OK, updated with the logic to download rustc and rust-std components for the rustfmt toolchain (now placed in build/$triple/rustfmt).

library/core/src/cmp.rs

calebcartwright · 2023-01-27T03:31:32Z

fwiw the rustfmt item is impacting downstream rustfmt users as well rust-lang/rustfmt#5675, and it also already has some minor impact on rustfmt developers https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/default.20sysroot.20for.20external.20tools/near/323956187

src/tools/bump-stage0/src/main.rs

pietroalbini · 2023-01-30T19:54:35Z

r=me with bjorn's comment addressed.

rustbot · 2023-01-31T15:30:03Z

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

Mark-Simulacrum · 2023-01-31T18:21:13Z

@bors r=pietroalbini rollup=iffy

bors · 2023-01-31T18:21:15Z

📌 Commit 652f79e has been approved by pietroalbini

It is now in the queue for this repository.

bors · 2023-01-31T19:24:32Z

⌛ Testing commit 652f79e with merge dc1d9d5...

bors · 2023-01-31T22:13:45Z

☀️ Test successful - checks-actions
Approved by: pietroalbini
Pushing dc1d9d5 to master...

rust-timer · 2023-01-31T23:33:38Z

Finished benchmarking commit (dc1d9d5): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	3.3%	[3.1%, 3.6%]	2
Regressions ❌ (secondary)	3.1%	[2.8%, 3.3%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	3.3%	[3.1%, 3.6%]	2

Cycles

This benchmark run did not return any relevant results for this metric.

jyn514 · 2023-02-01T04:56:31Z

src/bootstrap/download.rs

-        let rustfmt_path = self.initial_rustc.with_file_name(exe("rustfmt", host));
-        let bin_root = self.out.join(host.triple).join("stage0");
+        let bin_root = self.out.join(host.triple).join("rustfmt");
+        let rustfmt_path = bin_root.join("bin").join(exe("rustfmt", host));


This broke the rust-analyzer config we suggest in the dev-guide: https://rustc-dev-guide.rust-lang.org/building/suggested.html#configuring-rust-analyzer-for-rustc

"rust-analyzer.rustfmt.overrideCommand": [ "./build/host/stage0/bin/rustfmt", "--edition=2021" ],

since now rustfmt is in build/host/rustfmt/bin/rustfmt instead of build/host/stage0/bin/rustfmt. What was the reason for this change? Is it possible to revert it?

I think it's because we can't install multiple rustc components to the same prefix path. Maybe we can symlink rustfmt in the old place, although that won't work for Windows. Better would still be to use the same rustfmt as the bootstrap compiler, if we can figure out the unstable features for that. (or stabilize what we need!)

See #107547 for more discussion.

As per rust-lang/rust#107297 (comment), the change broke the rust-analyzer config. Hence, changing the docs to match the new path

Summary: Update to rustfmt 1.6.0-nightly (839e9a6 2023-07-02) The big ticket item here is that let-else is now supported! In fact, the suppport just landed two days ago (see [rust-lang/rustfmt#4914]) The unfortunate thing here though is that rustfmt is not statically linked anymore (see discussion in [rust-lang/rust#107297]). So we need all of librustc_driver and libstd - which makes our use case pretty big (~4-5MB to 50MB-100MB). We should explore building from source and statically linking, or using rustfmt from the toolchain. Both things that I don't want to deal with right now. [rust-lang/rustfmt#4914]: rust-lang/rustfmt#4914 [rust-lang/rust#107297]: rust-lang/rust#107297 Note to future updaters: To find out more-or-less what libs you need, you can use `objdump -p bin/rustfmt | grep NEEDED` on Linux for ELF bins, and `otool -L bin/rustfmt` on macOS for Mach-O bins. No idea what you do for Windows. Ran `tools/arcanist/lint/codemods/rustfmt-fbsource` to format the repo. Reviewed By: shayne-fletcher Differential Revision: D47203254 fbshipit-source-id: 6ffd3ce66c7f2b006d09505b93fed515ebc76902

Mark-Simulacrum added 2 commits January 25, 2023 09:44

Set version placeholders to 1.68

3653254

Bump stage0.json to 1.68 beta

17887f7

rustbot assigned pietroalbini Jan 25, 2023

This comment has been minimized.

Sign in to view

Noratrieb reviewed Jan 25, 2023

View reviewed changes

library/core/src/mem/mod.rs Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

Mark-Simulacrum force-pushed the bump-bootstrap branch from d054d5e to 5f0e161 Compare January 26, 2023 00:10

rustbot added the T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. label Jan 26, 2023

cuviper reviewed Jan 26, 2023

View reviewed changes

library/core/src/cmp.rs Show resolved Hide resolved

jyn514 mentioned this pull request Jan 29, 2023

Make x.py clippy download and use beta clippy #106394

Closed

stage-step cfgs

5b08c9f

Mark-Simulacrum force-pushed the bump-bootstrap branch from 5f0e161 to ac7b9c2 Compare January 30, 2023 18:09

bjorn3 reviewed Jan 30, 2023

View reviewed changes

src/tools/bump-stage0/src/main.rs Outdated Show resolved Hide resolved

Mark-Simulacrum force-pushed the bump-bootstrap branch from ac7b9c2 to 9239808 Compare January 30, 2023 19:56

This comment has been minimized.

Sign in to view

Download rustc component for rustfmt toolchain as well

652f79e

Mark-Simulacrum force-pushed the bump-bootstrap branch from 9239808 to 652f79e Compare January 31, 2023 15:30

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 31, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 31, 2023

bors merged commit dc1d9d5 into rust-lang:master Jan 31, 2023

rustbot added this to the 1.69.0 milestone Jan 31, 2023

bors mentioned this pull request Jan 31, 2023

Stabilize cmpxchg16b_target_feature #106774

Merged

jyn514 reviewed Feb 1, 2023

View reviewed changes

jyn514 mentioned this pull request Feb 1, 2023

Suggested rust-analyzer config can't run rustfmt #107547

Closed

zephaniahong added a commit to zephaniahong/rustc-dev-guide that referenced this pull request Feb 1, 2023

Update rustfmt path

709ff11

As per rust-lang/rust#107297 (comment), the change broke the rust-analyzer config. Hence, changing the docs to match the new path

zephaniahong mentioned this pull request Feb 1, 2023

Update rustfmt path rust-lang/rustc-dev-guide#1574

Merged

Mark-Simulacrum deleted the bump-bootstrap branch February 1, 2023 13:04

jyn514 pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Feb 2, 2023

Update rustfmt path

c97f571

As per rust-lang/rust#107297 (comment), the change broke the rust-analyzer config. Hence, changing the docs to match the new path

est31 mentioned this pull request Feb 5, 2023

x fmt broken on NixOS since stage0 update #107676

Closed

est31 mentioned this pull request Mar 8, 2023

Support RUSTC_BOOTSTRAP rust-lang/rustfmt#4884

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump bootstrap compiler to 1.68 #107297

Bump bootstrap compiler to 1.68 #107297

Mark-Simulacrum commented Jan 25, 2023 •

edited

Loading

rustbot commented Jan 25, 2023

This comment has been minimized.

Mark-Simulacrum commented Jan 25, 2023

bjorn3 commented Jan 25, 2023 •

edited

Loading

jyn514 commented Jan 25, 2023

bjorn3 commented Jan 25, 2023

Mark-Simulacrum commented Jan 25, 2023

jyn514 commented Jan 25, 2023

cuviper commented Jan 25, 2023 •

edited

Loading

Mark-Simulacrum commented Jan 25, 2023

Mark-Simulacrum commented Jan 25, 2023

This comment was marked as resolved.

Mark-Simulacrum commented Jan 25, 2023

Mark-Simulacrum commented Jan 26, 2023

Mark-Simulacrum commented Jan 26, 2023

calebcartwright commented Jan 27, 2023 •

edited

Loading

pietroalbini commented Jan 30, 2023

This comment has been minimized.

rustbot commented Jan 31, 2023

Mark-Simulacrum commented Jan 31, 2023

bors commented Jan 31, 2023

bors commented Jan 31, 2023

bors commented Jan 31, 2023

rust-timer commented Jan 31, 2023

jyn514 Feb 1, 2023

cuviper Feb 1, 2023

jyn514 Feb 2, 2023

Bump bootstrap compiler to 1.68 #107297

Bump bootstrap compiler to 1.68 #107297

Conversation

Mark-Simulacrum commented Jan 25, 2023 • edited Loading

rustbot commented Jan 25, 2023

This comment has been minimized.

Mark-Simulacrum commented Jan 25, 2023

bjorn3 commented Jan 25, 2023 • edited Loading

jyn514 commented Jan 25, 2023

bjorn3 commented Jan 25, 2023

Mark-Simulacrum commented Jan 25, 2023

jyn514 commented Jan 25, 2023

cuviper commented Jan 25, 2023 • edited Loading

Mark-Simulacrum commented Jan 25, 2023

Mark-Simulacrum commented Jan 25, 2023

This comment was marked as resolved.

Mark-Simulacrum commented Jan 25, 2023

Mark-Simulacrum commented Jan 26, 2023

Mark-Simulacrum commented Jan 26, 2023

calebcartwright commented Jan 27, 2023 • edited Loading

pietroalbini commented Jan 30, 2023

This comment has been minimized.

rustbot commented Jan 31, 2023

Mark-Simulacrum commented Jan 31, 2023

bors commented Jan 31, 2023

bors commented Jan 31, 2023

bors commented Jan 31, 2023

rust-timer commented Jan 31, 2023

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

jyn514 Feb 1, 2023

Choose a reason for hiding this comment

cuviper Feb 1, 2023

Choose a reason for hiding this comment

jyn514 Feb 2, 2023

Choose a reason for hiding this comment

Mark-Simulacrum commented Jan 25, 2023 •

edited

Loading

bjorn3 commented Jan 25, 2023 •

edited

Loading

cuviper commented Jan 25, 2023 •

edited

Loading

calebcartwright commented Jan 27, 2023 •

edited

Loading