make dist vendoring configurable #130110

onur-ozkan · 2024-09-08T17:20:36Z

Adds a new option dist.vendor which allows people to decide whether to vendor dependencies for their custom distribution tarball builds. Note that our builds will not be affected, as the default for this option is the same as the previous vendoring condition from bootstrap.

rustbot · 2024-09-08T17:20:45Z

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2024-09-08T17:20:47Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies config.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

bors · 2024-09-10T10:58:58Z

☔ The latest upstream changes (presumably #130179) made this pull request unmergeable. Please resolve the merge conflicts.

weihanglo

Thank you!

Kobzol · 2024-09-13T12:21:54Z

You can r=me (unless you want to wait for Mark :) ).

Mark-Simulacrum

r=me

Mark-Simulacrum · 2024-09-14T16:07:14Z

src/bootstrap/src/core/config/config.rs

-            set(&mut config.dist_include_mingw_linker, include_mingw_linker)
+            set(&mut config.dist_include_mingw_linker, include_mingw_linker);
+            config.dist_vendor = vendor.unwrap_or_else(|| {
+                // If we're building from git or tarball sources, enable it by default.


What's the alternative option to these two? Someone clones git and then deletes .git?

Someone clones git and then deletes .git?

That might be a bit awkward, but it’s still one possibility.

Another case that fits this scenario is using this download button:

Another case that fits this scenario is using this download button:

That produces a broken source archive as all git submodules missing. You can't even build the standard library from that archive as both stdarch (core::arch) and backtrace (std::backtrace) are submodules.

onur-ozkan · 2024-09-14T18:25:47Z

@bors r=Kobzol,Mark-Simulacrum

bors · 2024-09-14T18:25:49Z

📌 Commit 7e0fb6e has been approved by Kobzol,Mark-Simulacrum

It is now in the queue for this repository.

…r, r=Kobzol,Mark-Simulacrum make dist vendoring configurable Adds a new option `dist.vendor` which allows people to decide whether to vendor dependencies for their custom distribution tarball builds. Note that our builds will not be affected, as the default for this option is the same as the previous vendoring condition from bootstrap.

bors · 2024-09-15T07:39:28Z

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

git checkout configurable-dist-vendor (switch to your branch)
git fetch upstream master (retrieve the latest master)
git rebase upstream/master -p (rebase on top of it)
Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
git push self configurable-dist-vendor --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

Auto-merging src/bootstrap/src/utils/change_tracker.rs
CONFLICT (content): Merge conflict in src/bootstrap/src/utils/change_tracker.rs
Auto-merging src/bootstrap/src/core/config/config.rs
Automatic merge failed; fix conflicts and then commit the result.

workingjubilee · 2024-09-15T07:43:02Z

@bors r-

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan · 2024-09-15T11:10:27Z

Rebased.

@bors r=Kobzol,Mark-Simulacrum

bors · 2024-09-15T11:10:29Z

📌 Commit 13542cd has been approved by Kobzol,Mark-Simulacrum

It is now in the queue for this repository.

bjorn3 · 2024-09-15T11:29:14Z

config.example.toml

@@ -942,3 +942,6 @@
 # Copy the linker, DLLs, and various libraries from MinGW into the Rust toolchain.
 # Only applies when the host or target is pc-windows-gnu.
 #include-mingw-linker = true
+
+# Whether to vendor dependencies for the dist tarball.
+#vendor = if "is a tarball source" || "is a git repository" { true } else { false }


It shouldn't be necessary to vendor when building from a source tarball. We can just reuse the existing vendor directory.

I thought about that, but then decided to leave the task to cargo instead of handling it manually. I couldn't get rid of the paranoia about users potentially cluttering the vendor folder.

It shouldn't be necessary to vendor when building from a source tarball. We can just reuse the existing vendor directory.

Exactly!

I'm building using a source release tarball (when packaging for pkgsrc), and expect the tarball to be self-contained. Ref. issue #130708.

…iaskrgr Rollup of 5 pull requests Successful merges: - rust-lang#129439 (Implement feature `string_from_utf8_lossy_owned` for lossy conversion from `Vec<u8>` to `String` methods) - rust-lang#129828 (miri: treat non-memory local variables properly for data race detection) - rust-lang#130110 (make dist vendoring configurable) - rust-lang#130293 (Fix lint levels not getting overridden by attrs on `Stmt` nodes) - rust-lang#130342 (interpret, miri: fix dealing with overflow during slice indexing and allocation) Failed merges: - rust-lang#130394 (const: don't ICE when encountering a mutable ref to immutable memory) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#130110 - onur-ozkan:configurable-dist-vendor, r=Kobzol,Mark-Simulacrum make dist vendoring configurable Adds a new option `dist.vendor` which allows people to decide whether to vendor dependencies for their custom distribution tarball builds. Note that our builds will not be affected, as the default for this option is the same as the previous vendoring condition from bootstrap.

see rust-lang/rust#130099 from rust-lang/rust#130110

rustbot assigned Mark-Simulacrum Sep 8, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Sep 8, 2024

onur-ozkan force-pushed the configurable-dist-vendor branch from 4d980a1 to 427af35 Compare September 8, 2024 17:21

This comment has been minimized.

Sign in to view

onur-ozkan force-pushed the configurable-dist-vendor branch from 50f7227 to c2a510b Compare September 8, 2024 20:05

This comment has been minimized.

Sign in to view

onur-ozkan force-pushed the configurable-dist-vendor branch from c2a510b to 0207cd5 Compare September 8, 2024 20:14

onur-ozkan force-pushed the configurable-dist-vendor branch from 0207cd5 to 7e0fb6e Compare September 10, 2024 16:18

onur-ozkan mentioned this pull request Sep 11, 2024

rust 1.81.0 fails to build without network access #130099

Open

weihanglo reviewed Sep 11, 2024

View reviewed changes

Mark-Simulacrum reviewed Sep 14, 2024

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 14, 2024

Zalathar mentioned this pull request Sep 15, 2024

Rollup of 6 pull requests #130378

Closed

matthiaskrgr mentioned this pull request Sep 15, 2024

Rollup of 4 pull requests #130385

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 15, 2024

onur-ozkan added 2 commits September 15, 2024 12:23

make dist vendoring configurable

741b316

Signed-off-by: onur-ozkan <work@onurozkan.dev>

document dist.vendor in config.example.toml

9778f25

Signed-off-by: onur-ozkan <work@onurozkan.dev>

add change entry for dist.vendor

13542cd

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the configurable-dist-vendor branch from 7e0fb6e to 13542cd Compare September 15, 2024 09:31

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 15, 2024

bjorn3 reviewed Sep 15, 2024

View reviewed changes

matthiaskrgr mentioned this pull request Sep 15, 2024

Rollup of 5 pull requests #130401

Merged

bors merged commit 8ad52dd into rust-lang:master Sep 15, 2024
6 checks passed

rustbot added this to the 1.83.0 milestone Sep 15, 2024

onur-ozkan deleted the configurable-dist-vendor branch September 16, 2024 05:20

bjorn3 mentioned this pull request Sep 22, 2024

Building rust 1.81.0 does network access in the "build" phase #130708

Closed

pld-gitsync pushed a commit to pld-linux/rust that referenced this pull request Sep 25, 2024

upstream fix for network access during build

718b991

see rust-lang/rust#130099 from rust-lang/rust#130110

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make dist vendoring configurable #130110

make dist vendoring configurable #130110

onur-ozkan commented Sep 8, 2024

rustbot commented Sep 8, 2024

rustbot commented Sep 8, 2024

This comment has been minimized.

This comment has been minimized.

bors commented Sep 10, 2024

weihanglo left a comment

Kobzol commented Sep 13, 2024

Mark-Simulacrum left a comment

Mark-Simulacrum Sep 14, 2024

onur-ozkan Sep 14, 2024

bjorn3 Sep 14, 2024

onur-ozkan commented Sep 14, 2024

bors commented Sep 14, 2024

bors commented Sep 15, 2024

workingjubilee commented Sep 15, 2024

onur-ozkan commented Sep 15, 2024

bors commented Sep 15, 2024

bjorn3 Sep 15, 2024

onur-ozkan Sep 15, 2024

he32 Sep 22, 2024

make dist vendoring configurable #130110

make dist vendoring configurable #130110

Conversation

onur-ozkan commented Sep 8, 2024

rustbot commented Sep 8, 2024

rustbot commented Sep 8, 2024

This comment has been minimized.

This comment has been minimized.

bors commented Sep 10, 2024

weihanglo left a comment

Choose a reason for hiding this comment

Kobzol commented Sep 13, 2024

Mark-Simulacrum left a comment

Choose a reason for hiding this comment

Mark-Simulacrum Sep 14, 2024

Choose a reason for hiding this comment

onur-ozkan Sep 14, 2024

Choose a reason for hiding this comment

bjorn3 Sep 14, 2024

Choose a reason for hiding this comment

onur-ozkan commented Sep 14, 2024

bors commented Sep 14, 2024

bors commented Sep 15, 2024

workingjubilee commented Sep 15, 2024

onur-ozkan commented Sep 15, 2024

bors commented Sep 15, 2024

bjorn3 Sep 15, 2024

Choose a reason for hiding this comment

onur-ozkan Sep 15, 2024

Choose a reason for hiding this comment

he32 Sep 22, 2024

Choose a reason for hiding this comment