Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rustbuild: LLVM is not downloaded from CI after LLVM updates #122787

Closed
petrochenkov opened this issue Mar 20, 2024 · 5 comments · Fixed by #122632 or #129231
Closed

rustbuild: LLVM is not downloaded from CI after LLVM updates #122787

petrochenkov opened this issue Mar 20, 2024 · 5 comments · Fixed by #122632 or #129231
Labels
C-bug Category: This is a bug. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@petrochenkov
Copy link
Contributor

I tried this code:

./x build

download-ci-llvm in config.toml is not specified explicitly, the default value is used.

I expected to see this happen:

  • LLVM is downloaded from CI, when LLVM version is bumped the new LLVM is downloaded from CI as well.

Instead, this happened:

  • LLVM starts building from source code after an LLVM update, instead of being downloaded, unless I remove the build directory and do a fresh build, then it starts downloading from CI again.

Meta

rustc --version --verbose:

$ ./build/x86_64-pc-windows-gnu/stage0/bin/rustc --version --verbose
rustc 1.78.0-beta.1 (efd9d2df1 2024-03-18)
binary: rustc
commit-hash: efd9d2df12b5e17fac0b4d0fb89f612ecd79f259
commit-date: 2024-03-18
host: x86_64-pc-windows-gnu
release: 1.78.0-beta.1
LLVM version: 18.1.2
@petrochenkov petrochenkov added the C-bug Category: This is a bug. label Mar 20, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Mar 20, 2024
@petrochenkov
Copy link
Contributor Author

Or maybe it's a stage0 toolchain update that triggers this issue, not LLVM update (I observed this today).
But I've seen this issue several times in the past as well.
cc @onur-ozkan

@petrochenkov
Copy link
Contributor Author

The [llvm] section in config.toml is completely empty, nothing is overridden.

@onur-ozkan onur-ozkan added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Mar 20, 2024
@petrochenkov petrochenkov mentioned this issue Mar 22, 2024
Merged
@petrochenkov
Copy link
Contributor Author

Update: LLVM is built from source only on the first build attempt, after that it is correctly downloaded from CI.

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Apr 16, 2024
@matthiaskrgr
Rollup merge of rust-lang#122632 - onur-ozkan:fix-llvm-caching-bug, r…
04a2e3c 
…=albertlarsan68

fetch submodule before checking llvm stamp

Previously, we were checking the LLVM stamp before fetching the submodule which leads to not being able to compile llvm on submodule updates.

Fixes rust-lang#122612
Fixes rust-lang#122787
@bors bors closed this as completed in 592b1ca Apr 16, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 16, 2024
@rust-timer
Unrolled build for rust-lang#122632
b19896a 
Rollup merge of rust-lang#122632 - onur-ozkan:fix-llvm-caching-bug, r=albertlarsan68

fetch submodule before checking llvm stamp

Previously, we were checking the LLVM stamp before fetching the submodule which leads to not being able to compile llvm on submodule updates.

Fixes rust-lang#122612
Fixes rust-lang#122787
@petrochenkov
Copy link
Contributor Author

@onur-ozkan
This is still an issue that I observe regularly.

@petrochenkov petrochenkov reopened this Jul 29, 2024
@onur-ozkan
Copy link
Member

There were some issues with the change detection logic for LLVM and I think 2b5a982 should fix it. Let's see if it still happens with the next LLVM update.

@onur-ozkan onur-ozkan mentioned this issue Aug 18, 2024
Merged
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 18, 2024
@matthiaskrgr
Rollup merge of rust-lang#129231 - onur-ozkan:improve-submodule-updat…
3f3230b 
…es, r=Mark-Simulacrum

improve submodule updates

During config parsing, some bootstrap logic (e.g., `download-ci-llvm`) checks certain sources (for `download-ci-llvm`, it's `src/llvm-project`) and acts based on their state. This means that if path is a git submodule, bootstrap needs to update it before checking its state. Otherwise it may make incorrect assumptions by relying on outdated sources. To enable submodule updates during config parsing, we need to move the `update_submodule` function from the `Build` to `Config`, so we can access to it during the parsing process.

Closes rust-lang#122787
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 20, 2024
@jieyouxu
Rollup merge of rust-lang#129231 - onur-ozkan:improve-submodule-updat…
f90add5 
…es, r=Mark-Simulacrum

improve submodule updates

During config parsing, some bootstrap logic (e.g., `download-ci-llvm`) checks certain sources (for `download-ci-llvm`, it's `src/llvm-project`) and acts based on their state. This means that if path is a git submodule, bootstrap needs to update it before checking its state. Otherwise it may make incorrect assumptions by relying on outdated sources. To enable submodule updates during config parsing, we need to move the `update_submodule` function from the `Build` to `Config`, so we can access to it during the parsing process.

Closes rust-lang#122787
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 20, 2024
@jieyouxu
Rollup merge of rust-lang#129231 - onur-ozkan:improve-submodule-updat…
c7efc95 
…es, r=Mark-Simulacrum

improve submodule updates

During config parsing, some bootstrap logic (e.g., `download-ci-llvm`) checks certain sources (for `download-ci-llvm`, it's `src/llvm-project`) and acts based on their state. This means that if path is a git submodule, bootstrap needs to update it before checking its state. Otherwise it may make incorrect assumptions by relying on outdated sources. To enable submodule updates during config parsing, we need to move the `update_submodule` function from the `Build` to `Config`, so we can access to it during the parsing process.

Closes rust-lang#122787
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 22, 2024
@jieyouxu
Rollup merge of rust-lang#129231 - onur-ozkan:improve-submodule-updat…
44cdda4 
…es, r=Mark-Simulacrum

improve submodule updates

During config parsing, some bootstrap logic (e.g., `download-ci-llvm`) checks certain sources (for `download-ci-llvm`, it's `src/llvm-project`) and acts based on their state. This means that if path is a git submodule, bootstrap needs to update it before checking its state. Otherwise it may make incorrect assumptions by relying on outdated sources. To enable submodule updates during config parsing, we need to move the `update_submodule` function from the `Build` to `Config`, so we can access to it during the parsing process.

Closes rust-lang#122787
@bors bors closed this as completed in a08a2ef Aug 22, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Aug 22, 2024
@rust-timer
Unrolled build for rust-lang#129231
c2dd28a 
Rollup merge of rust-lang#129231 - onur-ozkan:improve-submodule-updates, r=Mark-Simulacrum

improve submodule updates

During config parsing, some bootstrap logic (e.g., `download-ci-llvm`) checks certain sources (for `download-ci-llvm`, it's `src/llvm-project`) and acts based on their state. This means that if path is a git submodule, bootstrap needs to update it before checking its state. Otherwise it may make incorrect assumptions by relying on outdated sources. To enable submodule updates during config parsing, we need to move the `update_submodule` function from the `Build` to `Config`, so we can access to it during the parsing process.

Closes rust-lang#122787
@onur-ozkan onur-ozkan mentioned this issue Aug 23, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-bug Category: This is a bug. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Milestone
No milestone
3 participants
@petrochenkov @onur-ozkan @rustbot