Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new download-ci-llvm = if-unchanged option and enable it by default for profile = codegen #116881

Merged
merged 1 commit into from
Nov 8, 2023

Conversation

LuuuXXX
Copy link
Contributor

@LuuuXXX LuuuXXX commented Oct 18, 2023

Three tasks have been implemented here.

Add a new download-ci-llvm = if-unchange option and enable if by
default for profile = codegen.

Include all build artifacts by traversing the llvm-project build output,
Keep the downloadable llvm the same state as if you have just run a full
source build.

After selecting the codegen profile during ./x.py setup, the submodule
will be automatically downloaded.

It should be noted that:
if you are using the 'codegen' profile, be aware that the default value changed. otherwise, you can ignore this.

Resolves #110087

@rustbot
Copy link
Collaborator

rustbot commented Oct 18, 2023

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Oct 18, 2023
@rustbot
Copy link
Collaborator

rustbot commented Oct 18, 2023

This PR changes config.example.toml. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/lib.rs and change-id in config.example.toml.

This PR changes src/bootstrap/defaults/config.codegen.toml. If appropriate, please also update config.compiler.toml so the defaults are in sync.

@rust-log-analyzer

This comment has been minimized.

@rustbot
Copy link
Collaborator

rustbot commented Oct 18, 2023

This PR changes src/bootstrap/defaults/config.codegen.toml. If appropriate, please also update config.compiler.toml so the defaults are in sync.

This PR changes config.example.toml. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/lib.rs and change-id in config.example.toml.

@rust-log-analyzer

This comment has been minimized.

@LuuuXXX
Copy link
Contributor Author

LuuuXXX commented Oct 19, 2023

By the way, triagebot.toml should be updated because the bootstrap codebase is reorganized. The tips above seem out of date.

Copy link
Member

@onur-ozkan onur-ozkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current PR ID(which is 116881) should be appended to this constant:

/// Keeps track of major changes made to the bootstrap configuration.
///
/// These values also represent the IDs of the PRs that caused major changes.
/// You can visit `https://github.com/rust-lang/rust/pull/{any-id-from-the-list}` to
/// check for more details regarding each change.
///
/// If you make any major changes (such as adding new values or changing default values), please
/// ensure that the associated PR ID is added to the end of this list.
pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898];

This should be replaced with the current PR ID:

rust/config.example.toml

Lines 22 to 33 in 020d008

# Keeps track of major changes made to this configuration.
#
# This value also represents ID of the PR that caused major changes. Meaning,
# you can visit github.com/rust-lang/rust/pull/{change-id} to check for more details.
#
# A 'major change' includes any of the following
# - A new option
# - A change in the default values
#
# If `change-id` does not match the version that is currently running,
# `x.py` will prompt you to update it and check the related PR for more details.
change-id = 115898

@rust-log-analyzer

This comment has been minimized.

Copy link
Member

@onur-ozkan onur-ozkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still need some work to confirm the second task, maybe a separate PR is needed.

I think we can handle it in this PR

@onur-ozkan onur-ozkan added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 21, 2023
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Oct 21, 2023
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Oct 21, 2023
Rollup merge of rust-lang#116928 - onur-ozkan:update-triagebot, r=Mark-Simulacrum

fix bootstrap paths in triagebot.toml

ref #rust-lang#116881 (comment)
@rustbot
Copy link
Collaborator

rustbot commented Oct 23, 2023

This PR modifies src/bootstrap/src/core/config. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/src/lib.rs and change-id in config.example.toml.

This PR modifies src/bootstrap/defaults. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/src/lib.rs and change-id in config.example.toml.

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Oct 24, 2023

☔ The latest upstream changes (presumably #116998) made this pull request unmergeable. Please resolve the merge conflicts.

@LuuuXXX LuuuXXX requested a review from onur-ozkan October 24, 2023 08:10
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 24, 2023
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 7, 2023
@onur-ozkan
Copy link
Member

Can you please rebase and push the commit with a more descriptive message?

Commit history still looks inappropriate.

@LuuuXXX
Copy link
Contributor Author

LuuuXXX commented Nov 7, 2023

reworded it

Copy link
Member

@onur-ozkan onur-ozkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you squash your commits into "Fix issue #110087" one ? Following commits alone doesn't make too much sense as far as I see. Specially the 2nd one.

src/bootstrap/src/core/build_steps/dist.rs Outdated Show resolved Hide resolved
Three tasks have been implemented here.

Add a new `download-ci-llvm = if-unchange` option and enable if by
default for `profile = codegen`.

Include all build artifacts by traversing the llvm-project build output,
Keep the downloadable llvm the same state as if you have just run a full
source build.

After selecting the codegen profile during ./x.py setup, the submodule
will be automatically downloaded.
Copy link
Member

@onur-ozkan onur-ozkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the fixes.

CC #110087
I still need some work to confirm the second task, maybe a separate PR is needed.

PR description seems to be outdated, it would be nice if you update it :) I will queue up the PR anyway.

@onur-ozkan
Copy link
Member

@bors r+ rollup=iffy

@bors
Copy link
Contributor

bors commented Nov 8, 2023

📌 Commit 1b8dee1 has been approved by onur-ozkan

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 8, 2023
@bors
Copy link
Contributor

bors commented Nov 8, 2023

⌛ Testing commit 1b8dee1 with merge 750c2ec...

@bors
Copy link
Contributor

bors commented Nov 8, 2023

☀️ Test successful - checks-actions
Approved by: onur-ozkan
Pushing 750c2ec to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Nov 8, 2023
@bors bors merged commit 750c2ec into rust-lang:master Nov 8, 2023
12 checks passed
@rustbot rustbot added this to the 1.75.0 milestone Nov 8, 2023
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (750c2ec): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.0% [0.5%, 1.5%] 3
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.7% [-0.8%, -0.6%] 4
Improvements ✅
(secondary)
-3.7% [-3.7%, -3.7%] 1
All ❌✅ (primary) 0.0% [-0.8%, 1.5%] 7

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.4% [0.4%, 0.4%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.4% [-0.4%, -0.4%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.0% [-0.4%, 0.4%] 2

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-0.0% [-0.0%, -0.0%] 4
All ❌✅ (primary) - - 0

Bootstrap: 663.473s -> 664.216s (0.11%)
Artifact size: 308.82 MiB -> 308.79 MiB (-0.01%)

@@ -77,7 +77,7 @@ const LLD_FILE_NAMES: &[&str] = &["ld.lld", "ld64.lld", "lld-link", "wasm-ld"];
///
/// If you make any major changes (such as adding new values or changing default values), please
/// ensure that the associated PR ID is added to the end of this list.
pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898, 116998, 117435];
pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898, 116998, 117435, 116881];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting a WARNING pointing me to this PR. Do I need to do something? Or can I keep my existing config? It's unclear to me. The PR description says there's a new config, but why does everyone have to be alerted of its existence?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this list is unordered now, that seems wrong?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update change-id = 116881 instead may be help.
AFAIK This is ordered by merge time to keep the source code synchronized with the config file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update change-id = 116881 instead may be help.

That will silence the warning, sure. But the reason the warning is shown must be that I should do something before silencing it. There's no point in having people mindlessly increment a number every week just to get rid of the latest warning.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this list is unordered now, that seems wrong?

/// If you make any major changes (such as adding new values or changing default values),
/// please ensure that the associated PR ID is added to the end of this list.
/// This is necessary because the list must be sorted by the merge date.

I'm getting a WARNING pointing me to this PR. Do I need to do something? Or can I keep my existing config? It's unclear to me

# build llvm from source
download-ci-llvm = "if-unchanged"

PR updated the default value here. I am pretty sure we have people who wants to know this change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the PR description needs to say: "if you are coming here from x.py, here's what do do: if you are using the 'codegen' profile, be aware that the default value changed. otherwise, you can ignore this".

Currently I'm coming here clueless and wasting 5min trying to figure out if I should do something, before giving up, frustrated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's my bad, I will add the description for this PR later

@RalfJung
Copy link
Member

For this PR I am also confused why we have if-available and if-unchanged as two different options. When would I ever want if-available? Shouldn't if-unchanged just be the default everywhere, since it will "do the right thing" in most situations?

@LuuuXXX
Copy link
Contributor Author

LuuuXXX commented Nov 11, 2023

When would I ever want if-available?

If there is no need to modify llvm locally, if-available may be able to give you faster build times. Since you can use Rust CI built LLVM instead of locally building it. and if-unchanged will consider whether there are local modifications in llvm.

Shouldn't if-unchanged just be the default everywhere, since it will "do the right thing" in most situations?

Is it similar to this? https://github.com/orgs/rust-lang/projects/35/views/1?pane=issue&itemId=40227883

@RalfJung
Copy link
Member

If there is no need to modify llvm locally, if-available may be able to give you faster build times.

Have you measured it? How long does the change-check take?

We also have true for skipping all the checks, after all.

@LuuuXXX
Copy link
Contributor Author

LuuuXXX commented Nov 11, 2023

Not yet, the change-check is based on git diff-index it will not take too much time.

Zalathar added a commit to Zalathar/rust that referenced this pull request Dec 17, 2023
This comment became obsolete after rust-lang#116881 switched over to
`download-ci-llvm = "if-unchanged"`.
Zalathar added a commit to Zalathar/rust that referenced this pull request Dec 17, 2023
This comment became obsolete after rust-lang#116881 switched over to
`download-ci-llvm = "if-unchanged"`.
@dtolnay dtolnay removed the has-merge-commits PR has merge commits, merge with caution. label Jan 21, 2024
celinval added a commit to celinval/rust-dev that referenced this pull request Jun 4, 2024
Update Rust toolchain from nightly-2023-11-08 to nightly-2023-11-09
without any other source changes.
This is an automatically generated pull request. If any of the CI checks
fail, manual intervention is required. In such a case, review the
changes at https://github.com/rust-lang/rust from
rust-lang@7adc89b
up to
rust-lang@fdaaaf9.
The log for this commit range is:
rust-lang@fdaaaf9f92 Auto merge of
rust-lang#116930 - RalfJung:raw-ptr-match, r=davidtwco
rust-lang@30588657b7 avoid unnecessary
nested conditionals
rust-lang@90fdc1fc27 Auto merge of
rust-lang#117716 - GuillaumeGomez:rollup-83gnhll, r=GuillaumeGomez
rust-lang@9d3c80248b Rollup merge of
rust-lang#117713 - GuillaumeGomez:document-hidden-json, r=notriddle
rust-lang@d8c52b378d Rollup merge of
rust-lang#117702 - davidtwco:target-tier-refactors, r=petrochenkov
rust-lang@5d00a5d936 Rollup merge of
rust-lang#117679 - aDotInTheVoid:yes-core, r=GuillaumeGomez
rust-lang@c828371179 Rollup merge of
rust-lang#117282 - clubby789:recover-wrong-function-header, r=TaKO8Ki
rust-lang@e05e4f38b5 Rollup merge of
rust-lang#117263 - onur-ozkan:change-id-fix, r=saethlin
rust-lang@341efb1017 Auto merge of
rust-lang#117560 - lqd:issue-117146, r=matthewjasper
rust-lang@33edea60f0 Add test for
reexported hidden item with `--document-hidden-items`
rust-lang@28acba3c61 Auto merge of
rust-lang#115460 - zachs18:borrowedcursor_write_no_panic, r=dtolnay
rust-lang@755629fe59 Auto merge of
rust-lang#117706 - matthiaskrgr:rollup-lscx7dg, r=matthiaskrgr
rust-lang@e0cb1cc296 bootstrap: add more
detail on change-id comments
rust-lang@e878100386 bootstrap: improve
`fn check_version`
rust-lang@ae4d18b2da handle the case when
the change-id isn't found
rust-lang@7e4ffa98b5 Rollup merge of
rust-lang#117700 - Zalathar:rename-run-coverage, r=onur-ozkan
rust-lang@55306535dd Rollup merge of
rust-lang#117698 - nnethercote:space_between-2, r=petrochenkov
rust-lang@b1b5a8ea9d Rollup merge of
rust-lang#117667 - Alexendoo:doc-clippy-config, r=albertlarsan68
rust-lang@adf4981969 Rollup merge of
rust-lang#117663 - klensy:bump-deps, r=davidtwco
rust-lang@8198864377 Rollup merge of
rust-lang#117650 - saethlin:inline-me-please, r=davidtwco
rust-lang@ba7ec56639 Rollup merge of
rust-lang#117531 - fmease:rustdoc-effects-properly-elide-x-crate-host-args,
r=GuillaumeGomez
rust-lang@b74a84c0bc Rollup merge of
rust-lang#114316 - ecnelises:aix_doc, r=workingjubilee
rust-lang@fab1054e17 Auto merge of
rust-lang#117542 - compiler-errors:only-normalize-predicate, r=lcnr
rust-lang@750c2ecd15 Auto merge of
rust-lang#116881 - LuuuXXX:issue-110087, r=onur-ozkan
rust-lang@b9b7982f72 Add AIX
platform-support doc
rust-lang@ef7ebaa788 rustc_target: move
file for uniformity
rust-lang@1af256fe8a targets: move target
specs to spec/targets
rust-lang@76aa83e3e1 target: move base
specs to spec/base
rust-lang@a573880373 coverage: Rename the
`run-coverage` test mode to `coverage-run`
rust-lang@7cc997d373 Auto merge of
rust-lang#117699 - weihanglo:update-cargo, r=weihanglo
rust-lang@0670466e2c Update cargo
rust-lang@438b9a6e82 More tests for token
stream pretty-printing with adjacent punctuation.
rust-lang@783d4b8b26 Clarify
`space_between`.
rust-lang@91cfcb0219 Auto merge of
rust-lang#117484 - Zalathar:tests, r=cjgillot
rust-lang@97c9d8f405 Only use
normalize_param_env when normalizing predicate in check_item_bounds
rust-lang@1b8dee19e8 Fix issue rust-lang#110087
rust-lang@0d5ec963bb Auto merge of
rust-lang#117692 - matthiaskrgr:rollup-umaf5pr, r=matthiaskrgr
rust-lang@f72e974e3f Rollup merge of
rust-lang#117655 - compiler-errors:method-tweaks, r=estebank
rust-lang@7552dd19ad Rollup merge of
rust-lang#117625 - nnethercote:clippy-perf, r=cuviper
rust-lang@3c6307240c Rollup merge of
rust-lang#116399 - WaffleLapkin:erase_small_things, r=cjgillot
rust-lang@b724d9c90e Rollup merge of
rust-lang#113925 - clubby789:const-ctor-repeat, r=estebank
rust-lang@fcdd99edca Add
-Zcross-crate-inline-threshold=yes
rust-lang@e8cf29b584 rustdoc: minor
changes suggested by clippy perf lints.
rust-lang@ff0b4b6091 Auto merge of
rust-lang#117672 - lqd:ci-gcc-lld, r=Kobzol
rust-lang@1b3733e5a4 rustc: minor changes
suggested by clippy perf lints.
rust-lang@eca9a1533f Add an explanation
for `transmute_unchecked`
rust-lang@1d1fe9a205 add note to remember
to update a url when bumping to gcc 10.1.0
rust-lang@434b69a1d6 tests/rustdoc-json:
Rewrite tests no not use `#![no_core]`.
rust-lang@0875f456f1 tests/rustdoc-json:
Remove more needless uses of `#![no_core]`.
rust-lang@f784fa7bd9 tests/rustdoc-json:
Remove some needless uses of `#![no_core]`.
rust-lang@d73eaaac5f ci: bump gcc on dist
x64 linux builder to 9.5
rust-lang@f2fd8ad788 Document
clippy_config in nightly-rustc docs
rust-lang@eed89185bb bump some deps
rust-lang@0add056dee Rework
print_disambiguation_help
rust-lang@88a37acb26 Yeet
MethodCallComponents
rust-lang@4e6f438d2a coverage: Register
`test::Coverage` as the test suite for `tests/coverage`
rust-lang@49127c64d6 coverage: Migrate
`tests/coverage-map` into `tests/coverage`
rust-lang@e9d04c5e24 coverage: Migrate
`tests/run-coverage` into `tests/coverage`
rust-lang@aea7c27eae coverage: Set up a
macro for declaring unified coverage test suites
rust-lang@3509aed632 coverage: Add
`./x.py test coverage`, an alias for `coverage-map` and `run-coverage`
rust-lang@e585a99230 coverage: Give each
coverage test mode a separate output directory
rust-lang@211d4cee8e coverage: Copy all
remaining run-coverage tests into coverage-map
rust-lang@4b76b97bc8 coverage: Copy all
remaining coverage-map tests into run-coverage
rust-lang@f5df56b26b coverage: Flatten
`coverage-map/status-quo/` into its parent directory
rust-lang@8eef39f082 coverage: Remove
`tests/coverage-map/if.rs`
rust-lang@7f8a6de72c coverage: Use
`-Copt-level=2` by default in run-coverage tests
rust-lang@1dcdf83927 rustdoc: properly
elide cross-crate host effect args
rust-lang@2dff90dc23 add test for issue
117146
rust-lang@de7a8305ae traverse region
graph instead of SCCs to compute polonius loan scopes
rust-lang@03b24f2756 remove some dead
code
rust-lang@70a8e157ab make
pointer_structural_match warn-by-default
rust-lang@af6c7e0ca1 also lint against fn
ptr and raw ptr nested inside the const
rust-lang@bec88ad4aa patterns: reject raw
pointers that are not just integers
rust-lang@be0b42fabe Recover from
incorrectly ordered/duplicated function keywords
rust-lang@b3d50255d9 Use consisntent
style of `size_of` in query type erasure
rust-lang@61361bb212 Use
`transmute_unchecked` and make the types explicit in query type erasure
rust-lang@11a64a1834 don't panic in
BorrowedCursor::write
rust-lang@86b112204a Improve diagnostic
for const ctors in array repeat expressions

Co-authored-by: celinval <celinval@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a new download-ci-llvm = if-unchanged option and enable it by default for profile = codegen
9 participants