-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: move most of the prepare config into scripts #65202
Conversation
The job Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
@bors: r+ Thanks for this @pietroalbini! |
📌 Commit 1591a8e has been approved by |
@bors rollup=never At least for linux we're sure it works (the PR builders were fine), but there is a chance I forgot to copy something/typoed stuff for the other platforms, and I don't want this to affect a rollup. |
⌛ Testing commit 1591a8e with merge f973dbb260d9d091ec71666190125d1df9d1abe6... |
💔 Test failed - checks-azure |
Your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem. Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
|
@bors r- (bors sync goes badly...) |
This wasn't actually awscli missing, but the whole prepare step failed to run as it used @bors r=alexcrichton |
📌 Commit 8d1c3ff has been approved by |
⌛ Testing commit 8d1c3ff with merge a103c61b9ccbce6c6f65b03d349efbad6c906a35... |
Your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem. Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
💔 Test failed - checks-azure |
@bors try |
ci: move most of the prepare config into scripts This PR moves most of the configuration from the CI yamls into bash scripts, driven by a small Python script (which understands and emulates the two `##vso[` commands we use). There are two reasons why we'd want to do this: * Being able to prepare the build environment locally by just running `src/ci/prepare.py` simplifies a lot setting up a local VM similar to CI (software pre-installed in the CI images won't be prepared, but it's a start anyway). * When we'll switch to GitHub Actions we'll need to either duplicate code in multiple workflows or write a preprocessor. Having all the prepare steps in a single one is going to simplify the implementation of both options. Along with the move I did a few changes to the actual scripts: * Mirrored all the remaining external URLs we download (except chocolatey) to the `rust-lang-ci-mirrors` bucket, to increase reliability and reduce the chance of supply chain attacks. I didn't audit and mirror the CI scripts outside this PR though. * Extracted CI-specific behavior (like issuing `##vso[` commands and detecting the host platform) into `shared.sh` and included it in most of the scripts. This way a switch to another CI provider will be less painful. It's possible (and easier) to review this commit-by-commit. r? @alexcrichton cc @rust-lang/infra
☀️ Try build successful - checks-azure |
af5cc3d
to
7e05123
Compare
@bors r=alexcrichton p=10 rollup=never |
📌 Commit 7e05123 has been approved by |
ci: move most of the prepare config into scripts This PR moves most of the configuration from the CI yamls into bash scripts, driven by a small Python script (which understands and emulates the two `##vso[` commands we use). There are two reasons why we'd want to do this: * Being able to prepare the build environment locally by just running `src/ci/prepare.py` simplifies a lot setting up a local VM similar to CI (software pre-installed in the CI images won't be prepared, but it's a start anyway). * When we'll switch to GitHub Actions we'll need to either duplicate code in multiple workflows or write a preprocessor. Having all the prepare steps in a single one is going to simplify the implementation of both options. Along with the move I did a few changes to the actual scripts: * Mirrored all the remaining external URLs we download (except chocolatey) to the `rust-lang-ci-mirrors` bucket, to increase reliability and reduce the chance of supply chain attacks. I didn't audit and mirror the CI scripts outside this PR though. * Extracted CI-specific behavior (like issuing `##vso[` commands and detecting the host platform) into `shared.sh` and included it in most of the scripts. This way a switch to another CI provider will be less painful. It's possible (and easier) to review this commit-by-commit. r? @alexcrichton cc @rust-lang/infra
☀️ Test successful - checks-azure |
Finally 🎉 |
…=alexcrichton Upload toolstates.json to rust-lang-ci2 This PR does two things: * Following up with rust-lang#65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended. * Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot. r? @alexcrichton cc @RalfJung
Upload toolstates.json to rust-lang-ci2 This PR does two things: * Following up with #65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended. * Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot. r? @alexcrichton cc @RalfJung
Upload toolstates.json to rust-lang-ci2 This PR does two things: * Following up with #65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended. * Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot. r? @alexcrichton cc @RalfJung
Upload toolstates.json to rust-lang-ci2 This PR does two things: * Following up with #65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended. * Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot. r? @alexcrichton cc @RalfJung
…=alexcrichton Upload toolstates.json to rust-lang-ci2 This PR does two things: * Following up with rust-lang#65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended. * Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot. r? @alexcrichton cc @RalfJung
This PR moves most of the configuration from the CI yamls into bash scripts, driven by a small Python script (which understands and emulates the two
##vso[
commands we use).There are two reasons why we'd want to do this:
src/ci/prepare.py
simplifies a lot setting up a local VM similar to CI (software pre-installed in the CI images won't be prepared, but it's a start anyway).Along with the move I did a few changes to the actual scripts:
rust-lang-ci-mirrors
bucket, to increase reliability and reduce the chance of supply chain attacks. I didn't audit and mirror the CI scripts outside this PR though.##vso[
commands and detecting the host platform) intoshared.sh
and included it in most of the scripts. This way a switch to another CI provider will be less painful.It's possible (and easier) to review this commit-by-commit.
r? @alexcrichton
cc @rust-lang/infra