Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: increase timeout from 4h to 6h #132605

Merged
merged 1 commit into from
Nov 27, 2024
Merged

Conversation

Kobzol
Copy link
Contributor

@Kobzol Kobzol commented Nov 4, 2024

Our CI got a bit slower since the last time we lowered the timeout, and if e.g. Docker build cache is broken, the timeout can be triggered.

Discussed here.

Our CI got a bit slower since the last time we lowered the timeout, and if e.g. Docker build cache is broken, the timeout can be triggered.
@rustbot
Copy link
Collaborator

rustbot commented Nov 4, 2024

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 4, 2024
@ehuss
Copy link
Contributor

ehuss commented Nov 4, 2024

Is there a particular reason to have a timeout longer than the bors timeout?

@Kobzol
Copy link
Contributor Author

Kobzol commented Nov 4, 2024

(For context: the homu timeout is configured here.)

Normally, I would say "no", but I wonder if the homu timeout even works. I vaguely remember that I lowered the timeout because we had some 6 or even 8 hour jobs due to some issue with slow tests on CI. But the homu timeout was 4.25 hours for the last four years, according to git, so it seems unlikely that we'd hit these durations if the homu timeout has worked. But maybe I remember it wrong.

@ehuss
Copy link
Contributor

ehuss commented Nov 4, 2024

I think it is working. Here is a recent example: #131397 (comment)

It may not show up very often now because homu's timeout is 4.25 hours, but the Actions timeout is set to 4.

@Kobzol
Copy link
Contributor Author

Kobzol commented Nov 4, 2024

I see, thanks for the example. What is interesting there is that an individual job timed out after 4 hours, but the job took ~15 minutes to start, so the whole workflow was actually timed out by homu sooner than that. However, it does not seem like homu actually cancelled that workflow! It posted the "timed out" comment at 10:27, but the workflow finished normally (with a failure) at 10:28. So it looks like homu does not in fact cancel timed out workflows? (new bors does this).

If that is the case, then we should perhaps also increase the homu timeout (but any homu change and redeploy is quite scary, the last one was 9 months ago 🥶) and configure the GHA timeout to be similar-ish to the homu timeout.

@ehuss
Copy link
Contributor

ehuss commented Nov 4, 2024

Yea, I was going to say I don't think homu actually cancels the workflow right away. However, I think that maybe when it force-pushes a new commit to the branch (when it moves to the next PR), GitHub will cancel it? I don't remember. It may also depend on cancel-in-progress. I do recall cancellation was a little strange.

@Kobzol
Copy link
Contributor Author

Kobzol commented Nov 4, 2024

It does not cancel it, because we explicitly allow parallel try builds, even on the same PR (https://github.com/rust-lang/rust/blob/master/.github/workflows/ci.yml#L37).

@Kobzol
Copy link
Contributor Author

Kobzol commented Nov 5, 2024

Also, this is relevant (again, I remembered this now...): https://docs.github.com/en/actions/administering-github-actions/usage-limits-billing-and-administration#usage-limits The maximum job runtime is actually 6 hours.

@workingjubilee
Copy link
Member

#132919 (comment) time explosion

@Mark-Simulacrum
Copy link
Member

@bors r+

Seems OK - and easily reversible. In general 4 vs 6 hours doesn't seem particularly meaningful.

@bors
Copy link
Contributor

bors commented Nov 24, 2024

📌 Commit b910e97 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 24, 2024
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Nov 25, 2024
…-Simulacrum

CI: increase timeout from 4h to 6h

Our CI got a bit slower since the last time we [lowered](rust-lang#127648) the timeout, and if e.g. Docker build cache is broken, the timeout can be triggered.

Discussed [here](https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/ci.20job.20timings.20stats).
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 25, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133134 (Don't use a SyntheticProvider for literally every type)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133134 (Don't use a SyntheticProvider for literally every type)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133134 (Don't use a SyntheticProvider for literally every type)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)

r? `@ghost`
`@rustbot` modify labels: rollup
jhpratt added a commit to jhpratt/rust that referenced this pull request Nov 26, 2024
…-Simulacrum

CI: increase timeout from 4h to 6h

Our CI got a bit slower since the last time we [lowered](rust-lang#127648) the timeout, and if e.g. Docker build cache is broken, the timeout can be triggered.

Discussed [here](https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/ci.20job.20timings.20stats).
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
Rollup of 28 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133136 (Support ranges in `<[T]>::get_many_mut()`)
 - rust-lang#133140 (Inline ExprPrecedence::order into Expr::precedence)
 - rust-lang#133248 (CI: split x86_64-msvc-ext job)
 - rust-lang#133282 (Shorten the `MaybeUninit` `Debug` implementation)
 - rust-lang#133304 (Revert diagnostics hack to fix ICE 132920)
 - rust-lang#133326 (Remove the `DefinitelyInitializedPlaces` analysis.)
 - rust-lang#133362 (No need to re-sort existential preds in relate impl)
 - rust-lang#133367 (Simplify array length mismatch error reporting (to not try to turn consts into target usizes))
 - rust-lang#133394 (Bail on more errors in dyn ty lowering)
 - rust-lang#133410 (target check_consistency: ensure target feature string makes some basic sense)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)
 - rust-lang#133443 (Remove dead code stemming from the old effects desugaring (II))
 - rust-lang#133449 (std: expose `const_io_error!` as `const_error!`)
 - rust-lang#133450 (remove "onur-ozkan" from  users_on_vacation)
 - rust-lang#133454 (Update test expectations to accept LLVM 'initializes' attribute)
 - rust-lang#133458 (Fix `Result` and `Option` not getting a jump to def link generated)
 - rust-lang#133462 (Use ReadCache for archive reading in bootstrap)
 - rust-lang#133464 (std::thread: avoid leading whitespace in some panic messages)
 - rust-lang#133467 (tests: Add recursive associated type bound regression tests)
 - rust-lang#133470 (Cleanup: delete `//@ pretty-expanded` directive)
 - rust-lang#133473 (tests: Add regression test for recursive enum with Cow and Clone)
 - rust-lang#133481 (Disable `avr-rjmp-offset` on Windows for now)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
Rollup of 28 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133136 (Support ranges in `<[T]>::get_many_mut()`)
 - rust-lang#133140 (Inline ExprPrecedence::order into Expr::precedence)
 - rust-lang#133248 (CI: split x86_64-msvc-ext job)
 - rust-lang#133282 (Shorten the `MaybeUninit` `Debug` implementation)
 - rust-lang#133304 (Revert diagnostics hack to fix ICE 132920)
 - rust-lang#133326 (Remove the `DefinitelyInitializedPlaces` analysis.)
 - rust-lang#133362 (No need to re-sort existential preds in relate impl)
 - rust-lang#133367 (Simplify array length mismatch error reporting (to not try to turn consts into target usizes))
 - rust-lang#133394 (Bail on more errors in dyn ty lowering)
 - rust-lang#133410 (target check_consistency: ensure target feature string makes some basic sense)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)
 - rust-lang#133443 (Remove dead code stemming from the old effects desugaring (II))
 - rust-lang#133449 (std: expose `const_io_error!` as `const_error!`)
 - rust-lang#133450 (remove "onur-ozkan" from  users_on_vacation)
 - rust-lang#133454 (Update test expectations to accept LLVM 'initializes' attribute)
 - rust-lang#133458 (Fix `Result` and `Option` not getting a jump to def link generated)
 - rust-lang#133462 (Use ReadCache for archive reading in bootstrap)
 - rust-lang#133464 (std::thread: avoid leading whitespace in some panic messages)
 - rust-lang#133467 (tests: Add recursive associated type bound regression tests)
 - rust-lang#133470 (Cleanup: delete `//@ pretty-expanded` directive)
 - rust-lang#133473 (tests: Add regression test for recursive enum with Cow and Clone)
 - rust-lang#133481 (Disable `avr-rjmp-offset` on Windows for now)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
Rollup of 28 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133136 (Support ranges in `<[T]>::get_many_mut()`)
 - rust-lang#133140 (Inline ExprPrecedence::order into Expr::precedence)
 - rust-lang#133248 (CI: split x86_64-msvc-ext job)
 - rust-lang#133282 (Shorten the `MaybeUninit` `Debug` implementation)
 - rust-lang#133304 (Revert diagnostics hack to fix ICE 132920)
 - rust-lang#133326 (Remove the `DefinitelyInitializedPlaces` analysis.)
 - rust-lang#133362 (No need to re-sort existential preds in relate impl)
 - rust-lang#133367 (Simplify array length mismatch error reporting (to not try to turn consts into target usizes))
 - rust-lang#133394 (Bail on more errors in dyn ty lowering)
 - rust-lang#133410 (target check_consistency: ensure target feature string makes some basic sense)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)
 - rust-lang#133443 (Remove dead code stemming from the old effects desugaring (II))
 - rust-lang#133449 (std: expose `const_io_error!` as `const_error!`)
 - rust-lang#133450 (remove "onur-ozkan" from  users_on_vacation)
 - rust-lang#133454 (Update test expectations to accept LLVM 'initializes' attribute)
 - rust-lang#133458 (Fix `Result` and `Option` not getting a jump to def link generated)
 - rust-lang#133462 (Use ReadCache for archive reading in bootstrap)
 - rust-lang#133464 (std::thread: avoid leading whitespace in some panic messages)
 - rust-lang#133467 (tests: Add recursive associated type bound regression tests)
 - rust-lang#133470 (Cleanup: delete `//@ pretty-expanded` directive)
 - rust-lang#133473 (tests: Add regression test for recursive enum with Cow and Clone)
 - rust-lang#133481 (Disable `avr-rjmp-offset` on Windows for now)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 27, 2024
…mpiler-errors

Rollup of 8 pull requests

Successful merges:

 - rust-lang#115293 (Remove -Zfuel.)
 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133304 (Revert diagnostics hack to fix ICE 132920)
 - rust-lang#133402 (Constify `Drop` and `Destruct`)
 - rust-lang#133458 (Fix `Result` and `Option` not getting a jump to def link generated)
 - rust-lang#133471 (gce: fix typing_mode mismatch)
 - rust-lang#133475 (`MaybeStorage` improvements)
 - rust-lang#133513 (Only ignore windows-gnu in avr-jmp-offset)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 24e7196 into rust-lang:master Nov 27, 2024
6 checks passed
@rustbot rustbot added this to the 1.85.0 milestone Nov 27, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Nov 27, 2024
Rollup merge of rust-lang#132605 - Kobzol:ci-increase-timeout, r=Mark-Simulacrum

CI: increase timeout from 4h to 6h

Our CI got a bit slower since the last time we [lowered](rust-lang#127648) the timeout, and if e.g. Docker build cache is broken, the timeout can be triggered.

Discussed [here](https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/ci.20job.20timings.20stats).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants