compiletest ice tracking #122997

matthiaskrgr · 2024-03-24T12:32:57Z

see https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/where.20to.20mass-add.20known.20ices.20.2F.20merging.20glacier.20into.20rust/near/429082963

This will allow us to sunset most of https://github.com/rust-lang/glacier
The rustc ices will be tracked directly inside the rust testsuite
There are a couple of .sh tests remaining that I have not ported over yet.

This adds tests/crashes, a file inside this directory MUST ice, otherwise it is considered test-fail.
This will be used to track ICEs from glacier and the bugtracker.
When someones pr accidentally fixes one of these ICEs, they can move the test from crashes into ui for example.

I also added a new tidy lint that warns when a test inside tests/crashes does not have a //@ known-bug: line

the env var COMPILETEST_VERBOSE_CRASHES can be set to get exit code, stderr and stdout of a crash-test to aid debugging/adding tests.

rustbot · 2024-03-24T12:33:04Z

r? @compiler-errors

rustbot has assigned @compiler-errors.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

tests/crashes/122909.rs

compiler-errors

Would it be possible also for you to implement a quick tidy check for all the tests/ui/crashes tests to have a known-bug: directive?

compiler-errors · 2024-03-24T14:33:01Z

src/tools/compiletest/src/runtest.rs

+        // if a test does not crash, consider it an error
+        match proc_res.status.code() {
+            Some(101) => (),
+            _ => self.fatal("expected ICE"),


Can you elaborate this message to tell the dev to give the test a better message and move it to tests/ui/something?

Specifically, I don't want people to just copy over the 122909.rs to tests/ui when they go from fail -> pass.

compiler-errors · 2024-03-24T14:33:45Z

tests/crashes/122909.rs

@@ -0,0 +1,15 @@
+//@ compile-flags: -Zpolymorphize=on -Zinline-mir=yes
+//@ known-bug: #12345


Suggested change

//@ known-bug: #12345

//@ known-bug: #122909

guys this is just a initial test to make sure that compile-flags are working :D

fmease · 2024-03-24T14:54:14Z

Could the folder be called ices/ instead if that's still available? I personally don't mind the terminology crashes for ICEs but @oli-obk recently posted a comment on one of the issues you reported which you might remember in which he gave the strong recommendation to abstain from using the term crashes to refer to ICEs since crashes are reserved for actual crashes (stack overflows, OOM kills, other interrupts), cc label I-crash.

matthiaskrgr · 2024-03-24T14:57:52Z

I don't think we should limit ourselves to only track "internal compiler errors" but also track stack overflows and llvm errors if possible, which is why I chose "crashes".

matthiaskrgr · 2024-03-24T16:50:21Z

@compiler-errors ^ :)

matthiaskrgr · 2024-03-24T21:51:26Z

Mhh, so current roablocks:

it seems everything for Crashes is run as check-fail but llvm assertion failures need something like build-fail
it would be beneficial to turn off backtraces for for all crash-tests as that could save 20-30% of runtime for small tests I think.

maybe I can wrap my head around some of that tomorrow

matthiaskrgr · 2024-03-29T21:02:53Z

src/tools/compiletest/src/runtest.rs

+        // if a test does not crash, consider it an error
+        match proc_res.status.code() {
+            Some(101) => (),
+            _ => self.fatal("expected ICE"),
+        }
+    }
+


Soo it turns out that this is not very sufficient at detecting when rustc is killed by a signal for example due to stack overflow.

There is something like std::os::unix::process::ExitStatusExt to get more information for cases like this, but it seems to be unix-only.

I think the best approach for now is to switch around the logic to only "deny" cases where exit status is 0 (no error) or 1 (compiler error) and accept everything else as ICE...?

if !proc_res.status.success() { match proc_res.status.code() { Some(1 | 0) => self.fatal(&format!("expected ICE")), _ => (), } }

rustbot · 2024-03-29T23:44:06Z

Some changes occurred in src/tools/compiletest

cc @jieyouxu

matthiaskrgr · 2024-03-29T23:44:26Z

r? @oli-obk

bors · 2024-04-14T21:46:31Z

💔 Test failed - checks-actions

…dows

matthiaskrgr · 2024-04-14T21:54:31Z

@bors r=oli-obk

bors · 2024-04-14T21:54:33Z

📌 Commit 2ce487c has been approved by oli-obk

It is now in the queue for this repository.

…-obk compiletest ice tracking see https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/where.20to.20mass-add.20known.20ices.20.2F.20merging.20glacier.20into.20rust/near/429082963 This will allow us to sunset most of https://github.com/rust-lang/glacier The rustc ices will be tracked directly inside the rust testsuite There are a couple of .sh tests remaining that I have not ported over yet. This adds `tests/crashes`, a file inside this directory MUST ice, otherwise it is considered test-fail. This will be used to track ICEs from glacier and the bugtracker. When someones pr accidentally fixes one of these ICEs, they can move the test from `crashes` into `ui` for example. I also added a new tidy lint that warns when a test inside `tests/crashes` does not have a `//@ known-bug: ` line the env var `COMPILETEST_VERBOSE_CRASHES` can be set to get exit code, stderr and stdout of a crash-test to aid debugging/adding tests.

bors · 2024-04-14T23:52:33Z

⌛ Testing commit 2ce487c with merge 22bfeee...

rust-log-analyzer · 2024-04-15T00:02:20Z

The job aarch64-gnu failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[RUSTC-TIMING] rustc_middle test:false 103.135
   Compiling rustc_lint v0.0.0 (/checkout/compiler/rustc_lint)
[RUSTC-TIMING] rustc_resolve test:false 65.818
   Compiling rustc_codegen_llvm v0.0.0 (/checkout/compiler/rustc_codegen_llvm)
##[error]The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

Session terminated, killing shell...::group::Clock drift check
  network time: Mon, 15 Apr 2024 00:02:00 GMT
##[endgroup]
 ...killed.
##[error]The operation was canceled.

bors · 2024-04-15T00:02:56Z

💔 Test failed - checks-actions

matthiaskrgr · 2024-04-15T04:51:57Z

@bors retry

bors · 2024-04-15T06:04:54Z

⌛ Testing commit 2ce487c with merge 85b884b...

bors · 2024-04-15T08:06:37Z

☀️ Test successful - checks-actions
Approved by: oli-obk
Pushing 85b884b to master...

rust-timer · 2024-04-15T09:29:21Z

Finished benchmarking commit (85b884b): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 677.058s -> 677.003s (-0.01%)
Artifact size: 315.97 MiB -> 316.05 MiB (0.02%)

lqd · 2024-06-12T13:10:15Z

src/tools/compiletest/src/runtest.rs

+        // if a test does not crash, consider it an error
+        if proc_res.status.success() || matches!(proc_res.status.code(), Some(1 | 0)) {
+            self.fatal(&format!(
+                "test no longer crashes/triggers ICE! Please give it a mearningful name, \


don't mind me, I'm just passing by

Suggested change

"test no longer crashes/triggers ICE! Please give it a mearningful name, \

"test no longer crashes/triggers ICE! Please give it a meaningful name, \

I think oli already fixed this in a drive-by cleanup somewhere 😅

Ah, did he? I just saw that in one of this PRs #126316 (comment)

I remember doing that, but it probably hasn't landed yet

rustbot assigned compiler-errors Mar 24, 2024

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 24, 2024

matthiaskrgr removed T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 24, 2024

This comment has been minimized.

Sign in to view

matthiaskrgr force-pushed the compiletest_ices branch from 3453ba6 to 02a00fb Compare March 24, 2024 12:42

rustbot added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Mar 24, 2024

Noratrieb reviewed Mar 24, 2024

View reviewed changes

tests/crashes/122909.rs Outdated Show resolved Hide resolved

matthiaskrgr force-pushed the compiletest_ices branch 2 times, most recently from ce95351 to 7fdf1e3 Compare March 24, 2024 12:51

This comment has been minimized.

Sign in to view

compiler-errors reviewed Mar 24, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

matthiaskrgr force-pushed the compiletest_ices branch 2 times, most recently from a8e19ac to da8597f Compare March 29, 2024 18:49

matthiaskrgr commented Mar 29, 2024

View reviewed changes

matthiaskrgr force-pushed the compiletest_ices branch 2 times, most recently from 1c2eca9 to de73263 Compare March 29, 2024 23:43

matthiaskrgr marked this pull request as ready for review March 29, 2024 23:44

rustbot assigned oli-obk and unassigned compiler-errors Mar 29, 2024

This comment has been minimized.

Sign in to view

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 14, 2024

crashes: limit a couple tests to only run on x86_64 and/or not on win…

2ce487c

…dows

matthiaskrgr force-pushed the compiletest_ices branch from 93c70a5 to 2ce487c Compare April 14, 2024 21:53

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 14, 2024

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 15, 2024

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 15, 2024

bors added the merged-by-bors This PR was explicitly merged by bors. label Apr 15, 2024

bors merged commit 85b884b into rust-lang:master Apr 15, 2024
12 checks passed

rustbot added this to the 1.79.0 milestone Apr 15, 2024

Zalathar mentioned this pull request Apr 17, 2024

Account for immutably borrowed locals in MIR copy-prop and GVN #123602

Merged

beetrees mentioned this pull request Apr 19, 2024

Correctly change type when adding adjustments on top of NeverToAny #123571

Merged

This was referenced May 22, 2024

🧊 Put glacier on ice 🧊 rust-lang/team#1459

Merged

Tracking issue for migrating from Glacier to compiletest ICE/crash tracking #125459

Open

lqd reviewed Jun 12, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

compiletest ice tracking #122997

compiletest ice tracking #122997

matthiaskrgr commented Mar 24, 2024 •

edited

Loading

rustbot commented Mar 24, 2024

This comment has been minimized.

This comment has been minimized.

compiler-errors left a comment

compiler-errors Mar 24, 2024

compiler-errors Mar 24, 2024

matthiaskrgr Mar 24, 2024

fmease commented Mar 24, 2024 •

edited

Loading

matthiaskrgr commented Mar 24, 2024

This comment has been minimized.

matthiaskrgr commented Mar 24, 2024

matthiaskrgr commented Mar 24, 2024

matthiaskrgr Mar 29, 2024

rustbot commented Mar 29, 2024

matthiaskrgr commented Mar 29, 2024

This comment has been minimized.

bors commented Apr 14, 2024

matthiaskrgr commented Apr 14, 2024

bors commented Apr 14, 2024

bors commented Apr 14, 2024

rust-log-analyzer commented Apr 15, 2024

bors commented Apr 15, 2024

matthiaskrgr commented Apr 15, 2024

bors commented Apr 15, 2024

bors commented Apr 15, 2024

rust-timer commented Apr 15, 2024

lqd Jun 12, 2024

matthiaskrgr Jun 12, 2024

lqd Jun 12, 2024

oli-obk Jun 12, 2024

		@@ -0,0 +1,15 @@
		//@ compile-flags: -Zpolymorphize=on -Zinline-mir=yes
		//@ known-bug: #12345

	"test no longer crashes/triggers ICE! Please give it a mearningful name, \
	"test no longer crashes/triggers ICE! Please give it a meaningful name, \

compiletest ice tracking #122997

compiletest ice tracking #122997

Conversation

matthiaskrgr commented Mar 24, 2024 • edited Loading

rustbot commented Mar 24, 2024

This comment has been minimized.

This comment has been minimized.

compiler-errors left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fmease commented Mar 24, 2024 • edited Loading

matthiaskrgr commented Mar 24, 2024

This comment has been minimized.

matthiaskrgr commented Mar 24, 2024

matthiaskrgr commented Mar 24, 2024

Choose a reason for hiding this comment

rustbot commented Mar 29, 2024

matthiaskrgr commented Mar 29, 2024

This comment has been minimized.

bors commented Apr 14, 2024

matthiaskrgr commented Apr 14, 2024

bors commented Apr 14, 2024

bors commented Apr 14, 2024

rust-log-analyzer commented Apr 15, 2024

bors commented Apr 15, 2024

matthiaskrgr commented Apr 15, 2024

bors commented Apr 15, 2024

bors commented Apr 15, 2024

rust-timer commented Apr 15, 2024

Overall result: no relevant changes - no action needed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthiaskrgr commented Mar 24, 2024 •

edited

Loading

fmease commented Mar 24, 2024 •

edited

Loading