Include both benchmarks and tests in the numbers given to TeFiltered{,Out}

[thomcc]

Fixes #103794

#[bench] is broken on nightly without this, sadly. It apparently has no test coverage. In addition to manually testing, I've added a run-make smokecheck for this (which would have caught the issue), but it would be nice to have a better way to test, err, libtest. For now we should get this in ASAP IMO

[rustbot]

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

[rustbot]

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

• Stabilizing library features
• Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
• Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
• Changing public documentation in ways that create new stability guarantees
• Changing observable runtime behavior of library APIs

[Mark-Simulacrum]

@bors r+ p=5

[bors]

📌 Commit 8c71820 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

[Dylan-DPC]

@bors p=10

[bors]

⌛ Testing commit 8c71820 with merge 3882ba490d61277e7ee92301eb86df397ee79c44...

[bors]

💔 Test failed - checks-actions

[Dylan-DPC]

Closing tree till this is merged
@bors tree-closed=1000
@bors p=1000

[Dylan-DPC]

@bors treeclosed=1000

[Mark-Simulacrum]

@bors treeclosed-

[Dylan-DPC]

@bors treeclosed-

[thomcc]

Unsure why this failed -- seems to pass locally. Guessing this either needs to be run-make-fulldeps (I usually avoid the compiler tests suites), or perhaps just to exclude those platforms? Pushed up a patch moving it to run-make-fulldeps.

[thomcc]

LMK if you want me to split the test into a different PR so that you can land it without waiting for me to figure out how to where to put the test for this.

[Mark-Simulacrum]

It failed for a cross-compile case, I think if you add # ignore-cross-compile to the test it will work. I'm not sure fulldeps will have the same effect.

r=me with that done

[ehuss]

For the test, if you want to keep it in run-make, then it needs something like ignore-cross-compile so you don't have to deal with all the various targets that need extra setup, or "none" targets which can't generate executables.

EDIT: oops, mark beat me to it 😛

[thomcc]

Oh, yeah, thanks; this tries to run the output, so indeed cross compile doesn't make sense. I had tried fulldeps since most of the other Makefiles that have $(RUSTC) --test were in there, and github wouldn't let me download the failure log for some reason (and I wanted to do something rather than block all merges into rust-lang/rust... 😅)

I've moved it back and added # ignore-cross-compile (I also waited for the tests to pass locally).

r=me with that done

@bors r=Mark-Simulacrum

[bors]

📌 Commit bf88755 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

[bors]

⌛ Testing commit bf88755 with merge 95a3a72...

[bors]

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 95a3a72 to master...

[rust-timer]

Finished benchmarking commit (95a3a72): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.3%	[1.2%, 1.4%]	2
Regressions ❌ (secondary)	3.6%	[3.2%, 4.1%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.3%	[1.2%, 1.4%]	2

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

[thomcc]

Seems pretty likely to be noise.

[lqd]

Indeed, these 2 benchmarks are currently noisy.

@rustbot label: +perf-regression-triaged