[Perf] Linux/x64: 30 Regressions on 3/12/2024 7:10:27 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	e171360c844ea9c1f4f98e8de7567342deb53e85
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Buffers.Tests.SearchValuesCharTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IndexOfAny - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	112.38 ns	122.75 ns	1.09	0.28	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.SearchValuesCharTests*'

Payloads

Baseline
Compare

System.Buffers.Tests.SearchValuesCharTests.IndexOfAny(Values: "abcdefABCDEF0123456789Ü")

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	e171360c844ea9c1f4f98e8de7567342deb53e85
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
LessThanOrEqualBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.45 ns	5.97 ns	1.73	0.86	False
InequalityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.61 ns	2.10 ns	3.44	0.91	False
MultiplyOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	19.70 ns	24.46 ns	1.24	0.72	False
SquareRootBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	49.93 ns	60.11 ns	1.20	0.43	False
NegateBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.60 ns	5.59 ns	1.55	0.87	False
LessThanOrEqualAllBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.00 ns	3.42 ns	∞	0.93	False
BitwiseOrOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.02 ns	8.74 ns	2.89	0.84	False
AbsBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.00 ns	6.21 ns	∞	0.92	False
DivisionOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	53.76 ns	63.99 ns	1.19	0.23	False
MultiplyBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	19.79 ns	28.22 ns	1.43	0.71	False
GreaterThanAnyBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.00 ns	4.18 ns	∞	0.90	False
LessThanAllBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.00 ns	1.73 ns	∞	0.91	False
BitwiseAndOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.75 ns	6.54 ns	1.74	0.85	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<SByte>*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.InequalityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.SquareRootBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.NegateBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualAllBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.BitwiseOrOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.AbsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.DivisionOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanAnyBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanAllBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<SByte>.BitwiseAndOperatorBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	e171360c844ea9c1f4f98e8de7567342deb53e85
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Add_Scalar - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.87 μs	2.54 μs	1.35	0.19	True
Divide_Scalar - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.00 μs	2.57 μs	1.29	0.12	True
Add_Scalar - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	35.34 μs	53.46 μs	1.51	0.15	True
SumOfSquares - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	84.11 ns	90.13 ns	1.07	0.38	False
Divide_Scalar - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	36.68 μs	55.13 μs	1.50	0.18	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>*'

Payloads

Baseline
Compare

System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.Add_Scalar(BufferLength: 128)

ETL Files

Histogram

JIT Disasms

System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.Divide_Scalar(BufferLength: 128)

ETL Files

Histogram

JIT Disasms

System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.Add_Scalar(BufferLength: 3079)

ETL Files

Histogram

JIT Disasms

System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.SumOfSquares(BufferLength: 128)

ETL Files

Histogram

JIT Disasms

System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.Divide_Scalar(BufferLength: 3079)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	e171360c844ea9c1f4f98e8de7567342deb53e85
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
LastIndexOfAnyValues - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.82 ns	14.00 ns	1.29	0.73	False
SequenceCompareTo - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	96.93 ns	106.56 ns	1.10	0.18	False
IndexOfAnyThreeValues - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	12.76 ns	15.66 ns	1.23	0.67	False
IndexOfValue - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	13.50 ns	15.24 ns	1.13	0.76	False
Reverse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	70.31 ns	76.02 ns	1.08	0.39	False
LastIndexOfAnyValues - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	17.09 ns	19.02 ns	1.11	0.79	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'

Payloads

Baseline
Compare

System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 4)

ETL Files

Histogram

JIT Disasms

System.Memory.Span<Char>.SequenceCompareTo(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 4)

ETL Files

Histogram

JIT Disasms

System.Memory.Span<Char>.IndexOfValue(Size: 33)

ETL Files

Histogram

JIT Disasms

System.Memory.Span<Char>.Reverse(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 33)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	e171360c844ea9c1f4f98e8de7567342deb53e85
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ConcurrentQueue - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	12.06 μs	13.33 μs	1.11	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<String>*'

Payloads

Baseline
Compare

System.Collections.CreateAddAndClear<String>.ConcurrentQueue(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	e171360c844ea9c1f4f98e8de7567342deb53e85
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.IO.Tests.Perf_File

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
WriteAllBytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.17 μs	11.11 μs	1.09	0.36	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_File.WriteAllBytes(size: 4096)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	9c828a17c347118968d7a1a170178daf4a9ce126
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryFormat - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	76.48 ns	84.36 ns	1.10	0.40	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	9c828a17c347118968d7a1a170178daf4a9ce126
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	297.17 ns	328.02 ns	1.10	0.35	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: HelloWorld)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	e171360c844ea9c1f4f98e8de7567342deb53e85
Compare	342bc7ecacec70dc0e50efd34b890110895a5ce2
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Array

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IndexOfChar - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	42.12 ns	45.52 ns	1.08	0.69	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

System.Tests.Perf_Array.IndexOfChar

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

@kg at least the tensor primitives seem to point at dotnet/runtime#99550 but I don't have a good theory as to why

[lewing]

one range is dotnet/runtime@27214a0...342bc7e

[kg]

Isn't this aot?

On March 19, 2024 2:01:23 PM PDT, Larry Ewing ***@***.***> wrote: @kg at least the tensor primitives seem to point at dotnet/runtime#99550 but I don't have a good theory as to why -- Reply to this email directly or view it on GitHub: #31479 (comment) You are receiving this because you were mentioned Message ID: ***@***.***>

- kg (mobile)