Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests #60001

Closed
performanceautofiler bot opened this issue Oct 5, 2021 · 3 comments
Closed
Labels
arch-x64 os-windows tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark untriaged New issue has not been triaged by the area owner

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 918aaec1d94cb5f80e4ed00493fbc2940c7fa132
Compare ff135fe769534cf2d9fbd87bffcf65a16ab44c4a
Diff Diff

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 191.26 ns 210.84 ns 1.10 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 918aaec1d94cb5f80e4ed00493fbc2940c7fa132
Compare ff135fe769534cf2d9fbd87bffcf65a16ab44c4a
Diff Diff

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MeasureReverseUsingNtoH - Duration of single invocation 454.26 ns 561.26 ns 1.24 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.MeasureReverseUsingNtoH


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 918aaec1d94cb5f80e4ed00493fbc2940c7fa132
Compare ff135fe769534cf2d9fbd87bffcf65a16ab44c4a
Diff Diff

Regressions in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NoArguments_FilteredByLevel - Duration of single invocation 18.50 ns 22.27 ns 1.20 0.13 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 918aaec1d94cb5f80e4ed00493fbc2940c7fa132
Compare ff135fe769534cf2d9fbd87bffcf65a16ab44c4a
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToLowerInvariant - Duration of single invocation 13.51 ns 16.71 ns 1.24 0.10 False
Replace_String - Duration of single invocation 53.21 ns 59.74 ns 1.12 0.12 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToLowerInvariant(s: "TEST")


System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 918aaec1d94cb5f80e4ed00493fbc2940c7fa132
Compare ff135fe769534cf2d9fbd87bffcf65a16ab44c4a
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 122.02 ns 143.50 ns 1.18 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine)


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 918aaec1d94cb5f80e4ed00493fbc2940c7fa132
Compare ff135fe769534cf2d9fbd87bffcf65a16ab44c4a
Diff Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor_Span - Duration of single invocation 17.44 ns 18.91 ns 1.08 0.14 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo EgorBo changed the title [Perf] Changes at 9/30/2021 11:27:06 PM Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests Oct 5, 2021
@EgorBo EgorBo transferred this issue from dotnet/perf-autofiling-issues Oct 5, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Oct 5, 2021
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@EgorBo
Copy link
Member

EgorBo commented Oct 5, 2021

BinaryReadAndWriteTests and a couple of other benchmarks likely regressed by #53450 ? cc @tannergooding

@EgorBo EgorBo added arch-x64 os-windows tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark labels Oct 5, 2021
@tannergooding
Copy link
Member

I checked a couple of these methods and there is no asm diff here. The regressions shown are due to some external factor, such as TC not completing in time where it was before or some other noise on the machine.

@ghost ghost locked as resolved and limited conversation to collaborators Nov 4, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-x64 os-windows tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark untriaged New issue has not been triaged by the area owner
Projects
None yet
Development

No branches or pull requests

2 participants