Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perf] Regression in Span Indexing #54172

Closed
DrewScoggins opened this issue Jun 14, 2021 · 3 comments
Closed

[Perf] Regression in Span Indexing #54172

DrewScoggins opened this issue Jun 14, 2021 · 3 comments
Labels
arch-x64 area-System.Memory os-linux Linux OS (any supported distro) os-windows tenet-performance Performance related issue
Milestone

Comments

@DrewScoggins
Copy link
Member

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c3126d0b0e3038c1bcb9a6ec4200841be3de03ba
Compare e0671e7ec6696449b61d61d19d3dd802b49a4fb3
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 42.03 ns 620.97 ns 14.78 0.51

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512)


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline c3126d0b0e3038c1bcb9a6ec4200841be3de03ba
Compare e0671e7ec6696449b61d61d19d3dd802b49a4fb3
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 286.54 ns 623.94 ns 2.18 0.01
BinarySearch - Duration of single invocation 15.75 ns 21.30 ns 1.35 0.06

graph
graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512)


System.Memory.Span<Int32>.BinarySearch(Size: 512)


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@DrewScoggins DrewScoggins added os-linux Linux OS (any supported distro) tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark arch-x64 labels Jun 14, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added area-System.Memory untriaged New issue has not been triaged by the area owner labels Jun 14, 2021
@ghost
Copy link

ghost commented Jun 14, 2021

Tagging subscribers to this area: @GrabYourPitchforks, @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c3126d0b0e3038c1bcb9a6ec4200841be3de03ba
Compare e0671e7ec6696449b61d61d19d3dd802b49a4fb3
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 42.03 ns 620.97 ns 14.78 0.51

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512)


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline c3126d0b0e3038c1bcb9a6ec4200841be3de03ba
Compare e0671e7ec6696449b61d61d19d3dd802b49a4fb3
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 286.54 ns 623.94 ns 2.18 0.01
BinarySearch - Duration of single invocation 15.75 ns 21.30 ns 1.35 0.06

graph
graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512)


System.Memory.Span<Int32>.BinarySearch(Size: 512)


Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: DrewScoggins
Assignees: -
Labels:

arch-x64, area-System.Memory, os-linux, tenet-performance, tenet-performance-benchmarks, untriaged

Milestone: -

@GrabYourPitchforks
Copy link
Member

Regression in IndexOfAnyFourValues is expected and is a consequence of #53652.

I don't see any recent changes to BinarySearch's source code. Perhaps codegen changed recently?

@DrewScoggins
Copy link
Member Author

Looking at the diff, for BinarySearch it happens over a PGO update, so my guess is that is the most likely cause. @AndyAyersMS already has an issue open for that, so I will close this one as the other issue is by design.

@adamsitnik adamsitnik removed tenet-performance-benchmarks Issue from performance benchmark untriaged New issue has not been triaged by the area owner labels Jun 15, 2021
@adamsitnik adamsitnik added this to the 6.0.0 milestone Jun 15, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Jul 15, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-x64 area-System.Memory os-linux Linux OS (any supported distro) os-windows tenet-performance Performance related issue
Projects
None yet
Development

No branches or pull requests

3 participants