Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add micro benchmark by JMH #1278

Merged
merged 8 commits into from
Jan 23, 2023

Conversation

dai-chen
Copy link
Collaborator

@dai-chen dai-chen commented Jan 12, 2023

Description

This PR is to add a new benchmarks module for ad-hoc microbenchmark on class level. This was motivated by discussion in #1196 (comment). For now we just use this for local ad-hoc benchmark test before submitting PR. We can explore https://github.com/marketplace/actions/continuous-benchmark for automation and performance degradation detect later.

Test

I added 1 benchmark class ComparisonOperatorBenchmark as an example.

$ ./gradlew :benchmark:jmh

...

Benchmark                                        (testDataType)  Mode  Cnt  Score    Error  Units
ComparisonOperatorBenchmark.testEqualOperator               int  avgt    3  0.005 ±  0.008  ms/op
ComparisonOperatorBenchmark.testEqualOperator            string  avgt    3  0.009 ±  0.004  ms/op
ComparisonOperatorBenchmark.testEqualOperator              date  avgt    3  0.001 ±  0.001  ms/op
ComparisonOperatorBenchmark.testGreaterOperator             int  avgt    3  0.005 ±  0.010  ms/op
ComparisonOperatorBenchmark.testGreaterOperator          string  avgt    3  0.010 ±  0.003  ms/op
ComparisonOperatorBenchmark.testGreaterOperator            date  avgt    3  0.001 ±  0.001  ms/op
ComparisonOperatorBenchmark.testLessOperator                int  avgt    3  0.004 ±  0.002  ms/op
ComparisonOperatorBenchmark.testLessOperator             string  avgt    3  0.010 ±  0.003  ms/op
ComparisonOperatorBenchmark.testLessOperator               date  avgt    3  0.001 ±  0.001  ms/op

Documentation

Please find more details in https://github.com/dai-chen/sql-1/blob/test-mircro-benchmark/benchmarks/README.md. Reference: https://github.com/opensearch-project/OpenSearch/blob/main/benchmarks/README.md

Issues Resolved

N/A

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest [except BWC]
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen added the infrastructure Changes to infrastructure, testing, CI/CD, pipelines, etc. label Jan 12, 2023
@dai-chen dai-chen self-assigned this Jan 12, 2023
@codecov-commenter
Copy link

codecov-commenter commented Jan 12, 2023

Codecov Report

Merging #1278 (b0caa1e) into main (6a90fe6) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main    #1278   +/-   ##
=========================================
  Coverage     98.34%   98.34%           
  Complexity     3600     3600           
=========================================
  Files           343      343           
  Lines          8908     8908           
  Branches        567      567           
=========================================
  Hits           8761     8761           
  Misses          142      142           
  Partials          5        5           
Flag Coverage Δ
sql-engine 98.34% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen marked this pull request as ready for review January 13, 2023 20:17
@dai-chen dai-chen requested a review from a team as a code owner January 13, 2023 20:17
benchmarks/build.gradle Outdated Show resolved Hide resolved
@dai-chen dai-chen merged commit ff9ac16 into opensearch-project:main Jan 23, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 23, 2023
* Add benchmark module and test implicit cast

Signed-off-by: Chen Dai <daichen@amazon.com>

* Benchmark the second approach from Yury

Signed-off-by: Chen Dai <daichen@amazon.com>

* Refactor code to make benchmark module for future use

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add readme

Signed-off-by: Chen Dai <daichen@amazon.com>

* Move benchmark class to right source location

Signed-off-by: Chen Dai <daichen@amazon.com>

* Change depedency accessiblity from api to impl

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit ff9ac16)
@dai-chen dai-chen deleted the test-mircro-benchmark branch January 23, 2023 19:11
dai-chen added a commit that referenced this pull request Jan 23, 2023
* Add benchmark module and test implicit cast

Signed-off-by: Chen Dai <daichen@amazon.com>

* Benchmark the second approach from Yury

Signed-off-by: Chen Dai <daichen@amazon.com>

* Refactor code to make benchmark module for future use

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add readme

Signed-off-by: Chen Dai <daichen@amazon.com>

* Move benchmark class to right source location

Signed-off-by: Chen Dai <daichen@amazon.com>

* Change depedency accessiblity from api to impl

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit ff9ac16)

Co-authored-by: Chen Dai <daichen@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x infrastructure Changes to infrastructure, testing, CI/CD, pipelines, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants