add microbenchmark for exchange and window function #5137

guo-shaoge · 2022-06-13T09:28:35Z

Signed-off-by: guo-shaoge shaoge1994@163.com

What problem does this PR solve?

Issue Number: close #5138, close #4276

Problem Summary:

To help compare performance of fine grained shuffle, add exchange and window function microbenchmark.
support fine grained shuffle for window function #5048 will use this benchmark as baseline.

What is changed and how it works?

bench_exchange.cpp
3.1. sender: unionBlockInputStream <- ExchangeSenderBlockInputStream(x concurrency)<- MockFixedRowBlockInputStream(x concurrency)
2.2. receiver: unionBlockInputStream <- ExchangeReceiverBlockInputStream(x concurrency)<- ExchangeReceiver
bench_window.cpp:
3.1. sender: unionBlockInputStream <- WindowBlockInputStream <- MergeSortingBlockInputStream(x concurrency) <- PartialSortInputStream(x concurrency) <- ExchangeSenderBlockInputStream(x concurrency)<- MockFixedRowBlockInputStream(x concurrency)
3.2. receiver: unionBlockInputStream <- ExchangeReceiverBlockInputStream(x concurrency)<- ExchangeReceiver
exchange_perftest.cpp: move code to bench_exchange.cpp

Usage:

cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_TESTS=ON && make bench_dbms -j15
cd build && ./dbms/bench_dbms --benchmark_filter='WindowFunctionBench.*'
cd build && ./dbms/bench_dbms --benchmark_filter='ExchangeBench.*'

Result:

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

Signed-off-by: guo-shaoge <shaoge1994@163.com>

ti-chi-bot · 2022-06-13T09:28:36Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

SeaRise
ywqzzy

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Signed-off-by: guo-shaoge <shaoge1994@163.com>

ywqzzy · 2022-06-13T10:05:27Z

This pr is also related to #4276

dbms/src/Flash/tests/bench_window.cpp

Signed-off-by: guo-shaoge <shaoge1994@163.com>

guo-shaoge · 2022-06-14T09:26:01Z

/run-all-tests

Signed-off-by: guo-shaoge <shaoge1994@163.com>

sre-bot · 2022-06-14T09:59:22Z

Coverage for changed files

Filename                                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
dbms/src/Debug/astToExecutor.h                                 36                15    58.33%          25                10    60.00%          53                21    60.38%          10                 6    40.00%
dbms/src/Flash/Coprocessor/DAGExpressionAnalyzer.h              2                 1    50.00%           2                 1    50.00%           2                 1    50.00%           0                 0         -
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.h           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
dbms/src/TestUtils/mockExecutor.cpp                            79                15    81.01%          49                12    75.51%         307                63    79.48%          36                 3    91.67%
dbms/src/TestUtils/mockExecutor.h                               6                 1    83.33%           6                 1    83.33%          15                 3    80.00%           0                 0         -
libs/libcommon/include/common/types.h                          23                 8    65.22%           8                 4    50.00%          24                12    50.00%          10                 3    70.00%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                         148                40    72.97%          92                28    69.57%         403               100    75.19%          56                12    78.57%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18322      9640             47.39%    205718  96571        53.06%

full coverage report (for internal network access only)

guo-shaoge · 2022-06-15T07:37:41Z

/merge

ti-chi-bot · 2022-06-15T07:37:42Z

@guo-shaoge: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-06-15T07:37:43Z

This pull request has been accepted and is ready to merge.

Commit hash: c77aed1

ti-chi-bot · 2022-06-15T07:37:57Z

@guo-shaoge: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

sre-bot · 2022-06-15T08:20:12Z

Coverage for changed files

Filename                                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
dbms/src/Debug/astToExecutor.h                                 33                12    63.64%          22                 7    68.18%          50                18    64.00%          10                 6    40.00%
dbms/src/Flash/Coprocessor/DAGExpressionAnalyzer.h              2                 1    50.00%           2                 1    50.00%           2                 1    50.00%           0                 0         -
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.h           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
dbms/src/TestUtils/mockExecutor.cpp                            79                15    81.01%          49                12    75.51%         307                63    79.48%          36                 3    91.67%
dbms/src/TestUtils/mockExecutor.h                               6                 1    83.33%           6                 1    83.33%          15                 3    80.00%           0                 0         -
libs/libcommon/include/common/types.h                          23                 8    65.22%           8                 4    50.00%          24                12    50.00%          10                 3    70.00%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                         145                37    74.48%          89                25    71.91%         400                97    75.75%          56                12    78.57%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18327      9653             47.33%    205795  96635        53.04%

full coverage report (for internal network access only)

sre-bot · 2022-06-15T08:20:37Z

Coverage for changed files

Filename                                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
dbms/src/Debug/astToExecutor.h                                 33                12    63.64%          22                 7    68.18%          50                18    64.00%          10                 6    40.00%
dbms/src/Flash/Coprocessor/DAGExpressionAnalyzer.h              2                 1    50.00%           2                 1    50.00%           2                 1    50.00%           0                 0         -
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.h           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
dbms/src/TestUtils/mockExecutor.cpp                            79                15    81.01%          49                12    75.51%         307                63    79.48%          36                 3    91.67%
dbms/src/TestUtils/mockExecutor.h                               6                 1    83.33%           6                 1    83.33%          15                 3    80.00%           0                 0         -
libs/libcommon/include/common/types.h                          23                 8    65.22%           8                 4    50.00%          24                12    50.00%          10                 3    70.00%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                         145                37    74.48%          89                25    71.91%         400                97    75.75%          56                12    78.57%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18325      9653             47.32%    205783  96653        53.03%

full coverage report (for internal network access only)

guo-shaoge · 2022-06-15T09:01:20Z

/build

guo-shaoge · 2022-06-15T09:30:51Z

/run-all-tests

sre-bot · 2022-06-15T09:43:30Z

Coverage for changed files

Filename                                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
dbms/src/Debug/astToExecutor.h                                 33                12    63.64%          22                 7    68.18%          50                18    64.00%          10                 6    40.00%
dbms/src/Flash/Coprocessor/DAGExpressionAnalyzer.h              2                 1    50.00%           2                 1    50.00%           2                 1    50.00%           0                 0         -
dbms/src/Flash/Coprocessor/DAGQueryBlockInterpreter.h           2                 0   100.00%           2                 0   100.00%           2                 0   100.00%           0                 0         -
dbms/src/TestUtils/mockExecutor.cpp                            79                15    81.01%          49                12    75.51%         307                63    79.48%          36                 3    91.67%
dbms/src/TestUtils/mockExecutor.h                               6                 1    83.33%           6                 1    83.33%          15                 3    80.00%           0                 0         -
libs/libcommon/include/common/types.h                          23                 8    65.22%           8                 4    50.00%          24                12    50.00%          10                 3    70.00%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                         145                37    74.48%          89                25    71.91%         400                97    75.75%          56                12    78.57%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18327      9653             47.33%    205795  96660        53.03%

full coverage report (for internal network access only)

guo-shaoge · 2022-06-15T15:03:54Z

/build

guo-shaoge · 2022-06-16T03:03:11Z

/run-integration-test

add macro benchmark for exchange and window function

e13959f

Signed-off-by: guo-shaoge <shaoge1994@163.com>

ti-chi-bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed do-not-merge/needs-linked-issue labels Jun 13, 2022

remove some commented code

5963cf0

Signed-off-by: guo-shaoge <shaoge1994@163.com>

guo-shaoge mentioned this pull request Jun 13, 2022

support fine grained shuffle for window function #5048

Merged

12 tasks

fix fmt

faeba39

Signed-off-by: guo-shaoge <shaoge1994@163.com>

guo-shaoge changed the title ~~add macro benchmark for exchange and window function~~ add microbenchmark for exchange and window function Jun 13, 2022

ywqzzy self-requested a review June 13, 2022 10:10

SeaRise reviewed Jun 13, 2022

View reviewed changes

dbms/src/Flash/tests/bench_window.cpp Outdated Show resolved Hide resolved

guo-shaoge added 3 commits June 14, 2022 11:14

Merge branch 'master' of github.com:pingcap/tiflash into exchange_bench

bab376f

use DAGRequestBuilder to build tipb::Window instead of json_string

3c3e7be

Signed-off-by: guo-shaoge <shaoge1994@163.com>

use mockTable instead of fakeDataSource

760fb5c

Signed-off-by: guo-shaoge <shaoge1994@163.com>

guo-shaoge added 2 commits June 14, 2022 17:39

fix

798e034

Signed-off-by: guo-shaoge <shaoge1994@163.com>

fix

9251383

Signed-off-by: guo-shaoge <shaoge1994@163.com>

guo-shaoge requested a review from SeaRise June 14, 2022 09:47

Merge branch 'master' into exchange_bench

c77aed1

SeaRise approved these changes Jun 15, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 15, 2022

ywqzzy approved these changes Jun 15, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 15, 2022

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 15, 2022

Merge branch 'master' into exchange_bench

ed7d43e

ti-chi-bot merged commit 617fe54 into pingcap:master Jun 16, 2022

guo-shaoge mentioned this pull request Oct 13, 2023

fix bench_dbms build #8194

Merged

12 tasks

ti-chi-bot mentioned this pull request Nov 3, 2023

fix bench_dbms build (#8194) #8306

Closed

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add microbenchmark for exchange and window function #5137

add microbenchmark for exchange and window function #5137

guo-shaoge commented Jun 13, 2022 •

edited by JaySon-Huang

Loading

ti-chi-bot commented Jun 13, 2022 •

edited

Loading

ywqzzy commented Jun 13, 2022

guo-shaoge commented Jun 14, 2022

sre-bot commented Jun 14, 2022

guo-shaoge commented Jun 15, 2022

ti-chi-bot commented Jun 15, 2022

ti-chi-bot commented Jun 15, 2022

ti-chi-bot commented Jun 15, 2022

sre-bot commented Jun 15, 2022

sre-bot commented Jun 15, 2022

guo-shaoge commented Jun 15, 2022

guo-shaoge commented Jun 15, 2022

sre-bot commented Jun 15, 2022

guo-shaoge commented Jun 15, 2022

guo-shaoge commented Jun 16, 2022

add microbenchmark for exchange and window function #5137

add microbenchmark for exchange and window function #5137

Conversation

guo-shaoge commented Jun 13, 2022 • edited by JaySon-Huang Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Jun 13, 2022 • edited Loading

ywqzzy commented Jun 13, 2022

guo-shaoge commented Jun 14, 2022

sre-bot commented Jun 14, 2022

Coverage for changed files

Coverage summary

guo-shaoge commented Jun 15, 2022

ti-chi-bot commented Jun 15, 2022

ti-chi-bot commented Jun 15, 2022

ti-chi-bot commented Jun 15, 2022

sre-bot commented Jun 15, 2022

Coverage for changed files

Coverage summary

sre-bot commented Jun 15, 2022

Coverage for changed files

Coverage summary

guo-shaoge commented Jun 15, 2022

guo-shaoge commented Jun 15, 2022

sre-bot commented Jun 15, 2022

Coverage for changed files

Coverage summary

guo-shaoge commented Jun 15, 2022

guo-shaoge commented Jun 16, 2022

guo-shaoge commented Jun 13, 2022 •

edited by JaySon-Huang

Loading

ti-chi-bot commented Jun 13, 2022 •

edited

Loading