Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

make mpp_fail test stable #4300

Merged
merged 6 commits into from
Mar 16, 2022
Merged

make mpp_fail test stable #4300

merged 6 commits into from
Mar 16, 2022

Conversation

windtalker
Copy link
Contributor

@windtalker windtalker commented Mar 16, 2022 •
edited
Loading

Signed-off-by: xufei xufeixw@mail.ustc.edu.cn

What problem does this PR solve?

Issue Number: close #4299

Problem Summary:
As issue described.

What is changed and how it works?

Root cause:

after #4285, if exception happens in ParallelAggregatingBlockInputStream, all the threads in its processor will throw exception, and ParallelAggregatingBlockInputStream uses a std::vector<std::exception_ptr> to store these exceptions(indexed by the thread_index), and it always throw exception generated from thread 0, which make the throwed exception unstable.

Solution:

Always throw the first met exception in ParallelAggregatingBlockInputStream

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@windtalker
make mpp_fail test stable
8a939ba 
Signed-off-by: xufei <xufeixw@mail.ustc.edu.cn>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Mar 16, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • bestwoody
  • fuzhe1989

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed do-not-merge/needs-triage-completed labels Mar 16, 2022
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 16, 2022
bestwoody
bestwoody approved these changes Mar 16, 2022
View reviewed changes
Copy link
Contributor

@bestwoody bestwoody left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 16, 2022
@windtalker
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

@windtalker: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: c581984

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 16, 2022
@sre-bot
Copy link
Collaborator

sre-bot commented Mar 16, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                                      18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
DataStreams/ParallelAggregatingBlockInputStream.cpp          93                93     0.00%          12                12     0.00%         150               150     0.00%          64                64     0.00%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 6     0.00%           6                 6     0.00%          14                14     0.00%           0                 0         -
Flash/Coprocessor/DAGUtils.cpp                              290               233    19.66%          36                24    33.33%         505               404    20.00%         324               230    29.01%
Flash/EstablishCall.h                                         2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/Mpp/MPPTunnel.cpp                                     185               185     0.00%          16                16     0.00%         241               241     0.00%          94                94     0.00%
Flash/Mpp/MPPTunnel.h                                         9                 9     0.00%           9                 9     0.00%          18                18     0.00%           0                 0         -
Functions/FunctionsDateTime.cpp                              27                19    29.63%           4                 2    50.00%          99                25    74.75%          16                13    18.75%
Functions/FunctionsDateTime.h                              1073               638    40.54%         322               209    35.09%        2018              1226    39.25%         534               340    36.33%
Functions/tests/gtest_dayofweekyear.cpp                     258                74    71.32%           2                 0   100.00%         135                 0   100.00%          80                42    47.50%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      1961              1259    35.80%         420               280    33.33%        3229              2081    35.55%        1120               783    30.09%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16930      9506             43.85%    190847  96512        49.43%

full coverage report (for internal network access only)

@JaySon-Huang
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 16, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                                      18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
DataStreams/ParallelAggregatingBlockInputStream.cpp          93                93     0.00%          12                12     0.00%         150               150     0.00%          64                64     0.00%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 6     0.00%           6                 6     0.00%          14                14     0.00%           0                 0         -
Flash/Coprocessor/DAGUtils.cpp                              290               233    19.66%          36                24    33.33%         505               404    20.00%         324               230    29.01%
Flash/EstablishCall.h                                         2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/Mpp/MPPTunnel.cpp                                     185               185     0.00%          16                16     0.00%         241               241     0.00%          94                94     0.00%
Flash/Mpp/MPPTunnel.h                                         9                 9     0.00%           9                 9     0.00%          18                18     0.00%           0                 0         -
Functions/FunctionsDateTime.cpp                              27                19    29.63%           4                 2    50.00%          99                25    74.75%          16                13    18.75%
Functions/FunctionsDateTime.h                              1073               638    40.54%         322               209    35.09%        2018              1226    39.25%         534               340    36.33%
Functions/tests/gtest_dayofweekyear.cpp                     258                74    71.32%           2                 0   100.00%         135                 0   100.00%          80                42    47.50%
Storages/Page/V3/WAL/WALReader.cpp                           72                10    86.11%           9                 0   100.00%         176                15    91.48%          54                14    74.07%
Storages/Page/V3/WALStore.cpp                                74                39    47.30%           8                 1    87.50%         134                42    68.66%          36                19    47.22%
Storages/Page/V3/WALStore.h                                   1                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
Storages/Page/V3/tests/gtest_wal_store.cpp                 1948               260    86.65%          12                 0   100.00%         465                13    97.20%         646               314    51.39%
Storages/tests/TiFlashStorageTestBasic.h                     27                 5    81.48%           6                 0   100.00%          51                12    76.47%          18                 9    50.00%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      4083              1573    61.47%         456               281    38.38%        4058              2163    46.70%        1874              1139    39.22%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16931      9505             43.86%    190879  96507        49.44%

full coverage report (for internal network access only)

@JaySon-Huang
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 16, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                                      18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
DataStreams/ParallelAggregatingBlockInputStream.cpp          93                93     0.00%          12                12     0.00%         150               150     0.00%          64                64     0.00%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 6     0.00%           6                 6     0.00%          14                14     0.00%           0                 0         -
Flash/Coprocessor/DAGUtils.cpp                              290               233    19.66%          36                24    33.33%         505               404    20.00%         324               230    29.01%
Flash/EstablishCall.h                                         2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/Mpp/MPPTunnel.cpp                                     185               185     0.00%          16                16     0.00%         241               241     0.00%          94                94     0.00%
Flash/Mpp/MPPTunnel.h                                         9                 9     0.00%           9                 9     0.00%          18                18     0.00%           0                 0         -
Functions/FunctionsDateTime.cpp                              27                19    29.63%           4                 2    50.00%          99                25    74.75%          16                13    18.75%
Functions/FunctionsDateTime.h                              1073               638    40.54%         322               209    35.09%        2018              1226    39.25%         534               340    36.33%
Functions/tests/gtest_dayofweekyear.cpp                     258                74    71.32%           2                 0   100.00%         135                 0   100.00%          80                42    47.50%
Storages/Page/V3/WAL/WALReader.cpp                           72                10    86.11%           9                 0   100.00%         176                15    91.48%          54                14    74.07%
Storages/Page/V3/WALStore.cpp                                74                39    47.30%           8                 1    87.50%         134                42    68.66%          36                19    47.22%
Storages/Page/V3/WALStore.h                                   1                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
Storages/Page/V3/tests/gtest_wal_store.cpp                 1948               260    86.65%          12                 0   100.00%         465                13    97.20%         646               314    51.39%
Storages/tests/TiFlashStorageTestBasic.h                     27                 5    81.48%           6                 0   100.00%          51                12    76.47%          18                 9    50.00%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      4083              1573    61.47%         456               281    38.38%        4058              2163    46.70%        1874              1139    39.22%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16931      9505             43.86%    190879  96500        49.44%

full coverage report (for internal network access only)

@JaySon-Huang
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@bestwoody
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

@bestwoody: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 16, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                                      18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
DataStreams/HashJoinProbeBlockInputStream.cpp                15                15     0.00%           4                 4     0.00%          28                28     0.00%          10                10     0.00%
DataStreams/HashJoinProbeBlockInputStream.h                   1                 1     0.00%           1                 1     0.00%           1                 1     0.00%           0                 0         -
DataStreams/ParallelAggregatingBlockInputStream.cpp          93                93     0.00%          12                12     0.00%         150               150     0.00%          64                64     0.00%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 6     0.00%           6                 6     0.00%          14                14     0.00%           0                 0         -
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp              390               390     0.00%          45                45     0.00%         875               875     0.00%         312               312     0.00%
Flash/Coprocessor/DAGUtils.cpp                              290               233    19.66%          36                24    33.33%         505               404    20.00%         324               230    29.01%
Flash/EstablishCall.h                                         2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/Mpp/MPPTunnel.cpp                                     185               185     0.00%          16                16     0.00%         241               241     0.00%          94                94     0.00%
Flash/Mpp/MPPTunnel.h                                         9                 9     0.00%           9                 9     0.00%          18                18     0.00%           0                 0         -
Functions/FunctionsDateTime.cpp                              27                19    29.63%           4                 2    50.00%          99                25    74.75%          16                13    18.75%
Functions/FunctionsDateTime.h                              1073               638    40.54%         322               209    35.09%        2018              1226    39.25%         534               340    36.33%
Functions/tests/gtest_dayofweekyear.cpp                     258                74    71.32%           2                 0   100.00%         135                 0   100.00%          80                42    47.50%
Storages/Page/V3/WAL/WALReader.cpp                           72                10    86.11%           9                 0   100.00%         176                15    91.48%          54                14    74.07%
Storages/Page/V3/WALStore.cpp                                74                39    47.30%           8                 1    87.50%         134                42    68.66%          36                19    47.22%
Storages/Page/V3/WALStore.h                                   1                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
Storages/Page/V3/tests/gtest_wal_store.cpp                 1948               260    86.65%          12                 0   100.00%         465                13    97.20%         646               314    51.39%
Storages/tests/TiFlashStorageTestBasic.h                     27                 5    81.48%           6                 0   100.00%          51                12    76.47%          18                 9    50.00%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      4489              1979    55.91%         506               331    34.58%        4962              3067    38.19%        2196              1461    33.47%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16936      9510             43.85%    190908  96536        49.43%

full coverage report (for internal network access only)

@JaySon-Huang
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 16, 2022

Coverage for changed files

Filename                                                Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/TiFlashMetrics.h                                      18                 0   100.00%          11                 0   100.00%          47                 1    97.87%           8                 0   100.00%
DataStreams/HashJoinProbeBlockInputStream.cpp                15                15     0.00%           4                 4     0.00%          28                28     0.00%          10                10     0.00%
DataStreams/HashJoinProbeBlockInputStream.h                   1                 1     0.00%           1                 1     0.00%           1                 1     0.00%           0                 0         -
DataStreams/ParallelAggregatingBlockInputStream.cpp          93                93     0.00%          12                12     0.00%         150               150     0.00%          64                64     0.00%
DataStreams/ParallelAggregatingBlockInputStream.h             6                 6     0.00%           6                 6     0.00%          14                14     0.00%           0                 0         -
Flash/Coprocessor/DAGQueryBlockInterpreter.cpp              390               390     0.00%          45                45     0.00%         875               875     0.00%         312               312     0.00%
Flash/Coprocessor/DAGUtils.cpp                              290               233    19.66%          36                24    33.33%         505               404    20.00%         324               230    29.01%
Flash/EstablishCall.h                                         2                 2     0.00%           2                 2     0.00%           2                 2     0.00%           0                 0         -
Flash/Mpp/MPPTunnel.cpp                                     185               185     0.00%          16                16     0.00%         241               241     0.00%          94                94     0.00%
Flash/Mpp/MPPTunnel.h                                         9                 9     0.00%           9                 9     0.00%          18                18     0.00%           0                 0         -
Functions/FunctionsDateTime.cpp                              27                19    29.63%           4                 2    50.00%          99                25    74.75%          16                13    18.75%
Functions/FunctionsDateTime.h                              1073               638    40.54%         322               209    35.09%        2018              1226    39.25%         534               340    36.33%
Functions/tests/gtest_dayofweekyear.cpp                     258                74    71.32%           2                 0   100.00%         135                 0   100.00%          80                42    47.50%
Storages/Page/V3/WAL/WALReader.cpp                           72                10    86.11%           9                 0   100.00%         176                15    91.48%          54                14    74.07%
Storages/Page/V3/WALStore.cpp                                74                39    47.30%           8                 1    87.50%         134                42    68.66%          36                19    47.22%
Storages/Page/V3/WALStore.h                                   1                 0   100.00%           1                 0   100.00%           3                 0   100.00%           0                 0         -
Storages/Page/V3/tests/gtest_wal_store.cpp                 1948               260    86.65%          12                 0   100.00%         465                13    97.20%         646               314    51.39%
Storages/tests/TiFlashStorageTestBasic.h                     27                 5    81.48%           6                 0   100.00%          51                12    76.47%          18                 9    50.00%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                      4489              1979    55.91%         506               331    34.58%        4962              3067    38.19%        2196              1461    33.47%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16936      9510             43.85%    190908  96531        49.44%

full coverage report (for internal network access only)

@ti-chi-bot ti-chi-bot merged commit 10536de into pingcap:master Mar 16, 2022
@JaySon-Huang JaySon-Huang deleted the make_mpp_fail_test_stable branch March 16, 2022 13:14
JaySon-Huang pushed a commit to JaySon-Huang/tiflash that referenced this pull request Mar 17, 2022
@windtalker @JaySon-Huang
make mpp_fail test stable (pingcap#4300)
c59074c 
close pingcap#4299
windtalker added a commit to ti-chi-bot/tiflash that referenced this pull request Jun 15, 2022
@windtalker
make mpp_fail test stable (pingcap#4300)
e471acf 
close pingcap#4299
windtalker added a commit to ti-chi-bot/tiflash that referenced this pull request Jun 15, 2022
@windtalker
make mpp_fail test stable (pingcap#4300)
90dcd38 
close pingcap#4299
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bestwoody bestwoody bestwoody approved these changes

@fuzhe1989 fuzhe1989 fuzhe1989 approved these changes

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

mpp_fail test unstable
6 participants
@windtalker @ti-chi-bot @sre-bot @JaySon-Huang @bestwoody @fuzhe1989