Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

infoschema,planner,execuor: support hot history region #27224

Closed
wants to merge 51 commits into from
Closed

infoschema,planner,execuor: support hot history region #27224

wants to merge 51 commits into from

Conversation

IcePigZDB
Copy link
Contributor

@IcePigZDB IcePigZDB commented Aug 15, 2021 •
edited
Loading

Signed-off-by: IcePigZDB icepigzdb@gmail.com

What problem does this PR solve?

This PR is oversize and splited into three small one:

  1. infoschema: add tidb_hot_regions_history virtual table #27373
  2. planner: add extractor for tidb_hot_regions_history #27374
  3. executor: add retriver for tidb_hot_regions_history  #27375
  4. planner: add test for HotRegionsHistoryTableExtractor #27899
  5. executor: add test for hotRegionsHistoryRetriver #27900

Proposal: pingcap/tidb#27487
Associated Issue: pingcap/tidb#25281
Associated PR in PD:
PRs splited from old oversize one: tikv/pd/pull/3989

Problem Summary: Create a new table TIDB_HOT_REGIONS_HISTORY in the INFORMATION_SCHEMAT database to retrieve history hotspot information stored by PD periodically.

What's Changed:

How it Works:

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note 

select * from information_schema.tidb_hot_regions_history where update_time>='2019-10-10 10:10:10' and update_time<'2019-10-11 10:10:10'

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 15, 2021
@CLAassistant
Copy link

CLAassistant commented Aug 15, 2021 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@ti-chi-bot ti-chi-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Aug 15, 2021
@ti-chi-bot
Copy link
Member

Welcome @IcePigZDB!

It looks like this is your first PR to pingcap/tidb 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!



Thank you, and welcome to pingcap/tidb. 😃

@github-actions github-actions bot added sig/execution SIG execution sig/sql-infra SIG: SQL Infra labels Aug 15, 2021
@IcePigZDB IcePigZDB force-pushed the hishotregionpr branch 2 times, most recently from f6ea401 to e6f2cf2 Compare August 15, 2021 07:57
@IcePigZDB IcePigZDB changed the title planner,execuor: support hot history region infoschema,planner,execuor: support hot history region Aug 15, 2021
@qidi1 qidi1 mentioned this pull request Aug 15, 2021
Closed
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 15, 2021
@ti-chi-bot ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 15, 2021
@ti-chi-bot ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 15, 2021
@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 15, 2021
@IcePigZDB
infoschema: add tidb_hot_regions_history virtual table
dd87813 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
planner: add extractor for tidb_hot_regions_history
fe67ab4 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
executor: add retriver for tidb_hot_regions_history
07b0cdd 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 15, 2021
qidi1
qidi1 reviewed Aug 16, 2021
View reviewed changes
executor/memtable_reader.go Outdated Show resolved Hide resolved
IcePigZDB
IcePigZDB commented Aug 16, 2021
View reviewed changes
Copy link
Contributor Author

@IcePigZDB IcePigZDB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self review.

executor/memtable_reader.go Outdated Show resolved Hide resolved
executor/memtable_reader.go Outdated Show resolved Hide resolved
executor/memtable_reader.go Outdated Show resolved Hide resolved
executor/memtable_reader.go Outdated Show resolved Hide resolved
executor/memtable_reader.go Show resolved Hide resolved
executor/memtable_reader.go Outdated Show resolved Hide resolved
executor/memtable_reader.go Outdated Show resolved Hide resolved
planner/core/memtable_predicate_extractor.go Outdated Show resolved Hide resolved
planner/core/memtable_predicate_extractor.go Outdated Show resolved Hide resolved
planner/core/memtable_predicate_extractor.go Outdated Show resolved Hide resolved
IcePigZDB and others added 9 commits August 25, 2021 20:27
@IcePigZDB
Merge branch 'master' into hishotregionpr
d3917b1
@IcePigZDB
planner: add removed cluster_log test case
9ed1f79 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
exectuor:change PD-Allow-follower-handle to false
a81eb94 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
Merge branch 'master' into hishotregionpr
6f2d228
@IcePigZDB
Merge branch 'master' into hishotregionpr
13c044b
@IcePigZDB
infoschema: add is_learner field
f7f7810 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
planner: add is_learner field
5092089 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
executor: add is_learner field
826693b 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
planner: clear logic in parseUint64
d81ccf8 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
This was referenced Sep 8, 2021
Closed
Closed
@IcePigZDB
planner: quantiles to uint64s and remove 3,4 in comment of IS_LEARNER
c881c52 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
This was referenced Sep 16, 2021
Merged
Merged
@IcePigZDB
planner: formate test
361e891 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
executor: formate retriver
f5e8925 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
executor: add init of cancel
6781bad 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
executor: formate
160e34b 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
IcePigZDB and others added 2 commits October 8, 2021 03:20
@IcePigZDB @nolouch
Update executor/memtable_reader.go
039220c 
Co-authored-by: ShuNing <nolouch@gmail.com>
@IcePigZDB
executor: simplfy code and move channel close to retrive func
6debfe2 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>

Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@IcePigZDB
executor: move close channel to startRetriving
c41c11c 
Signed-off-by: IcePigZDB <icepigzdb@gmail.com>
@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 17, 2021
@ti-chi-bot
Copy link
Member

@IcePigZDB: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@nolouch
Copy link
Member

nolouch commented Nov 22, 2021

I closed this PR. it is already merged with the sub PRs.

@nolouch nolouch closed this Nov 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qidi1 qidi1 qidi1 left review comments

@rleungx rleungx rleungx left review comments

Assignees
No one assigned
Labels
needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/execution SIG execution sig/sql-infra SIG: SQL Infra size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@IcePigZDB @ti-chi-bot @CLAassistant @nolouch @rleungx @qidi1