Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statistics: record last gc ts to avoid huge read on stats table #46138

Merged
merged 9 commits into from
Sep 11, 2023

Conversation

winoros
Copy link
Member

@winoros winoros commented Aug 16, 2023

What problem does this PR solve?

Issue Number: close #45966, close #31778

Problem Summary:

The previous GCStats will read nearly all the stats into the memory

What is changed and how it works?

This pull tries to limit its influence.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@winoros winoros requested a review from a team as a code owner August 16, 2023 05:17
@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 16, 2023
@codecov
Copy link

codecov bot commented Aug 16, 2023

Codecov Report

Attention: Patch coverage is 60.52632% with 30 lines in your changes missing coverage. Please review.

Project coverage is 72.7547%. Comparing base (789d6d0) to head (ff81270).
Report is 3406 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #46138        +/-   ##
================================================
- Coverage   73.3618%   72.7547%   -0.6071%     
================================================
  Files          1326       1347        +21     
  Lines        396702     403058      +6356     
================================================
+ Hits         291028     293244      +2216     
- Misses        87118      91210      +4092     
- Partials      18556      18604        +48     
Flag Coverage Δ
integration 27.8207% <2.6315%> (?)
unit 73.3748% <60.5263%> (+0.0129%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 54.0444% <ø> (ø)
parser 84.9911% <ø> (ø)
br 48.3897% <ø> (-4.4065%) ⬇️

@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 4, 2023
@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Sep 5, 2023
Copy link
Contributor

@AilinKid AilinKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

h.recordHistoricalStatsMeta(physicalID, statsVer, StatsMetaHistorySourceSchemaChange)
}
}()
h.mu.Lock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use the h.session.pool to avoid the lock the handle?

session/bootstrap.go Outdated Show resolved Hide resolved
session/bootstrap.go Outdated Show resolved Hide resolved
@ti-chi-bot
Copy link

ti-chi-bot bot commented Sep 11, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hawkingrei, time-and-fate

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [hawkingrei,time-and-fate]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 11, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Sep 11, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-09-05 06:02:06.268742823 +0000 UTC m=+2425290.817758807: ☑️ agreed by hawkingrei.
  • 2023-09-11 11:53:57.30775863 +0000 UTC m=+270219.232315024: ☑️ agreed by time-and-fate.

@hawkingrei
Copy link
Member

/test all

@tiprow
Copy link

tiprow bot commented Sep 11, 2023

@winoros: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
tiprow_fast_test 3960dcd link true /test tiprow_fast_test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ti-chi-bot ti-chi-bot bot merged commit 51f1a82 into pingcap:master Sep 11, 2023
@ti-chi-bot ti-chi-bot added the needs-cherry-pick-release-6.5 Should cherry pick this PR to release-6.5 branch. label Aug 13, 2024
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 13, 2024
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-6.5: #55381.

ti-chi-bot bot pushed a commit that referenced this pull request Aug 25, 2024
@ti-chi-bot ti-chi-bot bot added the needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. label Oct 28, 2024
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request could not be created: failed to create pull request against pingcap/tidb#release-7.1 from head ti-chi-bot:cherry-pick-46138-to-release-7.1: the GitHub API request returns a 403 error: {"message":"You have exceeded a secondary rate limit and have been temporarily blocked from content creation. Please retry your request again later. If you reach out to GitHub Support for help, please include the request ID ACA6:3F108B:1B829CA:33D7586:671F467F and timestamp 2024-10-28 08:08:31 UTC.","documentation_url":"https://docs.github.com/rest/overview/rate-limits-for-the-rest-api#about-secondary-rate-limits","status":"403"}

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Oct 28, 2024
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot bot removed the needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. label Nov 5, 2024
@AilinKid AilinKid added the needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. label Nov 6, 2024
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Nov 6, 2024
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.1: #57151.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm needs-cherry-pick-release-6.5 Should cherry pick this PR to release-6.5 branch. needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Huge memory pulse in non foreground workload cluster GCStats runs too many times
5 participants