Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Monitoring: Add Node & Pod info in TiDB Grafana #885

Merged
merged 11 commits into from
Sep 30, 2019
Merged

Monitoring: Add Node & Pod info in TiDB Grafana #885

merged 11 commits into from
Sep 30, 2019

Conversation

qiffang
Copy link
Contributor

@qiffang qiffang commented Sep 6, 2019 •
edited
Loading

What problem does this PR solve?

  • Add kubernetes Node and Pod metrics in TiDB Cluster Grafana
  • Support lighting monitoring and alert

It is related to PR in monitoring repo

What is changed and how does it work?

Check List

Code changes

  • Has Helm charts change

Side effects

  • Breaking backward compatibility

Tests

  • Manual test

Code changes

  • Has Helm charts change

Side effects

  • Breaking backward compatibility

Does this PR introduce a user-facing change?:

Action required: Users should migrate the configs in values.yaml of previous chart releases to the new values.yaml of the new chart. Otherwise, the monitor pods will be failed when upgrading monitor with the new chart.
For example, configs in old values.yaml:
monitor:
  ...
  imagePullPolicy: IfNotPresent

After migration, new values.yaml should be as below:
monitor:
  ...
   imagePullPolicy: Always
   config:
     K8S_PROMETHEUS_URL: http://prometheus-k8s.monitoring.svc:9090

@qiffang qiffang changed the title Monitoring add node port info in TiDB Grafana Monitoring: Add Node Port info in TiDB Grafana Sep 6, 2019
@tennix tennix self-requested a review September 10, 2019 10:05
@tennix tennix changed the title Monitoring: Add Node Port info in TiDB Grafana Monitoring: Add Node & Pod info in TiDB Grafana Sep 10, 2019
tennix
tennix reviewed Sep 10, 2019
View reviewed changes
Copy link
Member

@tennix tennix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider compatiblity when adding new features.

charts/tidb-cluster/templates/monitor-deployment.yaml Outdated Show resolved Hide resolved
charts/tidb-cluster/templates/monitor-deployment.yaml Show resolved Hide resolved
charts/tidb-cluster/templates/monitor-deployment.yaml Outdated Show resolved Hide resolved
charts/tidb-cluster/values.yaml Outdated Show resolved Hide resolved
@qiffang
Monitoring-Add-Node-Port-Info-In-TIDB-Grafana
a104f9e 
Signed-off-by: qiffang <947321353@qq.com>
@qiffang
Monitoring-Add-Node-Port-Info-In-TIDB-Grafana
c4b4e91 
Signed-off-by: qiffang <947321353@qq.com>
tennix
tennix previously approved these changes Sep 16, 2019
View reviewed changes
Copy link
Member

@tennix tennix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

weekface
weekface reviewed Sep 16, 2019
View reviewed changes
charts/tidb-cluster/values.yaml
@@ -359,7 +359,9 @@ monitor:
storage: 10Gi
initializer:
image: pingcap/tidb-monitor-initializer:v3.0.1
imagePullPolicy: IfNotPresent
imagePullPolicy: Always
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why change the imagePullPolicy to Always?

Copy link
Contributor Author

@qiffang qiffang Sep 16, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tidb operator is incompatible with old image and for now the monitoring image is changed frequency, so we think it is better to update the policy with Always

weekface
weekface previously approved these changes Sep 17, 2019
View reviewed changes
@tennix
Copy link
Member

tennix commented Sep 18, 2019

/run-e2e-in-kind

2 similar comments
@qiffang
Copy link
Contributor Author

qiffang commented Sep 18, 2019

/run-e2e-in-kind

@qiffang
Copy link
Contributor Author

qiffang commented Sep 26, 2019

/run-e2e-in-kind

@qiffang qiffang dismissed stale reviews from weekface and tennix via 5691a21 September 26, 2019 08:55
@qiffang
Copy link
Contributor Author

qiffang commented Sep 26, 2019

/run-e2e-in-kind

@qiffang
Copy link
Contributor Author

qiffang commented Sep 27, 2019

/run-e2e-in-kind

@qiffang
Copy link
Contributor Author

qiffang commented Sep 27, 2019

/run-e2e-in-kind

tennix
tennix approved these changes Sep 27, 2019
View reviewed changes
Copy link
Member

@tennix tennix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qiffang
Copy link
Contributor Author

qiffang commented Sep 27, 2019

/run-e2e-in-kind

weekface
weekface approved these changes Sep 27, 2019
View reviewed changes
Copy link
Contributor

@weekface weekface left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tennix
Copy link
Member

tennix commented Sep 29, 2019

/run-e2e-in-kind

@tennix
Copy link
Member

tennix commented Sep 30, 2019

/run-e2e-in-kind

@tennix tennix merged commit 65d7d5a into pingcap:master Sep 30, 2019
This was referenced Dec 19, 2019
Closed
Closed
@qiffang qiffang mentioned this pull request Dec 20, 2019
Merged
sre-bot pushed a commit that referenced this pull request Dec 23, 2019
@qiffang @sre-bot
Automated cherry pick of #885: Monitoring: add node and pod informati…
beb2b27 
…on in tidb cluster (#1385)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tennix tennix tennix approved these changes

@weekface weekface weekface approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@qiffang @tennix @weekface