Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set net.ipv4.tcp_keepalive_time and net.core.somaxconn for tidb and tikv in init container #1107

Merged
merged 4 commits into from
Nov 7, 2019

Conversation

DanielZhangQD
Copy link
Contributor

What problem does this PR solve?

fix #880 for GCP and Alibaba

What is changed and how does it work?

Set net.ipv4.tcp_keepalive_time and net.core.somaxconn for tidb in init container
Set net.core.somaxconn for tikv in init container

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)

Code changes

  • Has Helm charts change
  • Has Go code change
  • Has documents change

Related changes

  • Need to update the documentation

Does this PR introduce a user-facing change?:

Support configuring net.ipv4.tcp_keepalive_time and net.core.somaxconn for TiDB and configuring net.core.somaxconn for TiKV.

@DanielZhangQD
Copy link
Contributor Author

After discussion with @tennix and @weekface, will update the solution as below:

  • users still configure PodSecurityContext.sysctls in values.yaml
  • if annotations for init container is set in values.yaml, tidb-operator will convert PodSecurityContext.sysctl to init container;
  • otherwise, set PodSecurityContext with existing logic.

tcpKeepaliveIntvl: 75
{{- end }}
{{- if .Values.soMaxConn }}
soMaxConn: {{ .Values.soMaxConn }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting these parameters at cluster level is a little misleading, how about moving these to component? e.g. .sepc.tidb.tcpKeepaliveTime, .spec.tikv.soMaxConn

Copy link
Contributor

@cofyc cofyc Nov 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer a general solution. How about using the sysctl struct from Kubernetes?

It's an alternative way when configuring via Kubelet is not available, no need to use another structure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 config per component, like podSecurityContext which we are using now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reuse the PodSecurityContext in each component.

privileged := true
initContainers = append(initContainers, corev1.Container{
Name: "init",
Image: controller.GetSlowLogTailerImage(tc),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use another image or promote the slow log tailer image to a general util image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, the image does not matter, so I just reuse the image from the slow log tailer and we also have a default value for it, this does not involve any change to values.yaml and users do not need to know about it at all.

@@ -266,6 +266,31 @@ func getNewTiDBSetForTidbCluster(tc *v1alpha1.TidbCluster) *apps.StatefulSet {
})
}

sysctls := "sysctl -w"
if tc.Spec.TCPKeepaliveTime > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if tc.Spec.TCPKeepaliveTime > 0 {
if tc.Spec.TCPKeepaliveTime > 0 && tc.Spec.TCPKeepaliveIntvl > 0 {

@DanielZhangQD
Copy link
Contributor Author

@cofyc @aylei @tennix @weekface Please help review the new solution.

weekface
weekface previously approved these changes Nov 6, 2019
Copy link
Contributor

@weekface weekface left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

privileged := true
initContainers = append(initContainers, corev1.Container{
Name: "init",
Image: controller.GetSlowLogTailerImage(tc),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest using an annotation to configure the init container image, and also set a default image when the annotation is not available.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer a dedicate field instead of annotation if we really want it to be configurable, for troubleshooting purpose, changing the image via kubectl edit is more straightforward.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's unnecessary to introduce user interface change for this image, it does not matter what it is and for TiDB version >= v2.1.8, slow log tailer must be enabled to check slow log.

@DanielZhangQD
Copy link
Contributor Author

@weekface @tennix @aylei comments addressed, PTAL again.

weekface
weekface previously approved these changes Nov 7, 2019
@DanielZhangQD
Copy link
Contributor Author

@weekface @tennix @aylei comments addressed, PTAL again.

@DanielZhangQD
Copy link
Contributor Author

/run-e2e-in-kind

Copy link
Member

@tennix tennix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@aylei aylei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aylei aylei merged commit 0f359e4 into pingcap:master Nov 7, 2019
@DanielZhangQD DanielZhangQD deleted the sysctl branch November 8, 2019 01:19
yahonda pushed a commit that referenced this pull request Dec 27, 2021
change description when using kubectl deleting tidbmonitor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Terraform: TiDB nodes should configure tcp_keepalive_time
5 participants