-
Notifications
You must be signed in to change notification settings - Fork 14.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Documenting SCTP support in Kubernetes #10279
Conversation
Deploy preview for kubernetes-io-vnext-staging processing. Built with commit 3a31d3e https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/5ba2aed1c965925adb7934ab |
Deploy preview for kubernetes-io-vnext-staging processing. Built with commit 2f63fd5 https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/5b996dc5dd28ef64c7bd3bbb |
/assign @steveperry-53 |
/milestone v1.12 |
/unassign @steveperry-53 |
@thockin Could I please get a technical review here? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added some small stylistic notes that you might want to incorporate.
@@ -189,6 +189,15 @@ spec: | |||
|
|||
This ensures that even pods that aren't selected by any other NetworkPolicy will not be allowed ingress or egress traffic. | |||
|
|||
## SCTP support | |||
|
|||
{{< feature-state feature-state state="alpha" >}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah. This is a typo from the shortcode docs. Should just be: {{< feature-state state="alpha" >}}
. Looks like this is causing netlify to fail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, we may want to pin this to 1.12 with for_k8s_version="v1.12"
|
||
{{< feature-state feature-state state="alpha" >}} | ||
|
||
Kubernetes introduces the support of SCTP as a `protocol` value in `NetworkPolicy` definitions as alpha feature. To enable this feature, the cluster administrator needs to enable the `SCTPSupport` feature gate on the apiserver, for example, `“--feature-gates=SCTPSupport=true,...”`. When the feature gate is enabled, users can set the `protocol` field of a `NetworkPolicy` to `“SCTP”`. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for e.g. TCP connections. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as alpha feature -> as an alpha feature
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably best to remove quote-marks from inside the code blocks.
“SCTP”
can just be SCTP if you want to show a string value. The docs style guide talks about this here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you don't need the "e.g." at the end
"just like it does for e.g. TCP connections" -> "just like it does for TCP connections"
@@ -917,6 +931,31 @@ Service is a top-level resource in the Kubernetes REST API. More details about t | |||
API object can be found at: | |||
[Service API object](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#service-v1-core). | |||
|
|||
## SCTP support | |||
|
|||
{{< feature-state feature-state state="alpha" >}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same shortcode issue here.
|
||
{{< feature-state feature-state state="alpha" >}} | ||
|
||
Kubernetes introduces the support of SCTP as a `protocol` value in `Service`, `Endpoint`, `NetworkPolicy` and `Pod` definitions as alpha feature. To enable this feature, the cluster administrator needs to enable the `SCTPSupport` feature gate on the apiserver, for example, `“--feature-gates=SCTPSupport=true,...”`. When the feature gate is enabled, users can set the `protocol` field of a `Service`, `Endpoint`, `NetworkPolicy` and `Pod` to `“SCTP”`. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for e.g. TCP connections. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same feedback as the section in concepts/services-networking/network-policies.md
@tfogo Thank you! I updated the docs accordingly. |
Going to try to rebuild netlify |
/test |
Anyway, thanks for the changes @janosi |
One note: the kubectl CLI related docs contain much more (due to their generated nature) than the changes required due to the SCTP feature. The SCTP feature only affects "kubectl expose". The main change in the CLI due to SCTP is:
|
|
||
{{< feature-state for_k8s_version="v1.12" state="alpha" >}} | ||
|
||
Kubernetes introduces the support of SCTP as a `protocol` value in `NetworkPolicy` definitions as an alpha feature. To enable this feature, the cluster administrator needs to enable the `SCTPSupport` feature gate on the apiserver, for example, `“--feature-gates=SCTPSupport=true,...”`. When the feature gate is enabled, users can set the `protocol` field of a `NetworkPolicy` to `SCTP`. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for TCP connections. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know if we want words like "introduces" which are anchored in time, vs something like "supports".
I guess that, since this is related to the feature-gate, it might be OK.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, I change it to "supports".
is `TCP`. | ||
|
||
{{< note >}} | ||
**Note:** SCTP support is an alpha feature |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want a version number here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. Are you OK with something like "SCTP support is an alpha feature since Kubernetes 1.12"?
Unfortunately, I could not find a good way to include the version as a shortcode, without the usual "FEATURE STATE:..." prefix, which would look strange here in this note, I think.
@@ -166,6 +170,10 @@ By default, the choice of backend is round robin. | |||
|
|||
Note that in the above diagram, `clusterIP` is shown as `ServiceIP`. | |||
|
|||
{{< note >}} | |||
**Note:** SCTP is not supported in userspace mode. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How do we say this when we had that huge discussion about it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, here I mean the kube-proxy userspace mode. That is, if the kube-proxy is in userspace mode it does not proxy SCTP traffic through itself. But of course, the usage of userspace SCTP stacks by applications is allowed, as we discussed during the PR.
I update it like: "SCTP is not supported by the kube-proxy in userspace mode"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, I delete this note from here. There is a separate note about it in the main SCTP chapter at the bottom of this page.
{{< note >}} | ||
**Note:** The support of SCTP in the cloud provider's load balancer is up to the cloud provider's | ||
load balancer implementation. If SCTP is not supported by the cloud provider's load balancer the | ||
Service creation request is rejected. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think that is true - the service will be accepted, but the LB will fail to create.
|
||
{{< feature-state for_k8s_version="v1.12" state="alpha" >}} | ||
|
||
Kubernetes introduces the support of SCTP as a `protocol` value in `Service`, `Endpoint`, `NetworkPolicy` and `Pod` definitions as an alpha feature. To enable this feature, the cluster administrator needs to enable the `SCTPSupport` feature gate on the apiserver, for example, `“--feature-gates=SCTPSupport=true,...”`. When the feature gate is enabled, users can set the `protocol` field of a `Service`, `Endpoint`, `NetworkPolicy` and `Pod` to `SCTP`. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for TCP connections. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wordsmithing around tense - "introduces" is a bit awkward IMO, but docs people will finalize.
|
||
Kubernetes introduces the support of SCTP as a `protocol` value in `Service`, `Endpoint`, `NetworkPolicy` and `Pod` definitions as an alpha feature. To enable this feature, the cluster administrator needs to enable the `SCTPSupport` feature gate on the apiserver, for example, `“--feature-gates=SCTPSupport=true,...”`. When the feature gate is enabled, users can set the `protocol` field of a `Service`, `Endpoint`, `NetworkPolicy` and `Pod` to `SCTP`. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for TCP connections. | ||
|
||
The kube-proxy sets up iptables rules for some `Service` types: `Service` with `clusterIP`, `Service` with `nodePort`, and `Service` with `externalIP`. In order to support proper iptables based redirection the following kernel modules shall be loaded on the host: `nf_conntrack_proto_sctp` and `nf_nat_proto_sctp`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
iptables is an implementation detail. If we want to talk about this at all, we need to talk about the differences between iptables mode and ipvs mode (did we test IPVS mode?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, it is a bit complex topic (especially considering those cases when the kube-proxy starts managing iptables rules in ipvs mode, too). I delete this part.
We have not tested ipvs mode yet e2e, only unit tests run.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But have we verified SCTP with IPVS in general?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have created a single-master cluster in GCE based on Ubuntu 18.04 LTS. I used the new k8s images that contain SCTP support. I set the proxy-mode of kube-proxy to "ipvs" and then I created an SCTP Service + 2 busybox backend pods. ipvsadm showed the mapping of the ClusterIP to the backend IPs for SCTP like:
root@instance-2:~# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP instance-2:https rr
-> instance-2.c.k8s-sctp-test.i Masq 1 3 0
TCP instance-2:domain rr
-> 10.244.0.2:domain Masq 1 0 0
-> 10.244.0.3:domain Masq 1 0 0
UDP instance-2:domain rr
-> 10.244.0.2:domain Masq 1 0 0
-> 10.244.0.3:domain Masq 1 0 0
SCTP instance-2:7777 rr
-> 10.244.0.4:7777 Masq 1 0 0
-> 10.244.0.5:7777 Masq 1 0 0
|
||
The support of multihomed SCTP associations requires that the CNI plugin can support the assignment of multiple interfaces and IP addresses to a `Pod`. | ||
|
||
The kube-proxy sets up iptables rules for some `Service` types: `Service` with `clusterIP`, `Service` with `nodePort`, and `Service` with `externalIP`. The corresponding kernel module should provide the necessary logic in order to support proper iptables based redirection for multihomed associations. This logic is not present in the ordinary SCTP related kernel modules, namely `nf_conntrack_proto_sctp` and `nf_nat_proto_sctp`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a dup of words just 1 section before?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, but here from the perspective of multihomed SCTP associations - it is to explain that the relevant kernel modules should support multihomed SCTP NAT, which is not the case with the "standard" SCTP kernel modules. I update the wording.
@thockin I changed the doc according to your comments - I hope so. |
Hi @janosi would you mind reverting the changes you made to the generated files? We're going to be generating all of these all at once during the release process, so it would be cleaner to remove them from this PR. Thanks for all your work on this! |
This reverts commit 0d5a1e6.
Hi @tfogo Sure, done, no problem. :) |
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: zparnold The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once small nit and then LGTM
|
||
Kubernetes supports SCTP as a `protocol` value in `NetworkPolicy` definitions as an alpha feature. To enable this feature, the cluster administrator needs to enable the `SCTPSupport` feature gate on the apiserver, for example, `“--feature-gates=SCTPSupport=true,...”`. When the feature gate is enabled, users can set the `protocol` field of a `NetworkPolicy` to `SCTP`. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for TCP connections. | ||
|
||
The CNI plugin has to support SCTP as `protocol` value in `NetworkPolicy`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NetworkPolicy != CNI. I would just remove this line
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For some reasons it got merged after the approve from @zparnold. I update the doc with a new issue according to this comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@thockin, on the other hand, the existing documentation of Network Policy tells, that Network policies are implemented by the network plugin, so you must be using a networking solution which supports NetworkPolicy
So, that part of the documentation requires the support of Network Policy from the network plugin.
* Update docs for fields allowed at root of CRD schema (#9973) * add plugin docs and examples (#10053) * docs update to promote TaintNodesByCondition to beta (#9626) * HPA Specificity Improvements (#8757) Updated the HPA docs to reference the `autoscaling/v2beta2` API version, and added documentation about the new fields. * adjust docs for pod ready++ (#10049) * Remove --cadvisor-port - has been deprecated since v1.10 (#10023) Change-Id: Id2a685473a243aef492a98ff450759f39e362557 * Add Documentation for Snapshot Feature (#9948) * Add documentation for snapshot feature * Update volume-snapshots.md * Add dry-run to api-concepts (#10033) * kubeadm-init: Update the offline support section (#10062) The update includes the following things (in mind with Kubernetes 1.12): - Remove the 1.8 image versions - Add the 1.10 image versions that were missing until now - Include a comment for the missing arch suffixes in 1.12 Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com> * Say bye to `DynamicProvisioningScheduling` (#10157) The mentioned feature gate is now collapsed into `VolumeScheduling`. xref: kubernetes/kubernetes#67432 * Update ResourceQuota per PriorityClass state for 1.12 (#10229) * TokenRequest and TokenRequestProjection now beta (#10161) xref: kubernetes/kubernetes#67349 * Change feature state for kms provider to beta. (#10230) KMS Provider will be graduating to beta in v1.12, reflecting this change on the website. * coredns default (#10200) * Promote ShareProcessNamespace to beta in docs (#9996) * Add CoreDNS details to DNS Debug docs (#10201) * add coredns details * address nits, add query logging section * Update docs with topology aware dynamic provisioning (#9939) * Document topology aware volume binding feature * update for readability * Update storage-classes.md * comma splice * don't abbreviate * HPA Algorithm Information Improvements (#9780) * Update HPA docs with more algorithm details The HPA docs pointed to an out-of-date document for information on the algorithm details, which users were finding confusing. This sticks a section on the algorithm in the HPA docs instead, documenting both general behavior and corner cases. * Add glossary info, HPA docs on quantities People often ask about the quantity notation when working with the metrics APIs, so this adds a glossary entry on quantities (since they're used elsewhere in the system), and a short explantation in the HPA walkthough. * Information about HPA readiness and stabilization This adds information about the new changes to HPA readiness and stabilization from kubernetes/enhancements#591, and other minor changes that landed in Kubernetes 1.12. * Update horizontal-pod-autoscale.md * Audit 1.12 doc (#9953) * audit 1.12 document * remove legacy audit feature kubernetes/kubernetes#65862 * update feature gate doc * MountPropagation is now GA (#10090) * RuntimeClass documentation (#10102) * RuntimeClass documentation * Update runtime-class.md * Add documentation for Scheduler performance tuning (#10048) * Add documentation for Scheduler performance tuning * Update scheduler-perf-tuning.md * TTL controller for cleaning up finished resources (#10064) * TTL controller for cleaning up finished resources * Address comments * Update ttlafterfinished.md * Bump quota configuration api version (#10217) * Incremental update from master (#10278) * fix invalid href of cloud controller manager (#10240) * fix invalid yaml format (#10238) * update storage-limits doc with Azure disk part (#10224) update storage-limits doc with Azure disk part fix comments * Update kubelet-config-file.md (#10222) Update link to KubeletConfiguration struct. * fix a trivial misspelling (#10244) * Fix cassandra-statefulset.yaml indent level (#10243) * Mention minimum etcd versions (#10208) Source: https://groups.google.com/d/msg/kubernetes-dev/jMPA4JzKiY4/HIx2ugvLBAAJ * fix 404 error (#10250) * Small verb tweak (#10190) Present participle, ftw. * Add AnchorJS logic for header links (#10155) * Add AnchorJS JavaScript * Remove existing inpage_heading logic * Remove underline from anchor tags * Use single icon and add touch visibility * Use paragraph link icon for AnchorJS * Update Sass to use code formatting in docsContent headers * Update header size coverage to H3-H6 * fix broken link in kubefed.md (#10254) * Update the version numbers for the X-Remote-Extra- and Impersonate-Extra- key fixes (#9827) The fix was cherry picked into 1.11.3, 1.10.7, and 1.9.11: kubernetes/kubernetes#67162 kubernetes/kubernetes#67163 kubernetes/kubernetes#67164 * fix typo (#10168) * fix typo * addressing comments. * Update setup-ha-etcd-with-kubeadm.md * fix typos (#10252) * fix description of contribute guide (#10253) * describe truncate feature about advanced audit (#10236) * describe truncate feature about advanced audit * Update audit.md * docs update to promote ScheduleDaemonSetPods to beta (#9923) * Dynamic volume limit updates for 1.12 (#10211) * add a placeholder commit * Update docs for csi volume limits * Update storage-limits.md * Add "MayRunAs" value among other GroupStrategies (#9888) * Add CoreDNS details to the customize DNS doc (#10228) * Add CoreDNS details to the customize DNS doc Rewrite the document to include more details about CoreDNS, since it's now the default from v1.12 * Address comments * Improve doc wording * Fix link * Update dns-custom-nameservers.md * Update dns-custom-nameservers.md * Fix secrets docs in 1.12 branch (#10056) * Fix secrets docs * Update secret.md * Revert CoreDNS Docs (#10319) * Revert "Add CoreDNS details to DNS Debug docs (#10201)" This reverts commit 462817a. * Revert "Add CoreDNS details to the customize DNS doc (#10228)" This reverts commit e7319ee. * Revert "coredns default (#10200)" This reverts commit 698e93b. * Add CRI installation instructions page Added cri-installation page with CRI installation instructions Referenced it from kubeadm-init and install-kubeadm pages. * kubeadm: update API types documentation for 1.12 (#10283) v1alpha2 -> v1alpha3 MasterConfiguration -> [new-api-types] * TokenRequest feature documentation (#10295) * AdvancedAuditing is now GA (#10156) xref: kubernetes/kubernetes#65862 `AdvancedAuditing` feature is GA in 1.12. This PR adjusts the related docs. * update runtime-class.md (#10332) * update runtime-class.md * Update runtime-class.md * Document cross-authorizer permissions for creating RBAC roles (#10015) * Document cross-authorizer permissions for creating RBAC roles * Update rbac.md * kubeadm: update authored content for 1.12 (reference docs and cluster creation) (#10348) * kubeadm: update authored content in reference docs for 1.12 * kubeadm: add time frame in create-cluster-kubeadm for 1.12 * add AllowedProcMountTypes and ProcMountType to docs (#9911) Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * kubeadm: add new command line reference (#10306) Add: - placeholder files - include place holder files - include "renew" sub command - add missing tabs for "alpha phase kubelet" * Documenting SCTP support in Kubernetes (#10279) * Documenting SCTP support in Kubernetes Service, Endpoint, NetworkPolicy and Pod * Updates based on comments on the PR * kubectl expose update with SCTP support * Updated according to comments in the PR * Revert "kubectl expose update with SCTP support" This reverts commit 0d5a1e6. * TLS Bootstrap and Server Cert Rotation feature documentation (#10232) * TokenRequest feature documentation * line wrapping to make review not insane * update content for GA without major refactor * Update kubelet-tls-bootstrapping.md * Add clarifications for volume snapshots (#10296) * Update kubadm ha installation for 1.12 (#10264) * Update kubadm ha installation for 1.12 Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * update stable version Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * Update stacked control plane for v1.12 (#2) * use v1alpha3 Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * more v1alpha3 (#4) * updates Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * Document how to run in-tree cloud providers with kubeadm (#10357) Change-Id: Iab6b996a830503d74a6eb0c507c5f8ca7a39235b * kubeadm reference doc for release 1.12 (#10359) * Revert "Revert "Add CoreDNS details to DNS Debug docs (#10201)"" This reverts commit bb30f4d. * Revert "Revert "Add CoreDNS details to the customize DNS doc (#10228)"" This reverts commit bc23d45. * Revert "Revert "coredns default (#10200)"" This reverts commit 7f4350d. * add missing instruction for ha guide (#10374) Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * kubeadm - Ha upgrade updates (#10340) * Update HA upgrade docs * Adds external etcd HA upgrade guide Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * copyedit * more edits * add runasgroup in psp (#10076) * update KubeletPluginsWatcher feature gate (#10205) * generated 1.12 docs * Building Multi-arch images with Manifests (#10379) In 1.12, a variety of images used in a typical kubernetes installation have started to using manifests to better support environments with arm or ppc64le architectures. For example all images used with kubeadm by default have manifests, another would be all the tests in the conformance test suite. Here we capture the best practices for everyone to start using manifests in their own workflows. Change-Id: I5ba4c5fe55ffc9486a8251760f3352be4f2e1494 * Upgrade docs for v1.12 (#10344) * generated assets and docs * remove 1.7 * update 1.12 * update plugin documentation under docs>tasks>extend-kubectl (#10259) * update plugin documentation under docs>tasks>extend-kubectl * Update kubectl-plugins.md
Documentation update to reflect the changes due to the feature "SCTP support implementation for Kubernetes"
Fixes: #10158
/milestone 1.12