[KEP-2400] Node memory swap support (replacement for inactive issue #2400)

[iholder101]

This is a replacement issue for the outdated, inactive previous issue: #2400.

Enhancement Description

• One-line enhancement description (can be used as a release note): Kubernetes nodes support swap memory.
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2400-node-swap
• Discussion Link: https://docs.google.com/document/d/1CZtRtC8W8FwW_VWQLKP9DW_2H-hcLBcH9KBbukie67M/edit# Kubelet/Kubernetes should work with Swap Enabled kubernetes#53533
• Primary contact (assignee): @iholder101
• Responsible SIGs: Node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.22
  • Beta release target (x.y): 1.28
  • Stable release target (x.y):
• Alpha
  • KEP (k/enhancements) update PR(s): Node system swap support #2602
  • Code (k/k) update PR(s):
    • CI/test infra changes: Setup new node image with swap enabled test-infra#22602 Add swap periodics for node test-infra#22809
    • CRI and kubelet changes Alpha node swap support kubernetes#102823
  • Docs (k/website) update PR(s): KEP-2400: Alpha node support for swap memory website#28838
• Beta1
  • KEP (k/enhancements) update PR(s):
    • KEP-2400: Update swap KEP for 1.23 beta #2858
    • KEP-2400: Graduate swap to Beta 1 #3957
  • Issues and Code (k/k) update PR(s):
    • Support cgroups v2 with auto-calculated LimitedSwap for Burstable QoS pods
      • [KEP-2400] Enable Swap Support for Cgroup v2 Only kubernetes#119427
      • [KEP-2400] Enable auto-calculated LimitedSwap for Burstable QoS pods kubernetes#119428
      • Add full cgroup v2 swap support with automatically calculated swap limit for LimitedSwap and Burstable QoS Pods kubernetes#118764
    • Add Swap stats to /stats/summary and /metrics/resource
      • [KEP-2400] Add swap memory to the Kubelet stats API kubernetes#119425
      • Add swap to stats to Summary API and Prometheus endpoints (/stats/summary and /metrics/resource) kubernetes#118865
    • Determine a set of metrics for measuring node performance with swap enabled
      • [KEP-2400] Determine a set of metrics for measuring node performance with swap enabled kubernetes#119424
      • Add swap to stats to Summary API and Prometheus endpoints (/stats/summary and /metrics/resource) kubernetes#118865
      • Add a metric for machine swap capacity google/cadvisor#3339
    • Improve test coverage and make sure tests are green
      • [KEP-2400] Swap: improve test coverage and make sure tests are green kubernetes#119430
      • Add full cgroup v2 swap support with automatically calculated swap limit for LimitedSwap and Burstable QoS Pods kubernetes#118764
  • Docs (k/website) update(s):
    • Swap graduates to Beta1: add a blog-post website#42009
    • Swap graduates to Beta1: add documentation and update feature gates website#42167
  • Misc
    • Bugfix: MachineInfo Clone() to also clone "SwapCapacity" google/cadvisor#3293
    • Bump cadvisor version to v0.47.3 kubernetes#119225
  • Archive / Outdated
• Beta2
• Future (related PRs, can be assigned to a specific track later)
  • Add the ability to set a system-reserved quantity of swap on a node
    • [KEP-2400] Add the ability to set a system-reserved quantity of swap on a node kubernetes#105019
    • add --system-reserved support for swap(cgroup v2 only): only for burstable pods kubernetes#105271
  • Fail Kubelet at startup if swap is configured with cgroup v1 kubernetes#119327

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

/sig node

[iholder101]

/cc @harche @haircommander @rphillips @SergeyKanzhelev @mrunalp

[ffromani]

/cc

[sftim]

I've given kubernetes/website#42009 a first pass review. I strongly recommend a security review of the documentation we're providing for this newly beta feature. Lots of clusters run with beta features enabled, and users may need to understand the risks.

[pacoxu]

I've given kubernetes/website#42009 a first pass review. I strongly recommend a security review of the documentation we're providing for this newly beta feature. Lots of clusters run with beta features enabled, and users may need to understand the risks.

This beta feature is a little special in that the default FG is off.

NodeSwap: {Default: false, PreRelease: featuregate.Beta},  // 1.28

[sftim]

As soon as it's beta, we can't assume that security-minded folk would disable it out of habit. Instead we really must review the advice to end users. Someone might take the graduation to beta as a signal that it's now safe to enable as documented.

[iholder101]

As soon as it's beta, we can't assume that security-minded folk would disable it out of habit. Instead we really must review the advice to end users. Someone might take the graduation to beta as a signal that it's now safe to enable as documented.

@sftim Bear in mind that Swap is disabled by default in Beta1, and that we address the security both in the KEP and in the documentation.

However, I agree we need to make sure we warn and guide the users correctly.

Thanks!

[SergeyKanzhelev]

can we get back to working on #2400? Duplicating KEP issues is not working very well, lot's of confusion

[iholder101]

can we get back to working on #2400? Duplicating KEP issues is not working very well, lot's of confusion

The problem with #2400 is that I'm not the owner, so can't edit it which leads to outdated confusing information.

IMO we should either close the previous issue and clearly link to this issue as the updated one, or somehow grant me edit access to the previous issue if possible. WDYT?

[pacoxu]

The problem with #2400 is that I'm not the owner, so can't edit it which leads to outdated confusing information.

I can do that for you. I think I have already updated recent updates on the Swap feature to the issue description of #2400.

[iholder101]

The problem with #2400 is that I'm not the owner, so can't edit it which leads to outdated confusing information.

I can do that for you. I think I have already updated recent updates on the Swap feature to the issue description of #2400.

Thank you!
Do you mean that you can update the issue? Or also grant me edit rights (which is preferred)?

I see that the previous issue still contains outdated information. If you can't grant me edit rights, can you please copy-paste the description from this issue to the original one? Here's the description with all of the markdown tags:

This is a replacement issue for the outdated, inactive previous issue: https://github.com/kubernetes/enhancements/issues/2400.

# Enhancement Description
* One-line enhancement description (can be used as a release note): Kubernetes nodes support swap memory.
* Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2400-node-swap
* Discussion Link: https://docs.google.com/document/d/1CZtRtC8W8FwW_VWQLKP9DW_2H-hcLBcH9KBbukie67M/edit# [Kubelet/Kubernetes should work with Swap Enabled kubernetes#53533](https://github.com/kubernetes/kubernetes/issues/53533)
* Primary contact (assignee): @iholder101
* Responsible SIGs: Node
* Enhancement target (which target equals to which milestone):
  * Alpha release target (x.y): 1.22
  * Beta release target (x.y): 1.28
  * Stable release target (x.y):
* [x]  Alpha
  * [x]  KEP (`k/enhancements`) update PR(s): https://github.com/kubernetes/enhancements/pull/2602
  * [x]  Code (`k/k`) update PR(s):
    * [x]  CI/test infra changes: https://github.com/kubernetes/test-infra/pull/22602 https://github.com/kubernetes/test-infra/pull/22809
    * [x]  CRI and kubelet changes https://github.com/kubernetes/kubernetes/pull/102823
  * [x]  Docs (`k/website`) update PR(s): https://github.com/kubernetes/website/pull/28838
* [ ]  Beta1
  * [x]  KEP (`k/enhancements`) update PR(s):
    * [x]  https://github.com/kubernetes/enhancements/pull/2858
    * [x]  https://github.com/kubernetes/enhancements/pull/3957
  * [x]  Issues and Code (`k/k`) update PR(s):
    * Support cgroups v2 with auto-calculated LimitedSwap for Burstable QoS pods
      * [x]  https://github.com/kubernetes/kubernetes/issues/119427
      * [x]  https://github.com/kubernetes/kubernetes/issues/119428
      * [x]  https://github.com/kubernetes/kubernetes/pull/118764
    * [x] Add Swap stats to `/stats/summary` and `/metrics/resource`
      * [x]  https://github.com/kubernetes/kubernetes/issues/119425
      * [x]  https://github.com/kubernetes/kubernetes/pull/118865
    * [x] Determine a set of metrics for measuring node performance with swap enabled
      * [x]  https://github.com/kubernetes/kubernetes/issues/119424
      * [x]  https://github.com/kubernetes/kubernetes/pull/118865
      * [x]  https://github.com/google/cadvisor/pull/3339
    * [x] Improve test coverage and make sure tests are green
      * [x]  https://github.com/kubernetes/kubernetes/issues/119430
      * [x]  https://github.com/kubernetes/kubernetes/pull/118764
  * [ ]  Docs (`k/website`) update(s):
    * [ ] https://github.com/kubernetes/website/pull/42009
    * [X] https://github.com/kubernetes/website/pull/42167
  * [x] Misc
    * [x] https://github.com/google/cadvisor/pull/3293
    * [x] https://github.com/kubernetes/kubernetes/pull/119225
  * Archive / Outdated
* [ ]  Beta2
* [ ]  Future (related PRs, can be assigned to a specific track later)
  * Add the ability to set a system-reserved quantity of swap on a node
    * [ ]  https://github.com/kubernetes/kubernetes/issues/105019
    * [ ]  https://github.com/kubernetes/kubernetes/pull/105271
  * [ ] https://github.com/kubernetes/kubernetes/pull/119327

_Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently._

/sig node

[pacoxu]

Updated.

Sorry, I have no permission to grant you the permission. I will try to keep up with your steps and follow up on the feature. When there are any updates needed, you can ping me on GitHub or Slack.

[iholder101]

Too bad Github doesn't have an issue-ownership-transfer feature :D
Thank you @pacoxu!

/close

[k8s-ci-robot]

@iholder101: Closing this issue.

In response to this:

Too bad Github doesn't have an issue-ownership-transfer feature :D
Thank you @pacoxu!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.