CSI inline volume size #1409

pohly · 2019-12-13T10:06:16Z

The proposal is to introduce a standardized parameter for the size of ephemeral inline volumes. This will help users (consistent API for different drivers) and enable the implementation of new functionality in Kubernetes (capacity-aware pod scheduling).

Original this was part of the Storage Capacity Constraints for Pod Scheduling KEP, but as it also makes sense by itself it was split out.

Enhancement issue: #1472

k8s-ci-robot · 2019-12-13T10:06:41Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pohly
To complete the pull request process, please assign saad-ali
You can assign the PR to them by writing /assign @saad-ali in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

keps/sig-storage/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pohly · 2019-12-13T10:06:45Z

/cc @satoru-takeuchi @msau42

k8s-ci-robot · 2019-12-13T10:06:47Z

@pohly: GitHub didn't allow me to request PR reviews from the following users: satoru-takeuchi.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @satoru-takeuchi @msau42

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

It was moved to kubernetes#1409.

satoru-takeuchi · 2019-12-18T21:51:10Z

keps/sig-storage/20191213-csi-inline-volume-size.md

+mount](https://kubernetes-csi.github.io/docs/pod-info.html) feature:
+if (and only if) the driver enables that, then a new
+`csi.storage.k8s.io/size` entry in
+`NodePublishVolumeRequest.publish_context` is set to the string


It's not publish_context, but volume_context. For more information, please refer to the following PR.

kubernetes-csi/docs#249

As a side note, I found this documentation error when reviewing this KEP, thanks!

Good catch and thanks for taking care of it also in the docs from which I copied this. I wonder how many other developers have skipped over this without noticing the mistake...

Fixed.

satoru-takeuchi · 2019-12-18T21:51:23Z

keps/sig-storage/20191213-csi-inline-volume-size.md

+string.
+
+This has to be optional because CSI drivers written for 1.16 might do
+strict validation of the `publish_context` content and reject volumes


Same as above.

satoru-takeuchi · 2019-12-18T21:52:07Z

I left some comments, others are LGTM.

This was found by Satoru Takeuchi and also fixed by him in kubernetes-csi/docs#249.

satoru-takeuchi

LGTM

pohly · 2020-01-17T10:42:39Z

I added a75f17c since @satoru-takeuchi reviewed this PR, with only one minor change to the proposal: the size string passed to the CSI driver is now explicitly documented as value in plain decimal to simplify parsing.

The main change is the addition of the parts that are needed for "implementable".

/assign @saad-ali

For final review and approval.

This adds the missing details (test plan, implementation history). Also clarifies what exactly the string representation of the size is when passed to the CSI driver.

msau42 · 2020-01-22T22:29:29Z

keps/sig-storage/20191213-csi-inline-volume-size.md

+vendor-specific parameter, it's not available to the Kubernetes
+scheduler.
+
+### fsSize field


Dose the scheduler have to look at this field and compare it with the storage pool capacity reported in #1353?

Yes, that was the original motivation for adding this field.

pohly · 2020-01-28T07:25:28Z

/hold

With #1353 on hold there is no urgent need to get this merged. More importantly, I'm starting to suspect that adding more features to CSI inline ephemeral volumes is just going to make the code base too complicated.

To me it seems more promising to map inline volumes to the normal provisioning and then let the normal code paths deal with provisioning, failure recovery, etc. One additional bonus would be that CSI drivers do not need to be modified to work with ephemeral inline volumes.

Here's a rough outline of that idea (from https://kubernetes.slack.com/archives/C8EJ01Z46/p1580043788038400?thread_ts=1579890578.025900&cid=C8EJ01Z46):

inline we allow a volume to be specified with a complete PVC (not the name, the actual spec of a PVC)
when the scheduler sees such a pod, it creates a new PVC in the namespace of the pod with a generated name (details to be decided...)
this PVC has selected-node set and then is treated like a normal PVC with late binding
it also has the pod as owner ref, so when the pod gets deleted, so does the PVC ("ephemeral")

One concern about this approach is that these generated PVCs will be visible to users and users might do "bad things" (TM) with them because they typically have permission to do so. But the same is true for pods created by a stateful set.

fejta-bot · 2020-04-27T08:02:09Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

xing-yang · 2020-05-13T19:21:10Z

I think this KEP can be closed now that we are going with this new one: #1701.

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 13, 2019

k8s-ci-robot requested review from childsb and saad-ali December 13, 2019 10:06

k8s-ci-robot requested a review from msau42 December 13, 2019 10:06

k8s-ci-robot added kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/storage Categorizes an issue or PR as relevant to SIG Storage. labels Dec 13, 2019

csi inline volume size: initial draft

4bd8edc

pohly force-pushed the csi-inline-volume-size branch from b4e3be6 to 4bd8edc Compare December 13, 2019 10:07

pohly added a commit to pohly/enhancements that referenced this pull request Dec 13, 2019

storage capacity: remove fsSize API proposal

06064c7

It was moved to kubernetes#1409.

pohly mentioned this pull request Dec 13, 2019

Storage Capacity Constraints for Pod Scheduling #1353

Merged

satoru-takeuchi mentioned this pull request Dec 18, 2019

k8s KEP: comment on a KEP to support CSI inline volume size cybozu-go/neco#703

Closed

4 tasks

satoru-takeuchi reviewed Dec 18, 2019

View reviewed changes

inline volume size: publish_context -> volume_context

f5cfb2d

This was found by Satoru Takeuchi and also fixed by him in kubernetes-csi/docs#249.

satoru-takeuchi approved these changes Dec 20, 2019

View reviewed changes

ymmt2005 mentioned this pull request Jan 9, 2020

Ephemeral volume topolvm/topolvm#75

Closed

4 tasks

pohly force-pushed the csi-inline-volume-size branch 2 times, most recently from ea36596 to a75f17c Compare January 17, 2020 10:40

k8s-ci-robot assigned saad-ali Jan 17, 2020

pohly mentioned this pull request Jan 20, 2020

Storage Capacity Tracking #1472

Closed

4 tasks

inline volume size: move to implementable

1a557c6

This adds the missing details (test plan, implementation history). Also clarifies what exactly the string representation of the size is when passed to the CSI driver.

pohly force-pushed the csi-inline-volume-size branch from a75f17c to 1a557c6 Compare January 21, 2020 08:20

msau42 reviewed Jan 22, 2020

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 28, 2020

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 27, 2020

pohly closed this May 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CSI inline volume size #1409

CSI inline volume size #1409

pohly commented Dec 13, 2019 •

edited

Loading

k8s-ci-robot commented Dec 13, 2019

pohly commented Dec 13, 2019

k8s-ci-robot commented Dec 13, 2019

satoru-takeuchi Dec 18, 2019

pohly Dec 20, 2019

satoru-takeuchi Dec 18, 2019

pohly Dec 20, 2019

satoru-takeuchi commented Dec 18, 2019

satoru-takeuchi left a comment

pohly commented Jan 17, 2020

msau42 Jan 22, 2020

pohly Jan 23, 2020

pohly commented Jan 28, 2020

fejta-bot commented Apr 27, 2020

xing-yang commented May 13, 2020

CSI inline volume size #1409

CSI inline volume size #1409

Conversation

pohly commented Dec 13, 2019 • edited Loading

k8s-ci-robot commented Dec 13, 2019

pohly commented Dec 13, 2019

k8s-ci-robot commented Dec 13, 2019

satoru-takeuchi Dec 18, 2019

Choose a reason for hiding this comment

pohly Dec 20, 2019

Choose a reason for hiding this comment

satoru-takeuchi Dec 18, 2019

Choose a reason for hiding this comment

pohly Dec 20, 2019

Choose a reason for hiding this comment

satoru-takeuchi commented Dec 18, 2019

satoru-takeuchi left a comment

Choose a reason for hiding this comment

pohly commented Jan 17, 2020

msau42 Jan 22, 2020

Choose a reason for hiding this comment

pohly Jan 23, 2020

Choose a reason for hiding this comment

pohly commented Jan 28, 2020

fejta-bot commented Apr 27, 2020

xing-yang commented May 13, 2020

pohly commented Dec 13, 2019 •

edited

Loading