Add StatefulSet Volume Expansion Kep #660

SidakM-zz · 2018-12-21T03:21:32Z

Add Kep for StatefulSet Volume Expansion.
Wrote this up quickly after the discussion here:
kubernetes/kubernetes#71384

Feature tracking issue: #661

Let me know if something is missing or if this is the right way to merge this 🤔

SidakM-zz · 2018-12-21T03:25:02Z

mlmhl · 2018-12-21T09:10:10Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+
+### Risks and Mitigations
+
+Under the `ExpandPersistentVolumes` feature, pods referencing a volume must be restarted for file system expansion to occur after the `FileSystemResizePending` condition is True on the persistent volume claim. If the StatefulSet is configured with the `RollingUpdate` update strategy, then the StatefulSet controller would need to wait for the `FileSystemResizePending` condition to be satisfied on each persistent volume claim referenced by the pod it is updating before restarting the pod. This could noticeably increase the update time of a StatefulSet with many replicas. 


Maybe we can check if the ExpandInUsePersistentVolumes feature gate enabled, and restart the pods immediately without waiting controller side resizing finished if it is enabled.

Indeed we can. That would just translate into not running the following check if that feature flag in enabled in the current implementation: https://github.com/kubernetes/kubernetes/pull/71384/files#diff-988743e0eb9313bcfa8fd43ac3a0977fR229

janetkuo · 2018-12-21T21:56:42Z

@kubernetes/sig-apps-feature-requests @kubernetes/sig-storage-api-reviews

janetkuo · 2018-12-21T22:36:19Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+
+While updating a pod, the StatefulSet controller will update a referenced persistent volume claim object if its storage request in the associated `volumeClaimTemplate` has been increased.
+
+The pod will be restarted after the `FileSystemResizePending` condition is True on all updated persistent volume claims it references.


nit: "The StatefulSet controller will delete and recreate the pod after..."

Also, if ExpandInUsePersistentVolumes feature is enabled, we don't need to kill the pod.

I was looking into this right now and to not kill the pod we would have to recognize that only the the volumeClaimTemplate storage requests have been modified and ExpandInUsePersistentVolumes is enabled. This would be achieved by comparing the current and updated version of the statefulsets which are formed by applying the patches in the updateRevision and currentRevision to a representation of the statefulset (in this case it's just named set).

The issue is that the representation of the statefulset these patches are applied to seems to be the updated version of the StatefulSet. This is problematic whilst forming the current representation of the statefulset. For example, if the updated representation of the set has two labels {"label1": "value1", "label2": "value2"} while the current version only has 1 label {"label1": "value1"}. Applying the currentRevision patch yields a set with labels {"label1": "value1", "label2": "value2"} which is not the current version of the StatefulSet. This issue is problematic as it makes comparisons between the currentSet and updateSet unreliable to determine if termination of pods is required(as we are trying to achieve above).

I think we can start with destroying the Pod and recreating it. It's a trade off in that workloads that need to rebuild a large in memory working set from persistent data would greatly benefit from in place resizing. However, all other operations that operate on a StatefulSet Pod currently require that the Pod be destroyed and recreated. If we want to support in place mutation later we can do so.

Alright, that's also how its currently implemented in the PR so I will leave that as is. I'll make changes to this proposal to reflect that.

janetkuo · 2018-12-21T22:42:11Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+While updating a pod, the StatefulSet controller will update a referenced persistent volume claim object if its storage request in the associated `volumeClaimTemplate` has been increased.
+
+The pod will be restarted after the `FileSystemResizePending` condition is True on all updated persistent volume claims it references.
+


VolumeClaimTemplate should be recorded in the StatefulSet's ControllerRevision object as well for rollback.

Being done in implementation forgot to record it here 🤦‍♂️

janetkuo · 2018-12-21T22:45:43Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+
+The apiserver will allow for increases to storage requests in the `volumeClaimTemplates` component of a StatefulSet.
+
+During the StatefulSet update process, the StatefulSet controller will detect an update to a `volumeClaimTemplate` by comparing the updated and current revision of the StatefulSet.


Change should be detected by looking at the StatefulSet's ControllerRevision objects.

The updateSet is formed by applying the updateRevision (ControllerRevision) to the current set. That should be adequate right?

janetkuo · 2018-12-21T22:50:30Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+Under the `ExpandPersistentVolumes` feature, pods referencing a volume must be restarted for file system expansion to occur after the `FileSystemResizePending` condition is True on the persistent volume claim. If the StatefulSet is configured with the `RollingUpdate` update strategy, then the StatefulSet controller would need to wait for the `FileSystemResizePending` condition to be satisfied on each persistent volume claim referenced by the pod it is updating before restarting the pod. This could noticeably increase the update time of a StatefulSet with many replicas. 
+
+A potential mitigation would be the eventual adoption of the `ExpandInUsePersistentVolumes` alpha feature in Kubernetes v1.11 which enables file system expansion on volumes being used by a pod.  
+


It's optional, but you can include the name of the feature gate of this new feature in this proposal, e.g. StatefulSetVolumeExpansion (just a candidate, feel free to propose other names).

StatefulSetVolumeExpansion sound good to me :)

msau42 · 2018-12-26T22:02:19Z

/assign @gnufied
who is looking at potential changes to the online volume resizing feature

gnufied · 2019-01-03T16:43:14Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+
+While updating a pod, the StatefulSet controller will update a referenced persistent volume claim object if its storage request in the associated `volumeClaimTemplate` has been increased.
+
+The Statefulset controller will delete and recreate the pod after the `FileSystemResizePending` condition is True on all updated persistent volume claims it references. However, if the `ExpandInUsePersistentVolumes` feature is enabled, then deleting the pod is unnecessary to complete file system expansion on updated persistent volumes.


I am not sure I like this bit. It kinda circumvents the fact that currently volume resizing is offline by default and it kinda implements a redundant mechanism that will become unnecessary when online resizing graduates to beta.

At least for alpha feature I propose skipping this mechanism because once we implement this - user will come to rely on it and it will be harder to go back on it. We should also recognize the fact that certain volume types can not be resized when in-use (like AzureDisk for example). Currently this information is surfaced to the user via PVC events if a PVC can't be resized when used in a pod.

I am not sure I like this bit. It kinda circumvents the fact that currently volume resizing is offline by default and it kinda implements a redundant mechanism that will become unnecessary when online resizing graduates to beta.

Not sure what mechanism you are referring to here. Is it the mechanism of detecting the Alpha feature being enabled and not killing the pod? Or are you suggesting that we don't kill the pod in any case since online resizing will eventually graduate to beta.

Yeah I am suggesting we skip killing the pods in stateful set controller since online resizing will eventually graduate to beta (most likely in 1.14 release itself).

Oh ok makes sense. I was just confused because it is killing the pod by default and a mechanism would need to be implemented to ensure that the pod is not killed at the end of the update loop*.

We should also recognize the fact that certain volume types can not be resized when in-use (like AzureDisk for example). Currently this information is surfaced to the user via PVC events if a PVC can't be resized when used in a pod.

So do we want to make an exception in this case to kill the pod from the controller or just update and let the event surface on the PVCs(and possibly add an additional indicator) ?

I'd like to start with always restarting the Pod after the volume increase request is fulfilled. (As below) the workloads controllers don't to in place updates for any other container or Pod modifications today. We can (maybe should) consider optimizing to detect and implement in place updates at a later point in time.

kow3ns · 2019-01-17T00:31:34Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+
+The Statefulset controller will delete and recreate the pod after the `FileSystemResizePending` condition is True on all updated persistent volume claims it references. However, if the `ExpandInUsePersistentVolumes` feature is enabled, then deleting the pod is unnecessary to complete file system expansion on updated persistent volumes.
+
+The functionality provided by this enhancement will be gated by the `StatefulSetVolumeExpansion` feature gate.


You need to think about the patch mechanism used to apply controller revisions and the implications to rollback (both of StatefulSets and Kubernetes API Servers) when this feature is enabled. Also, you to think about modifications to validation that are necessary to allow mutation of the VolumeClaimsTemplate and how that effects backward compatibility during upgrade. It may be necessary to relax the constraint in the API server one release prior to adding the expansion feature into StatefulSet to enable upgrade in multi-API Server clusters and to allow for safe downgrades/rollbacks.

In the implementation PR, the apiserver only allows increases to storageRequests if the feature is enabled (relaxing validation). Mentioned it here in this proposal.

I don't think there should be issues with backwards compatibility for the api, unless a client has (possibly inadvertently) built functionality that relied upon receiving an error response while modifying storageRequests in volumeClaimTemplates before this feature is added.

In regards to patches, for the implementation PR I just added the volumeClaimTemplates to the patch. However, rollbacks seem to be a problem since only volume expansion is supported. So rolling back to a previous revision after expanding a volume would imply the client is attempting to shrink volumes (which is unsupported). Currently the apiserver will err if the client attempts such a rollback and relay to the client that storage requests can only be increased. This can be an issue if the client has attempted a volume expansion, which for some reason failed on all PVCs, and then attempts to rollback the change. In all other cases of rolling back volume expansion changes (partial or full successes) I'm not sure if a rollback should be allowed. On the other hand, this can be problematic if the client attempts to rollback some other change but is prevented from using that functionality as they had also expanded a volumeClaimTemplate in the same or any following revision.

For PVC expansion feature, we also check if expansion is explicitly enabled in SC via an admission controller. Will this be possible to do that here for Stateful sets? It looks like - if feature gate is enabled volume expansion will generally be available for all volume types which is problematic (since not all volume types support expansion).

The other tricky bit this design does not address is - how it will handle expansion of volumes that only can be expanded offline. For PVC themselves, it is not that big of deal. But for stateful sets - user must take the Pod offline before volume that the pod was using can be expanded. So in effect - will this mean scaling entire stateful set to 0?

The other tricky bit this design does not address is - how it will handle expansion of volumes that only can be expanded offline. For PVC themselves, it is not that big of deal. But for stateful sets - user must take the Pod offline before volume that the pod was using can be expanded. So in effect - will this mean scaling entire stateful set to 0?

As it says in the design The Statefulset controller will delete and recreate the pod after the FileSystemResizePending condition is True on all updated persistent volume claims it references.. Thus it will wait until that condition is reached before deleting the pod. This is how I have it implemented in the PR. This way the pod will continue to be available until all updated PVCs that must be expanded offline have the FileSystemResizePending condition (and be deleted and recreated after that).

Let me know if this addresses that concern.

For PVC expansion feature, we also check if expansion is explicitly enabled in SC via an admission controller. Will this be possible to do that here for Stateful sets? It looks like - if feature gate is enabled volume expansion will generally be available for all volume types which is problematic (since not all volume types support expansion).

I did see the PersistentVolumeClaimResize admission controller when initially implementing this. I wasn't sure if it would be necessary to build a similar(perhaps redundant) admission controller for a statefulset as when the statefulset controller eventually attempts to update the pvc it would catch the error sent by the admission controller and the update to the statefulset would not succeed. On the other hand building a specific admission controller for this would ensure that such an update to a statefulset is never validated.

Let me know if this addresses that concern.

Hmm not entirely. Thing is for some volume types(Azure Disk for example), the volume can not be expanded via control-plane call if volume is being used. So literally no expansion can happen to volume while it is in use. The user must first detach the volume from the node before he can call Azure API call that expands the volume. So in this case - there will not be a FileSystemResizePending added to PVC because control-plane call itself has not succeeded (and will not succeed until volume is taken offline).

I wasn't sure if it would be necessary to build a similar(perhaps redundant) admission controller for a statefulset as when the statefulset controller eventually attempts to update the pvc it would catch the error sent by the admission controller and the update to the statefulset would not succeed.

I think, this is already too late. api-server has at this point "accepted" the API request that it can not fulfill. Will stateful set controller not retry PVC update if first update has failed? The idea behind rejecting the request as early as possible is - controllers shouldn't have to keep retrying it. It may be worth expanding existing admission controller to reject stateful set updates in similar fashion.

Thing is for some volume types(Azure Disk for example), the volume can not be expanded via control-plane call if volume is being used. So literally no expansion can happen to volume while it is in use.

Oh ok. I read through this PR and I think I get what you mean now. I see two issues with trying to get it to work with such volumes types.

For each pvc template in the statefulset it would now be necessary to query the api for its storage class to find the provisioner. Then it would be possible to determine how to apply the update. This is doable.

The statefulset controller is structured so that all updates to associated resources and the pod spec occur before the pod it deleted and recreated (as this makes sense in almost every context when updating a statefulset). However, to support the volume types above we would be required to delete the pod before its associated PVCs can be updated. This changes/breaks the flow of how the statefulset is currently updated even though it should be possible to implement.

So in effect - will this mean scaling entire stateful set to 0?

Well since there is a 1:1 map from PVC to PV monotonic updates should still be possible. (delete pod, update pvc request, wait until condition reached, restart pod).

@kow3ns how do you think this should this be addressed

kow3ns · 2019-01-17T00:35:33Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+
+A potential mitigation would be the eventual adoption of the `ExpandInUsePersistentVolumes` feature, which enables file system expansion on in-use volumes thus, making it unecessary to delete and recreate any refrencing pods during volume expansion.
+
+## Graduation Criteria


Might want to add a bit more (as suggested above) about we plan to roll the feature out and how we plan to test it.

kow3ns · 2019-01-17T00:37:10Z

keps/sig-apps/0034-20181220-statefulset-volume-expansion.md

+
+Move to Alpha after initial implementation and approvals.
+
+Move to Beta once feature has been in Alpha for a set duration with no issues being reported.


I'd also like a notional outline of GA graduation criteria. Another concern that should be addressed is that, as the modification effects a GA API, enabling by default at Beta will effectively release it globally. This should at least be called out.

What is a notional outline? 😅 Is it an outline which attempts to identify possible scenarios and the suggested approach to handling them?

kow3ns · 2019-01-17T00:41:09Z

I don't see any API changes proposed so I don't think this needs API approvers. I'd like to discuss this at SIG Apps. Generally, I think its something that is necessary to do in order to support the volume resizing feature implemented at that storage layer, and I think we should (pending a few changes) merge it as accepted and work on getting it to implementable.
@mattfarina @prydonius

justaugustus

Please remove any references to NEXT_KEP_NUMBER and rename the KEP to just be the draft date and KEP title.
KEP numbers will be obsolete once #703 merges.

gnufied · 2019-01-23T23:31:24Z

@kow3ns since this relies on relaxing an API validation, it was decided to run this by api reviewers as well.

fejta-bot · 2020-05-22T14:57:25Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

ayanamist · 2020-05-23T14:43:35Z

/remove-lifecycle stale

fejta-bot · 2020-08-21T15:22:48Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

k8s-ci-robot · 2020-08-21T15:22:52Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

fejta-bot · 2020-09-25T14:01:02Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

fejta-bot · 2020-10-25T14:44:27Z

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

k8s-ci-robot · 2020-10-25T14:44:41Z

@fejta-bot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

travisghansen · 2020-10-25T14:48:33Z

/reopen

k8s-ci-robot · 2020-10-25T14:48:45Z

@travisghansen: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Bessonov · 2021-01-17T22:04:39Z

/remove-lifecycle rotten

cprivitere · 2021-02-19T17:05:39Z

So...where is this at?

m-yosefpor · 2021-09-22T10:52:15Z

So...where is this at?

See here #2842

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 21, 2018

k8s-ci-robot requested review from calebamiles and kow3ns December 21, 2018 03:21

k8s-ci-robot assigned janetkuo Dec 21, 2018

SidakM-zz mentioned this pull request Dec 21, 2018

Support Volume Expansion Through StatefulSets #661

Open

mlmhl reviewed Dec 21, 2018

View reviewed changes

janetkuo added the sig/storage Categorizes an issue or PR as relevant to SIG Storage. label Dec 21, 2018

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API labels Dec 21, 2018

janetkuo reviewed Dec 21, 2018

View reviewed changes

janetkuo added this to the v1.14 milestone Dec 21, 2018

This was referenced Dec 21, 2018

[Need-Feedback] Support Volume Expansion Through StatefulSets kubernetes/kubernetes#71384

Closed

Support PVC Expansion Through StatefulSet volumeClaimTemplates kubernetes/kubernetes#71477

Closed

SidakM-zz force-pushed the statefulset-volume-expansion-kep branch 2 times, most recently from 8920739 to d9ea0b2 Compare December 22, 2018 17:07

k8s-ci-robot assigned gnufied Dec 26, 2018

gnufied reviewed Jan 3, 2019

View reviewed changes

Freyert mentioned this pull request Jan 7, 2019

Ability to update persistent volumes size in StatefulSet kubernetes/kubernetes#72198

Closed

kow3ns reviewed Jan 17, 2019

View reviewed changes

justaugustus requested changes Jan 20, 2019

View reviewed changes

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 22, 2020

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 23, 2020

satoru-takeuchi mentioned this pull request Jun 8, 2020

Increase OSD size when the template size increases rook/rook#5613

Merged

10 tasks

kk-src mentioned this pull request Jun 8, 2020

KEP-0661: StatefulSet volume resize #1848

Closed

cofyc mentioned this pull request Jul 30, 2020

Patch PVC when storage request is increased pingcap/tidb-operator#3004

Closed

k8s-ci-robot added lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Aug 21, 2020

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Aug 21, 2020

justaugustus requested review from justaugustus and removed request for justaugustus August 22, 2020 17:05

WolfspiritM mentioned this pull request Sep 14, 2020

Support expansion of PVCs mongodb/mongodb-kubernetes-operator#197

Closed

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Sep 25, 2020

surajssd mentioned this pull request Oct 8, 2020

Document that PVC created by STS does not support auto expansion of volume kinvolk/lokomotive#1057

Open

k8s-ci-robot closed this Oct 25, 2020

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jan 17, 2021

gnufied mentioned this pull request Jan 19, 2022

KEP-0661: StatefulSet volume resize #2842

Closed

areller mentioned this pull request Jun 20, 2022

KEP-0661: StatefulSet volume resize #3412

Closed


		### Risks and Mitigations

		Under the `ExpandPersistentVolumes` feature, pods referencing a volume must be restarted for file system expansion to occur after the `FileSystemResizePending` condition is True on the persistent volume claim. If the StatefulSet is configured with the `RollingUpdate` update strategy, then the StatefulSet controller would need to wait for the `FileSystemResizePending` condition to be satisfied on each persistent volume claim referenced by the pod it is updating before restarting the pod. This could noticeably increase the update time of a StatefulSet with many replicas.


		While updating a pod, the StatefulSet controller will update a referenced persistent volume claim object if its storage request in the associated `volumeClaimTemplate` has been increased.

		The pod will be restarted after the `FileSystemResizePending` condition is True on all updated persistent volume claims it references.


		The apiserver will allow for increases to storage requests in the `volumeClaimTemplates` component of a StatefulSet.

		During the StatefulSet update process, the StatefulSet controller will detect an update to a `volumeClaimTemplate` by comparing the updated and current revision of the StatefulSet.

		Under the `ExpandPersistentVolumes` feature, pods referencing a volume must be restarted for file system expansion to occur after the `FileSystemResizePending` condition is True on the persistent volume claim. If the StatefulSet is configured with the `RollingUpdate` update strategy, then the StatefulSet controller would need to wait for the `FileSystemResizePending` condition to be satisfied on each persistent volume claim referenced by the pod it is updating before restarting the pod. This could noticeably increase the update time of a StatefulSet with many replicas.

		A potential mitigation would be the eventual adoption of the `ExpandInUsePersistentVolumes` alpha feature in Kubernetes v1.11 which enables file system expansion on volumes being used by a pod.


		While updating a pod, the StatefulSet controller will update a referenced persistent volume claim object if its storage request in the associated `volumeClaimTemplate` has been increased.

		The Statefulset controller will delete and recreate the pod after the `FileSystemResizePending` condition is True on all updated persistent volume claims it references. However, if the `ExpandInUsePersistentVolumes` feature is enabled, then deleting the pod is unnecessary to complete file system expansion on updated persistent volumes.


		The Statefulset controller will delete and recreate the pod after the `FileSystemResizePending` condition is True on all updated persistent volume claims it references. However, if the `ExpandInUsePersistentVolumes` feature is enabled, then deleting the pod is unnecessary to complete file system expansion on updated persistent volumes.

		The functionality provided by this enhancement will be gated by the `StatefulSetVolumeExpansion` feature gate.


		A potential mitigation would be the eventual adoption of the `ExpandInUsePersistentVolumes` feature, which enables file system expansion on in-use volumes thus, making it unecessary to delete and recreate any refrencing pods during volume expansion.

		## Graduation Criteria


		Move to Alpha after initial implementation and approvals.

		Move to Beta once feature has been in Alpha for a set duration with no issues being reported.

Add StatefulSet Volume Expansion Kep #660

Add StatefulSet Volume Expansion Kep #660

Conversation

SidakM-zz commented Dec 21, 2018 • edited by janetkuo Loading

SidakM-zz commented Dec 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janetkuo commented Dec 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SidakM-zz Dec 22, 2018 • edited Loading

Choose a reason for hiding this comment

janetkuo Dec 21, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msau42 commented Dec 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SidakM-zz Jan 4, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SidakM-zz Jan 5, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SidakM-zz Jan 24, 2019 • edited Loading

Choose a reason for hiding this comment

gnufied Jan 24, 2019 • edited Loading

Choose a reason for hiding this comment

SidakM-zz Jan 25, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kow3ns commented Jan 17, 2019

justaugustus left a comment

Choose a reason for hiding this comment

gnufied commented Jan 23, 2019

fejta-bot commented May 22, 2020

ayanamist commented May 23, 2020

fejta-bot commented Aug 21, 2020

k8s-ci-robot commented Aug 21, 2020

fejta-bot commented Sep 25, 2020

fejta-bot commented Oct 25, 2020

k8s-ci-robot commented Oct 25, 2020

travisghansen commented Oct 25, 2020

k8s-ci-robot commented Oct 25, 2020

Bessonov commented Jan 17, 2021

cprivitere commented Feb 19, 2021

m-yosefpor commented Sep 22, 2021

SidakM-zz commented Dec 21, 2018 •

edited by janetkuo

Loading

SidakM-zz Dec 22, 2018 •

edited

Loading

janetkuo Dec 21, 2018 •

edited

Loading

SidakM-zz Jan 4, 2019 •

edited

Loading

SidakM-zz Jan 5, 2019 •

edited

Loading

SidakM-zz Jan 24, 2019 •

edited

Loading

gnufied Jan 24, 2019 •

edited

Loading

SidakM-zz Jan 25, 2019 •

edited

Loading