diff --git a/keps/sig-storage/2848-transfer-volumesnapshots-namespaces/README.md b/keps/sig-storage/2848-transfer-volumesnapshots-namespaces/README.md new file mode 100644 index 00000000000..01742c143b5 --- /dev/null +++ b/keps/sig-storage/2848-transfer-volumesnapshots-namespaces/README.md @@ -0,0 +1,801 @@ + +# KEP-2848: Transfer VolumeSnapshots between namespaces + + + + + + +- [Release Signoff Checklist](#release-signoff-checklist) +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [User Stories (Optional)](#user-stories-optional) + - [Story 1](#story-1) + - [Story 2](#story-2) + - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [Test Plan](#test-plan) + - [Graduation Criteria](#graduation-criteria) + - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) + - [Version Skew Strategy](#version-skew-strategy) +- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) + - [Feature Enablement and Rollback](#feature-enablement-and-rollback) + - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning) + - [Monitoring Requirements](#monitoring-requirements) + - [Dependencies](#dependencies) + - [Scalability](#scalability) + - [Troubleshooting](#troubleshooting) +- [Implementation History](#implementation-history) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Infrastructure Needed (Optional)](#infrastructure-needed-optional) + + +## Release Signoff Checklist + + + +Items marked with (R) are required *prior to targeting to a milestone / release*. + +- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) +- [ ] (R) KEP approvers have approved the KEP status as `implementable` +- [ ] (R) Design details are appropriately documented +- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors) + - [ ] e2e Tests for all Beta API Operations (endpoints) + - [ ] (R) Ensure GA e2e tests for meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) + - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free +- [ ] (R) Graduation criteria is in place + - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) +- [ ] (R) Production readiness review completed +- [ ] (R) Production readiness review approved +- [ ] "Implementation History" section is up-to-date for milestone +- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] +- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes + + + +[kubernetes.io]: https://kubernetes.io/ +[kubernetes/enhancements]: https://git.k8s.io/enhancements +[kubernetes/kubernetes]: https://git.k8s.io/kubernetes +[kubernetes/website]: https://git.k8s.io/website + +## Summary + + +This KEP proposes a method for transferring VolumeSnapshot between namespaces. + +Currently, VolumeSnapshot is bound to specific namespace, and VolumeSnapshotContent is Cluster scoped. The binding relationship between these two storage resources is one-to-one. +This proposal targets allowing users to conveniently move these resources without manual creation +of each independent object (VolumeSnapshot, VolumeSnapshotContent). + +Based on community design sessions, and inspired by [API for PVC Namespace Transfer](https://github.com/kubernetes/enhancements/pull/2110/files), the following is being proposed :- + +- `StorageTransferRequest` resources are created in the source namespace. +- `StorageTransferApproval` resources are created in the target namespace. +- When a controller detects matching request/approval resources, a transfer in initiated. + +## Motivation + + +Allow Kubernetes users to quickly and conveniently transfer VolumeSnapshots across namespaces. +### Goals +- Use CRDs for VolumeSnapshot Namespace Transfer request/approval. +- Edit external SnapshotController to accept/reject/init VolumeSnapshot transfer. +- The target VolumeSnapshot will point to a newly created VolumeSnapshotContent. +- Use Validation Webhook to reject any transfer contains secret. +- Options to be introduced allowing transfer of VolumeSnapshot with secret for specific CSI Drivers (e.g. Ceph). + + + +### Non-Goals + + +- Transfer of resources other than VolumeSnapshot will not be discussed. +- Transfer of PersistentVolumeClaim may be discussed in the future. +- Transfer of VolumeSnapshots which contain secret by default. + +## Proposal + + +`StorageTransferRequest` resource: + +```yaml +apiVersion: v1alpha1 +kind: StorageTransferRequest +metadata: + name: transfer-foo + namespace: target-namespace +spec: + source: + name: foo + namespace: source-namespace + kind: VolumeSnapshot + approvalName: approve-foo + targetName: bar +``` + +`StorageTransferApproval` resource: + +```yaml +apiVersion: v1alpha1 +kind: StorageTransferApproval +metadata: + name: approve-foo + namespace: source-namespace +spec: + source: + name: foo + kind: VolumeSnapshot + targetNamespace: target-namespace + requestName: transfer-foo +``` + +When matching resources are detected, the controller initiates the transfer process. +### User Stories (Optional) + + +Users should be able to transfer VolumeSnapshots across namespaces. This allows users to quickly share snapshots, utilizing data for testing +or backups in separate namespaces. A new VolumeSnapshotContent must be created and bound to the new VolumeSnapshot, as the mapping between VolumeSnapshot and VolumeSnapshotContent is `1:1`. + +#### Story 1 + +- The `prod` namespace contains a PVC that holds user information. VolumeSnapshots are taken of +this PVC on a regular basis. + +- An existing VolumeSnapshot can be transferred to a test namespace. The user initiates the request +by creating a `StorageTransferRequest` in the source namespace. + +- Once created, a `StorageTransferApproval` resource in the target namespace will be created. +If matching resources are found, the VolumeSnapshot controller can init the transfer. + +- When the transfer is successful, a new VolumeSnapshotContent is being created one-to-one mapping between VolumeSnapshot and VolumeSnapshotContent. + + +### Notes/Constraints/Caveats (Optional) + + + +### Risks and Mitigations + + +This KEP proposes the minimal to transfer VolumeSnapshot between namespaces, while avoiding breaking the current API. +Many caveats have been discussed, including but not limited to :- + +- [StorageClass Secrets](https://kubernetes-csi.github.io/docs/secrets-and-credentials-storage-class.html). + +## Design Details + + +A `StorageTransferRequest` named `r` and `StorageTransferApproval` `a` are matched if all of the following conditions are met: + +- `r.metadata.name == a.spec.requestName` +- `r.metadata.namespace == a.spec.targetNamespace` +- `r.spec.source.namespace == a.metadata.namespace` +- `r.spec.source.name == a.spec.source.name` +- `r.spec.source.kind == a.spec.source.kind` +- `r.spec.approvalName == a.metadata.name` + +Once matching is found, the transfer process is initiated by the controller, and it performs the following: + +- Changes source VolumeSnapshot `ReadyToUse` to `false`, so it can not be used. +- Changes VolumeSnapshotContent deletion policy to retain. +- Creates a VolumeSnapshot in the target namespace. +- Creates a new VolumeSnapShotContent. +- Binds the newly created VolumeSnapshotContent to the newly created VolumeSnapshot. +- Changes original VolumeSnapshotContent deletion policy to delete. +- Deletes the source VolumeSnapshot. + +### Test Plan + + +E2E test for VolumeSnapshot: +1. Create a PVC containing data. +2. Create a VolumeSnapshot +3. Wait for the VolumeSnapshot to be `readyToUse`. +4. Transfer the VolumeSnapshot to a new namespace. +5. Create a new PVC from the VolumeSnapshot. +6. Confirm that the data matches what was initially created on the PVC. +7. Confirm deletion of `StorageTransfer*` resources. + +### Graduation Criteria + + + +### Upgrade / Downgrade Strategy + + + +### Version Skew Strategy + + + +## Production Readiness Review Questionnaire + + + +### Feature Enablement and Rollback + + + +###### How can this feature be enabled / disabled in a live cluster? + + + +- [ ] Feature gate (also fill in values in `kep.yaml`) + - Feature gate name: + - Components depending on the feature gate: +- [ ] Other + - Describe the mechanism: + - Will enabling / disabling the feature require downtime of the control + plane? + - Will enabling / disabling the feature require downtime or reprovisioning + of a node? (Do not assume `Dynamic Kubelet Config` feature is enabled). + +###### Does enabling the feature change any default behavior? + + + +###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? + + + +###### What happens if we reenable the feature if it was previously rolled back? + +###### Are there any tests for feature enablement/disablement? + + + +### Rollout, Upgrade and Rollback Planning + + + +###### How can a rollout or rollback fail? Can it impact already running workloads? + + + +###### What specific metrics should inform a rollback? + + + +###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? + + + +###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? + + + +### Monitoring Requirements + + + +###### How can an operator determine if the feature is in use by workloads? + + + +###### How can someone using this feature know that it is working for their instance? + + + +- [ ] Events + - Event Reason: +- [ ] API .status + - Condition name: + - Other field: +- [ ] Other (treat as last resort) + - Details: + +###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? + + + +###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service? + + + +- [ ] Metrics + - Metric name: + - [Optional] Aggregation method: + - Components exposing the metric: +- [ ] Other (treat as last resort) + - Details: + +###### Are there any missing metrics that would be useful to have to improve observability of this feature? + + + +### Dependencies + + + +###### Does this feature depend on any specific services running in the cluster? + + + +### Scalability + + + +###### Will enabling / using this feature result in any new API calls? + + + +###### Will enabling / using this feature result in introducing new API types? + + + +###### Will enabling / using this feature result in any new calls to the cloud provider? + + + +###### Will enabling / using this feature result in increasing size or count of the existing API objects? + + + +###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs? + + + +###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components? + + + +### Troubleshooting + + + +###### How does this feature react if the API server and/or etcd is unavailable? + +###### What are other known failure modes? + + + +###### What steps should be taken if SLOs are not being met to determine the problem? + +## Implementation History + + + +## Drawbacks + + + +## Alternatives + + + +## Infrastructure Needed (Optional) + + diff --git a/keps/sig-storage/2848-transfer-volumesnapshots-namespaces/kep.yaml b/keps/sig-storage/2848-transfer-volumesnapshots-namespaces/kep.yaml new file mode 100644 index 00000000000..7388d352531 --- /dev/null +++ b/keps/sig-storage/2848-transfer-volumesnapshots-namespaces/kep.yaml @@ -0,0 +1,55 @@ +title: Transfer VolumeSnapshot Namespaces +kep-number: 2848 +authors: + - "@elbehery" + - "@huffmanca" +owning-sig: sig-storage +participating-sigs: + - sig-storage +status: provisional +creation-date: 2021-08-02 +reviewers: + - "@xing-yang" + - "@mhenriks" + - "@jsafrane" + - "@mkimuram" + - "@bswartz" +approvers: + - "@xing-yang" + - "@jsafrane" + - "@saad-ali" + +##### WARNING !!! ###### +# prr-approvers has been moved to its own location +# You should create your own in keps/prod-readiness +# Please make a copy of keps/prod-readiness/template/nnnn.yaml +# to keps/prod-readiness/sig-xxxxx/00000.yaml (replace with kep number) +#prr-approvers: + +see-also: + - "https://github.com/kubernetes/enhancements/pull/2326" + - "https://github.com/kubernetes/enhancements/pull/2110" +replaces: + +# The target maturity stage in the current dev cycle for this KEP. +stage: alpha + +# The most recent milestone for which work toward delivery of this KEP has been +# done. This can be the current (upcoming) milestone, if it is being actively +# worked on. +latest-milestone: "v1.23" + +# The milestone at which this feature was, or is targeted to be, at each stage. +milestone: + alpha: "v1.23" + +# The following PRR answers are required at alpha release +# List the feature gate name and the components for which it must be enabled +feature-gates: + - name: VolumeSnapshotNamespaceTransfer + components: + - kubernetes-csi/external-snapshotter (out-of-tree feature gate) +disable-supported: true + +# The following PRR answers are required at beta release +metrics: