Support for Ephemeral Local Volumes

[arjunrn]

Is your feature request related to a problem? Please describe.
The EBS driver currently only support Persistent Volumes. We run several workloads which would benefit from having support for Ephemeral Local Volumes where we don't need to create persistent volumes for Jobs, Deployments or any other custom controllers.

Describe the solution you'd like in detail
The EBS supports Ephemeral Volumes as well.

Describe alternatives you've considered
None

[leakingtapan]

What kind of Ephemeral Volumes are you looking for? EBS is by nature a persistent volume

[jgallucci32]

This sounds like the inverse of the problem the EBS CSI driver is trying to solve. If you do not require volumes for Job, Deployments, or any other custom controllers, then why use a storage class at all?

[arjunrn]

@leakingtapan For Kubernetes jobs which need some kind of storage for computations generally a PVC has to be provisioned and then mounted to in the job pod. The volume is not provisioned automatically when the pod is created but has to be created separately. The lifecycle of the volume is also independent of the pod. Also if multiple instances of the job have to be run concurrently it is not possible with PVC. Inline ephemeral volumes try to solve this issue by dynamically creating volumes in such cases.

[leakingtapan]

You are right about the ephemeral volume. But is it designed to support cloud provider provisioned persistence volume and tied the volume lifecycle with pod lifecycle? @msau42

[arjunrn]

@leakingtapan I suspected it wont be supported. The docs over here say:

The driver will not receive separate gRPC calls for provisioning, attaching, detaching, and deleting of volumes.

But I was hoping to that the maintainer might know more.

[msau42]

There is a proposal being designed in kubernetes that will allow any persistent volume driver to be used for ephemeral use cases without changing the CSI driver implementation: kubernetes/enhancements#1701

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[palsivertsen]

/remove-lifecycle stale

I'm setting up a service where I have a RW primary set up in a StatefulSet using a PV and several RO followers. It's a read heavy service by nature so only the followers are used by other services. It would be nice if the followers could have a dedicated volume that follows the life cycle of the pod for caching.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[apryiomka]

PVs and PVCs do not solve the problem of ephemeral volumes. As the name suggest, the volume acts as temp storage while the pod is running. Deployments come with replica sets and some volumes cannot be mounted across multiple replicas running on different nodes or AZs. The purpose of ephemeral storage is to provide a temp local files storage for running pods like map reduce jobs or file based merge sorts. Creation of such a storage should be a unobtrusive process and not require creating VPs beforehand.

[msau42]

The generic ephemeral volumes feature which is alpha in Kubernetes 1.19 should address this use case. It creates PVCs underneath so any CSI driver that supports PVCs will work without any special support. And the lifetime of the PVC follows the lifetime of the Pod.

[arjunrn]

Closing this issue because the ephemeral volumes backed by PVCs is now directly supported by Kubernetes as mentioned in @msau42 's comment.

[pkit]

@msau42 @arjunrn I'm not sure I follow
"generic ephemeral volumes" docs specifically say that the backing storage must be local to the node.
Which is not the case for EBS-backed volumes.

[msau42]

I'm not sure I follow "generic ephemeral volumes" docs specifically say that the backing storage must be local to the node. Which is not the case for EBS-backed volumes.

Those docs are for the "csi inline volume" feature, which is different (sorry the names are too similar).

Here are the docs for "generic ephemeral volumes": https://kubernetes.io/docs/concepts/storage/ephemeral-volumes/#generic-ephemeral-volumes