-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VolumeSource: OCI Artifact and/or Image #4639
Comments
I've had informal discussions about this - there's enough interest IMO to open a KEP & I will present this issue at upcoming sig-node, sig-storage mtgs with a KEP draft /sig node |
Can you use the Volume Populator? It allows you to create a PVC from an extenal data source. https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1495-volume-populators |
Happy to support here from a SIG node perspective. cc @kubernetes/sig-node-proposals |
+1 happy to help from the SIG-Node/CRI and OCI image/distribution spec perspectives.. |
/label lead-opted-in as discussed at SIG Node meeting this week, we will try and see if this can make it to 1.31 |
/stage alpha |
For reference, in KServe a workaround for directly accessing files within an OCI image is currently implemented and available via a sidecar approach ("modelcar") by leveraging root FS system access via the So KServe would be more than happy to leverage such a volume type, and we are happy to support any efforts in this direction. |
/stage alpha |
Hello @sallyom 👋, v1.31 Enhancements team here. Just checking in as we approach enhancements freeze on 02:00 UTC Friday 14th June 2024 / 19:00 PDT Thursday 13th June 2024. This enhancement is targeting for stage Here's where this enhancement currently stands:
For this KEP, most of the above items are taken care of in #4642. We'd need to do the following:
The status of this enhancement is marked as If you anticipate missing enhancements freeze, you can file an exception request in advance. Let me know if you have any questions! Thank you! |
@sallyom Pinging once again as a slight reminder that we're approaching the enhancements freeze deadline on 14th June, this Friday! |
Hi @sallyom @SergeyKanzhelev 👋, 1.31 Enhancements team here, Just a quick friendly reminder as we approach the enhancements freeze in few hours, at 02:00 UTC Friday 14th June 2024 / 19:00 PDT Thursday 13th June 2024. The current status of this enhancement is marked as If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you! |
Hello @sallyom @SergeyKanzhelev 👋, 1.31 Enhancements team here. Unfortunately, this enhancement did not meet requirements for enhancements freeze. If you still wish to progress this enhancement in v1.31, please file an exception request as soon as possible, within three days. If you have any questions, you can reach out in the #release-enhancements channel on Slack and we'll be happy to help. Thanks! |
/milestone clear |
/assign |
Hello @saschagrunert @sallyom 👋, Enhancements team here. Just checking in as we approach enhancements freeze on 02:00 UTC Friday 11th October 2024 / 19:00 PDT Thursday 10th October 2024. This enhancement is targeting stage Here's where this enhancement currently stands:
For this KEP, we would just need to update the following:
The status of this enhancement is marked as If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you! |
@saschagrunert @sallyom, I'm part of the SIG-NODE KEPs wrangler team for this release. It looks like the plan for this release is to push this feature to Beta. Could one of you open a PR to move it forward? |
@saschagrunert is out this week, but I'm going to open a PR in his stead to get it on PRR's purview, then he'll probably correct all the incorrect stuff I've written next week 🙂 |
update: I've opened #4897 |
/stage beta |
/stage alpha I believe we won't make stage progress in 1.32. We can still push for subpath support but I don't think we need to track that here. |
1.32 Enhancements team here. Per the above comments, I've updated the 1.32 status to |
/milestone clear |
Hi 👋 One of the great advantage of the OCI Artifact from the OCI spec, is the possibility to define media_type (and/or artifact_type) with a semantic meaning. This would potentially allow a user to mount only a set of determined layers instead of the whole OCI Image (as it is the case in practice today with alpha), ie something ~similar to: volumes:
- name: volume
image:
reference: quay.io/crio/artifact:v1
layer_mediaType_filter_IN: ["application/x-mlmodel"] # mount only layer(s) from Artifact which is the ML model, not Data layers, etc. or like: volumes:
- name: volume
image:
reference: quay.io/crio/artifact:v1
layer_mediaType_filter_IN: ["application/x", "application/y"] # mount only layer(s) marked for X and Y, and for example ignoring layers of type Z if the Artifact provides them please notice these examples (and naming this As one can imagine, if the single OCI Artifact is comprehensive of many assets duly separated in their own semantic layers, but only some layers are functional to runtime scope, this would be a great advantage for the definition of the Deployment workload. wdyt? I hope this comment is helpful to the wide CNCF community and of interest in the scope of the continued work for this KEP-4639 🚀 |
When mounting other media types, what are the filesystem settings for the mounted layer? Is something extracted from a tar, requiring all Artifacts to be packaged as a tar? Or is the Artifact layer mounted as a file? How is the filename set, owner, permissions, and other filesystem parameters? From my understanding, the KEP authors were not ready to answer those questions and intentionally marked OCI Artifacts as out of scope for now, leaving individual runtimes free to experiment with their own implementations. |
From my perspective, tarballs are a natural starting point for packaging OCI artifact layers, and these layers are generally going to be compatible with OCI image layer types (e.g. in testing the existing alpha-level feature, I'm able to mount artifacts by just changing the mediaType on layers and generating a simple image config). Considering that the current state of the proposal is limited to OCI images, it's not a large leap to extend support for any OCI artifact whose media types are compatible with OCI image layers. The mediatype filtering field would then be an assertion from the user that these mediaTypes can be processed like image layers. Alternatively, we could consider making this explicit in some way -- runtimes understand certain media types (tar, tar+gzip, etc.); does it make sense to enable defining artifact media types as synonyms for "known" mediatypes? |
I think it's a good idea for container runtimes to understand certain media types (tar, tar+gzip, etc.). As |
I concur. I wonder, though, if there is a discussion already in the runtime space, how to adopt OCI artifacts or if it needs to be kicked off? |
I have not seen any recent conversations, but it is a topic that would be good to raise and get additional mindshare |
As you folks already outlined: runtimes are free to do anything with the feature. I don't think that Kubernetes itself should maintain the list of allowed and supported media types, that would be a better fit for the OCI. But said, runtimes like CRI-O are free to support any format which makes sense. |
Enhancement Description
One-line enhancement description (can be used as a release note): VolumeSource: OCI Artifact or OCI Image
Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4639-oci-volume-source
Discussion Link:
Primary contact (assignee): sgrunert@redhat.com, somalley@redhat.com
Responsible SIGs: sig-node, sig-storage
Enhancement target (which target equals to which milestone):
Alpha
k/enhancements
) update PR(s):k/k
) update PR(s):ImageVolumeSource
API kubernetes#125660ImageVolumeSource
implementation kubernetes#125663fsGroupChangePolicy
has no effect kubernetes#126281CanSupport
method kubernetes#126323ImageVolumeSource
node e2e tests kubernetes#126220crictl [create|run] --with-pull
kubernetes-sigs/cri-tools#1464pull-kubernetes-node-crio-cgrpv2-imagevolume-e2e
test job test-infra#33071k/website
) update PR(s):Beta
k/enhancements
) update PR(s):k/k
) update PR(s):k/website
) update(s):Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.
The text was updated successfully, but these errors were encountered: