KEP-3162: Add Deallocate and PostStopContainer to Device Manager API

[zvonkok]

Add Deallocate and PostStopContainer to Device Manager API #1948

Added the original Device Manager design doc as reference,
this KEP will add an addendum and expand the API. See #1949 (comment) @RenaudWasTaken FYI

Issue: #3162

Related Issues:

• No Device Plugin Callback for Releasing devices against Allocate kubernetes#86539
• Device Plugin API feature: Add Deallocate and PostStopContainer to device plugin API #1948

Related PRs:

• KEP-1948: Adding KEP for allowing deallocate in device plugin API call #1949
• Kubelet device plugin APIs: Deallocate kubernetes#59110
• Add release api in device plugin kubernetes#87223
• Add Deallocate and PostStopContainer to device plugin API kubernetes#91190
• KEP-1948: Device Plugin API feature: Adding KEP for allowing deallocate in device plugin API call (take over) #3074

[k8s-ci-robot]

Welcome @zvonkok!

It looks like this is your first PR to kubernetes/enhancements 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/enhancements has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @zvonkok. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: zvonkok
To complete the pull request process, please assign derekwaynecarr, ehashman after the PR has been reviewed.
You can assign the PR to them by writing /assign @derekwaynecarr @ehashman in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• keps/prod-readiness/OWNERS
• keps/sig-node/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[klueska]

/cc klueska

[klueska]

/ok-to-test

[zvonkok]

@fujitatomoya Do you want to share any use-cases from your side? I just want to make sure that we're not missing "anything".
I could create a dedicated "Use Cases" section if needed.

[zvonkok]

Added Use Cases section with the comments from @kad on kubernetes/kubernetes#59110 and the use-case from @sseetharaman6 from kubernetes/kubernetes#91190

[derekwaynecarr]

I think we need to reach some consensus on this KEP and #3064 before proceeding forward on implementation.

[swatisehgal]

/cc

[fujitatomoya]

@zvonkok are you still working on this? just checking.

[zvonkok]

@dchen1107 Please remove this from 1.25 and plan it for 1.26, I am not able to do enough work, right now, to get this in the right shape.

[kikisdeliveryservice]

@dchen1107 @klueska there was a question as to whether or not this replaces #1949

If so, let's close the other one. If you need help, lmk.

Thanks!

[swatisehgal]

@dchen1107 Please remove this from 1.25 and plan it for 1.26, I am not able to do enough work, right now, to get this in the right shape.

@zvonkok Are you planning to work on this in 1.26 release? I would like to make sure that I am capturing the correct state of the device plugin API as part of Device Manager GA graduation planned for 1.26 release.

[fujitatomoya]

@zvonkok friendly ping.

[fujitatomoya]

@klueska asking advice how to proceed this PR. seems that @zvonkok is not responding more than a year. if we want to this PR, we should probably revise another PR from this?

[FengGaoCSC]

@johnbelamaric @zvonkok

friendly ping, How about the current status with this pull request.
that #3064 already merged

[bart0sh]

2All: Would it make sense to close this PR, considering that this functionality can be implemented by creating CDI file and providing CDI device id as an annotation to the Kubelet, or (after #118254 is merged) to return CDI device id in the Allocate response.

/cc @kad

[kad]

IMHO, yes, if CDI support will be available for device plugins, the urgent need on those calls will disappear: the user stories of "cleaning up device state" would be handled by hooks on the CDI side. However, if there are some plugins that would like to clean-up some internal state related to allocations, then probably this KEP still makes sense to implement.

[fujitatomoya]

@bart0sh thanks for the heads-up, i will take a look at it!

[fujitatomoya]

IMHO, yes, if CDI support will be available for device plugins, the urgent need on those calls will disappear: the user stories of "cleaning up device state" would be handled by hooks on the CDI side.

@bart0sh as @kad mentioned above, having CDI support in DevicePlugin works for our use cases.

However, if there are some plugins that would like to clean-up some internal state related to allocations, then probably this KEP still makes sense to implement.

true, but i do not really think of the use cases for this requirement for now.

[elezar]

@zvonkok given that we are driving development of DRA, is this KEP still current? Should we close it instead?

[bart0sh]

Closing as CDI support for device plugins is in Beta now. Feel free to reopen if needed.
/close

[k8s-ci-robot]

@bart0sh: Closed this PR.

In response to this:

Closing as CDI support for device plugins is in Beta now. Feel free to reopen if needed.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.