Priority/preemption

[davidopp]

Feature Description

• One-line feature description (can be used as a release note): A mechanism for imposing a total order on pods, that determines which pods run and which go pending when the cluster is overcomitted, implemented through a priority scheme, preemption of running pods by the default scheduler, having kubelet eviction order take the priority into account, and modifications to the quota mechanism to take priority into account.
• Primary contact (assignee): @bsalamat
• Responsible SIGs: @kubernetes/sig-scheduling-feature-requests, @kubernetes/sig-node-feature-requests
• Design proposal link (community repo): high-level architecture doc (currently under review/discussion/iteration) here; detailed design doc for first chunk is in Design proposal for adding priority to Kubernetes API community#604
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @davidopp, @derekwaynecarr, @erictune, @k82cn, @timothysc, @wojtek-t
• Approver (likely from SIG/area to which feature belongs): @davidopp
• Initial target stage (alpha/beta/stable) and release (x.y): for 1.7 we anticipate having the detailed design, API (alpha), and implementation of kubelet eviction ordering (alpha); some implementation work for the other pieces will hopefully have begun but will not be in 1.7

ref/ kubernetes/kubernetes#22212
cc/ @dchen1107

Note: this feature is intimately related to #269.

[davidopp]

The minimum we want for 1.7 is an alpha API fo expressing pod priority, and hopefully incorporating that into kubelet pod eviction ordering when out of resource. Most of this feature is not expected to ship in 1.7, especially since the design is still under discussion.

[0xmichalis]

when the cell is overcomitted

This is Borg terminology, right? Many people coming to k8s will not be familiar with Borg so I would suggest switching "cell" to "node".

[davidopp]

You caught me! I meant cluster, not cell. Fixed now.

Glad to know someone is reading this stuff. :)

[obeattie]

Is the document linked in the description (and that of #269) intended to be private? This is a feature which is very important for our deployment. I'm mostly just curious to follow its development, so if this is private for a reason that's fine – I just figure it may be an oversight.

[davidopp]

You can view (and commnt in) the document by joining the kubernetes-dev mailing list
https://groups.google.com/forum/#!forum/kubernetes-dev

Any feedback on the proposal is welcome. (It's still in the proposal phase.)

[obeattie]

Aha, thank you 👍

[idvoretskyi]

@bsalamat please, provide us with the docs PR link (and update the features tracking spreadsheet with it).
/cc @kubernetes/sig-scheduling-feature-requests @davidopp

[bsalamat]

Here is one of the design docs to add pod priority to the API: kubernetes/community#604
More docs to follow.

Task tracker: kubernetes/kubernetes#47604

[spiffxp]

FYI, I couldn't find this in the spreadsheet, I'm pulling the one-line release note from this issue's description

[luxas]

@bsalamat Correct me if I'm wrong, but IIUC, this is a "design only" feature with no code shipping in time for v1.7. Actual implementation will be done as alpha in v1.8.

So based on that this is not in the spreadsheet, the linked task tracker kubernetes/kubernetes#47604 is targeted at v1.8 and the design proposal kubernetes/community#604 isn't merged, I'm moving this out of the v1.7 milestone.

[bsalamat]

@luxas your understanding is correct.

[luxas]

@bsalamat Thanks!

[kacole2]

@davidopp This feature was worked on in the previous milestone, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.12 since there is nothing in the original post. This still has the 1.11 milestone as well so we need to update it accordingly.

If there are any updates, please explicitly ping @justaugustus, @kacole2, @robertsandoval, @rajendar38 to note that it is ready to be included in the Features Tracking Spreadsheet for Kubernetes 1.12.

---

Please note that the Features Freeze is July 31st, after which any incomplete Feature issues will require an Exception request to be accepted into the milestone.

In addition, please be aware of the following relevant deadlines:

• Docs deadline (open placeholder PRs): 8/21
• Test case freeze: 8/28

Please make sure all PRs for features have relevant release notes included as well.

Happy shipping!

[derekwaynecarr]

I am not sure if we want a separate feature issue to track, but we need to promote quota by priority to beta. To ensure that this is understood, I am adding this back to the 1.12 milestone.

[kacole2]

@derekwaynecarr isn't this feature already in beta? therefore it's not graduating. Looking for clarification to see if it's worth tracking because it's not new or graduating.

[k82cn]

@derekwaynecarr , prefer to track in a separate feature to control the scope of this item.

@kacole2 , derekwaynecarr@ is asking "quota by priority" feature, which is also a feature in priority/preemption umbrella.

[kacole2]

@k82cn thanks for the clarification. Lets get a new feature issue added to track it separately so it can be added before feature freeze ends tonight.

[justaugustus]

Sounds like there's a need for a separate tracking feature to allay some confusion.

@k82cn or @derekwaynecarr -- can you open a separate feature for Quota by Priority and ping me when it's updated?

[justaugustus]

@k82cn / @derekwaynecarr -- any update on this?

Sounds like there's a need for a separate tracking feature to allay some confusion.
@k82cn or @derekwaynecarr -- can you open a separate feature for Quota by Priority and ping me when it's updated?

[zparnold]

Hey there! @davidopp I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?

[justaugustus]

@k82cn / @derekwaynecarr --
Any update on docs status for this feature? Are we still planning to land it for 1.12?
At this point, code freeze is upon us, and docs are due on 9/7 (2 days).
If we don't here anything back regarding this feature ASAP, we'll need to remove it from the milestone.

cc: @zparnold @jimangel @tfogo

[tfogo]

For visibility: I got confirmation from @bsalamat that no docs changes are necessary for 1.12

[justaugustus]

Thanks for the update!

[justaugustus]

Just as an update to #268 (comment):

I am not sure if we want a separate feature issue to track, but we need to promote quota by priority to beta. To ensure that this is understood, I am adding this back to the 1.12 milestone.

I'm removing this specific issue from the milestone as we're now tracking Quota by Priority in #612.
/milestone clear

[kacole2]

Hi
This enhancement has been tracked before, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.13. This release is targeted to be more ‘stable’ and will have an aggressive timeline. Please only include this enhancement if there is a high level of confidence it will meet the following deadlines:

• Docs (open placeholder PRs): 11/8
• Code Slush: 11/9
• Code Freeze Begins: 11/15
• Docs Complete and Reviewed: 11/27

Please take a moment to update the milestones on your original post for future tracking and ping @kacole2 if it needs to be included in the 1.13 Enhancements Tracking Sheet

Thanks!

[bsalamat]

There's no graduation plan for 1.13. This feature has been Beta since 1.11. We would like to get more mileage out of it before moving it to GA. We can tentatively target 1.14 for GA.

[pires]

Is this going to be GA in 1.14, then?

[claurence]

@bsalamat Hello - I’m the enhancement’s lead for 1.14 and I’m checking in on this issue to see what work (if any) is being planned for the 1.14 release. Enhancements freeze is Jan 29th and I want to remind that all enhancements must have a KEP

[kacole2]

Hello @bsalamat , I'm the Enhancement Lead for 1.15. Is this feature going to be graduating alpha/beta/stable stages in 1.15? Please let me know so it can be tracked properly and added to the spreadsheet. This will also require a formal KEP to be merged before moving further.

Once coding begins, please list all relevant k/k PRs in this issue so they can be tracked properly.

[kacole2]

Hi @bsalamat , I'm the 1.16 Enhancement Lead/Shadow. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet. If not's graduating, I will remove it from the milestone and change the tracked label.

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

As a reminder, every enhancement requires a KEP in an implementable state with Graduation Criteria explaining each alpha/beta/stable stages requirements.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

[k82cn]

Priority/Preemption is GA now, I think we can close this enhancement now :)

/cc @bsalamat @ahg-g

[k82cn]

/close

[k8s-ci-robot]

@k82cn: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.