-
Notifications
You must be signed in to change notification settings - Fork 114
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ClusterServingRuntime support #81
Comments
Summary of design discussion with @chinhuang007:
Code changes required:
We should have the CSR parts conditional on some flag so that things will work the same as they do now if that CRD does not exist (or controller does not have permisison to list/read them). We may also want to expose and option to explicitly enable/disable use of CSRs in namespace-scope case. |
Add ClusterServingRuntime (CSR) support so that the CSRs can be shared between namespaces and overridden by namespaece level ServingRuntimes (SR). #### Motivation This PR provides the support for ClusterServingRuntime (CSR), addressing [issue ](#81). The use cases and design can be found [here](https://docs.google.com/document/d/1lSqwqmiOeS7rJTtxfSdvxuTK_GzaOwTV5BXfnw2IEJs/edit#heading=h.x1r6y17xd8u7). #### Modifications - ServingRuntime controller watches CSRs - ServingRuntime controller determines which CSRs can be used to create deployments in ModelMesh enabled namespaces - ServingRuntime controller creates deployments using CSRs - ServingRuntime controller decides whether a SR or a CSR to use for the deployment - ServingRuntime controller sets deployment owner=CSR for CSRs - Refactor code to take SR.spec instead of SR and use the spec from either a SR or a CSR. - Update unit tests to work with refactored code Note that changes to install the CSR CRD and required controller roles, and change built-in SRs to be CSRs will be done in a follow-on PR. #### Result ModelMesh supports ClusterServingRuntimes while namespace level ServingRuntimes have overriding power. Contributes to #81 Signed-off-by: Chin Huang <chhuang@us.ibm.com>
Required changes to the controller have now been completed in #241. Remaining tasks:
|
KServe supports cluster-scoped ServingRuntimes called
ClusterServingRuntimes
. These act as the built-in or default serving runtimes accessible to any user/namespace in the cluster. Currently ModelMesh-Serving only considers the the namespace-scopedServingRunimes
. Let's think about how ModelMesh-Serving can handle these cluster-level resources.The text was updated successfully, but these errors were encountered: