diff --git a/docs/blog/articles/2022-02-18-KServe-0.8-release.md b/docs/blog/articles/2022-02-18-KServe-0.8-release.md new file mode 100644 index 000000000..82fea6e74 --- /dev/null +++ b/docs/blog/articles/2022-02-18-KServe-0.8-release.md @@ -0,0 +1,144 @@ +--- +title: "Announcing: KServe v0.8" +date: 2022-02-18 +description: " KServe v0.8.0 Release Blog Post" +type: "blog" +--- + +### Authors + **Dan Sun**, **Paul Van Eck**, **Vedant Padwal**, **Andrews Arokiam** on behalf of the KServe Working Group. + +### **Announcing: KServe v0.8** + +_February 18, 2022_ + +Today, we are pleased to announce the v0.8.0 release of KServe! While the last release was focused on the +[transition](https://blog.kubeflow.org/release/official/2021/09/27/kfserving-transition.html) of KFServing to KServe, +this release was focused on unifying the InferenceService API for deploying models on KServe and ModelMesh. + +>**Note**: For current users of KFServing/KServe, please take a few minutes to answer this [short survey](https://kserve-survey.com/) and provide your feedback! + +Now, let's take a look at some of the changes and additions to KServe. + +### **:warning: What’s changed?** + +- **ONNX Runtime Server** has been removed from the supported serving runtime list. KServe by default now uses the **Triton Inference Server** to serve ONNX models. +- KServe’s **PyTorchServer** has been removed from the supported serving runtime list. KServe by default now uses **TorchServe** to serve PyTorch models. +- A few main KServe SDK class names have been changed: + - **KFModel** is renamed to **Model** + - **KFServer** is renamed to **ModelServer** + - **KFModelRepository** is renamed to **ModelRepository** + +### **:rainbow: What's new?** + +Some notable updates are: + +- **ClusterServingRuntime** and **ServingRuntime** CRDs are introduced. Learn more [below](#servingruntimes-and-clusterservingruntimes). +- A new **Model Spec** was introduced to the InferenceService Predictor Spec as a new way to specify models. Learn more [below](#updated-inferenceservice-predictor-spec). +- **Knative 1.0** is now supported and certified for the KServe Serverless installation. +- **gRPC** is now supported for transformer to predictor network communication. +- **TorchServe** Serving runtime has been updated to 0.5.2 which now supports the KServe V2 REST protocol. +- **ModelMesh** now has multi-namespace support, and users can now deploy GCS or HTTP(S) hosted models. + +To see all release updates, check out the KServe [release notes](https://github.com/kserve/kserve/releases/tag/v0.8.0) and ModelMesh Serving [release notes](https://github.com/kserve/modelmesh-serving/releases/tag/v0.8.0)! + +### **ServingRuntimes and ClusterServingRuntimes** + +This release introduces two new CRDs *ServingRuntimes* and *ClusterServingRuntimes* with the only difference between these two is that one is +namespace-scoped and one is cluster-scoped. A ServingRuntime defines the templates for Pods that can serve one or more particular model formats. +Each ServingRuntime defines key information such as the container image of the runtime and a list of the model formats that the runtime supports. + +In previous versions of KServe, supported predictor formats and container images were defined in a +[config map](https://github.com/kserve/kserve/blob/release-0.7/config/configmap/inferenceservice.yaml#L7) in the control plane namespace. +The ServingRuntime CRD should allow for improved flexibility and extensibility for defining or customizing runtimes to how you see fit without having to modify +any controller code or any resources in the controller namespace. + +Several out-of-the-box ClusterServingRuntimes are provided with KServe so that users can continue to use KServe how they did before without having to define the runtimes themselves. + +**Example SKLearn ClusterServingRuntime:** + +```yaml +apiVersion: serving.kserve.io/v1alpha1 +kind: ClusterServingRuntime +metadata: + name: kserve-sklearnserver +spec: + supportedModelFormats: + - name: sklearn + version: "1" + autoSelect: true + containers: + - name: kserve-container + image: kserve/sklearnserver:latest + args: + - --model_name={{.Name}} + - --model_dir=/mnt/models + - --http_port=8080 + resources: + requests: + cpu: "1" + memory: 2Gi + limits: + cpu: "1" + memory: 2Gi + +``` + +### **Updated InferenceService Predictor Spec** + +A new Model spec was also introduced as a part of the Predictor spec for InferenceServices. One of the problems KServe was having was that the InferenceService CRD was +becoming unwieldy with each model serving runtime being an object in the Predictor spec. This generated a lot of field duplication in the schema, +bloating the overall size of the CRD. If a user wanted to introduce a new model serving framework for KServe to support, the CRD would have to be modified, +and subsequently the controller code. + +Now, with the Model spec, a user can specify a model format and optionally a corresponding version. The KServe control plane will automatically select and use the +*ClusterServingRuntime* or *ServingRuntime* that supports the given format. Each *ServingRuntime* maintains a list of supported model formats and versions. If a format has +`autoselect` as `true`, then that opens the *ServingRuntime* up for automatic model placement for that model format. + + +=== "New Schema" + ```yaml + apiVersion: serving.kserve.io/v1beta1 + kind: InferenceService + metadata: + name: example-sklearn-isvc + spec: + predictor: + model: + modelFormat: + name: sklearn + storageUri: s3://bucket/sklearn/mnist.joblib + ``` +=== "Previous Schema" + ```yaml + apiVersion: serving.kserve.io/v1beta1 + kind: InferenceService + metadata: + name: example-sklearn-isvc + spec: + predictor: + sklearn: + storageUri: s3://bucket/sklearn/mnist.joblib + ``` + +The previous way of defining predictors is still supported, however, the new approach will be the preferred one going forward. Eventually, the previous schema, with the framework names as keys in the predictor spec, will be removed. + +### **ModelMesh Updates** + +[ModelMesh](https://developer.ibm.com/blogs/kserve-and-watson-modelmesh-extreme-scale-model-inferencing-for-trusted-ai/) has been in the process of integrating as KServe’s +multi-model serving backend. With the inclusion of the aforementioned ServingRuntime CRDs and the Predictor Model spec, the two projects are now much more aligned, +with continual improvements underway. + +ModelMesh now supports multi-namespace reconciliation. Previously, the ModelMesh controller would only reconcile against resources deployed in the same namespace as the controller. Now, by default, ModelMesh will be able to handle InferenceService deployments in any "modelmesh-enabled" namespace. Learn more [here](https://github.com/kserve/modelmesh-serving/blob/release-0.8/docs/install/install-script.md#setup-additional-namespaces). + +Also, while ModelMesh previously only supported S3-based storage, we are happy to share that ModelMesh now works with models hosted using GCS and HTTP(S). + +### **Join the community** + +- Visit our [Website](https://kserve.github.io/website/) or [GitHub](https://github.com/kserve) +- Join the Slack ([#kubeflow-kfserving]((https://kubeflow.slack.com/join/shared_invite/zt-n73pfj05-l206djXlXk5qdQKs4o1Zkg#/))) +- Attend a [biweekly community meeting on Wednesday 9am PST](https://docs.google.com/document/d/1KZUURwr9MnHXqHA08TFbfVbM8EAJSJjmaMhnvstvi-k/edit#heading=h.4i9fb8ndp9vp) +- View our [developer](https://github.com/kserve/website/blob/main/docs/developer/developer.md) and [doc](https://github.com/kserve/website/blob/main/docs/help/contributor/mkdocs-contributor-guide.md) contribution guides to learn how to make contributions. We are excited to work with you to make KServe better and promote its adoption! + +Thank you for trying out KServe! + diff --git a/mkdocs.yml b/mkdocs.yml index 37e818229..e4133d213 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -74,6 +74,7 @@ nav: - Debugging guide: developer/debug.md - Blog: - Releases: + - KServe 0.8 Release: blog/articles/2022-02-18-KServe-0.8-release.md - KServe 0.7 Release: blog/articles/2021-10-11-KServe-0.7-release.md - Articles: - KFserving Transition: blog/articles/2021-09-27-kfserving-transition.md