Test kserve with updated Operator

README.md

@@ -31,7 +31,7 @@ Any components that were removed with the update to ODH 1.4 have been relocated
 
 | Manifest Version | ODH Dashboard | ODH Notebook Controller | ODH Notebook Images | Data Science Pipelines | ModelMesh |
 | ---------------- | ------------- | ----------------------- | ------------------- |----------------------- | --------- |
-| master           | v2.11.0       | v1.6                    | 2023a               | v0.2.2                 | v0.11.0   |
+| master           | v2.12.0       | v1.6                    | 2023a               | v0.2.2                 | v0.11.0   |
 
 Release notes and component versions for each ODH release is available on [opendatahub.io](https://opendatahub.io/blog/?type=release)
 

kserve/OWNERS

@@ -0,0 +1,17 @@
+approvers:
+  - anishasthana
+  - danielezonca
+  - heyselbi
+  - israel-hdez
+  - Jooho
+  - VedantMahabaleshwarkar
+  - Xaenalt
+
+reviewers:
+  - anishasthana
+  - danielezonca
+  - heyselbi
+  - israel-hdez
+  - Jooho
+  - VedantMahabaleshwarkar
+  - Xaenalt

kserve/README.md

@@ -0,0 +1,137 @@
+# KServe
+
+KServe comes with one component:
+
+1. [KServe](#KServe)
+
+
+## KServe
+
+Contains deployment manifests for the KServe controller.
+
+- [kserve-controller](https://github.com/opendatahub-io/kserve)
+  - Forked upstream kserve/kserve repository
+
+
+## KServe Architecture
+
+A complete architecture can be found at https://kserve.github.io/website/0.10/modelserving/control_plane.
+
+KServe Control Plane: Responsible for reconciling the InferenceService custom resources. It creates the Knative serverless deployment for predictor, transformer, explainer to enable autoscaling based on incoming request workload including scaling down to zero when no traffic is received. When raw deployment mode is enabled, control plane creates Kubernetes deployment, service, ingress, HPA.
+
+
+## Original manifests
+
+> ❗️Note: Unfortunately, `kfctl` used an outdated version of `kustomize` which cannot process some fields in the KServe manifests.  
+> Thus, we are pre-building the KServe manifests to the [kserve-built](./kserve-built) folder. The [patches](#patches) then use
+> those pre-built KServe manifests to apply our kustomizations.
+
+KServe also uses `kustomize` so we can (indirectly) use [their manifests](https://github.com/opendatahub-io/kserve/tree/master/config).
+
+* `default` is the entrypoint for CRDs, KServe controller and RBAC resources.
+
+The [pre-built KServe manifests](./kserve-built/kserve-built.yaml) are directly referenced in [base/kustomization.yaml](./base/kustomization.yaml).
+
+
+### Updating the manifests
+
+Run the script in [hack](./hack) to manually update the pre-built manifests from the upstream manifests. [this file](./hack/kustomization.yaml) defines the version that is being used:
+
+```bash
+hack/build-kserve-manifests.sh
+```
+```text
+Building KServe manifests
+KServe manifests fetched from upstream and assembled into /odh-manifests/kserve/kserve-built/kserve-built.yaml
+```
+
+
+## Patches
+
+There are patches defined in [kserve/base](./base) to update images and tags for the KServe resources.
+
+
+## Installation process
+
+Following are the steps to install Model Mesh as a part of OpenDataHub install:
+
+1. Install the OpenDataHub operator.
+2. Make sure you install Service Mesh and Serverless components and configure them appropriately.
+See [OCP official instructions](https://docs.openshift.com/serverless/1.29/integrations/serverless-ossm-setup.html) and the 
+3. related documentation for [KServe on Openshift](https://github.com/kserve/kserve/blob/master/docs/OPENSHIFT_GUIDE.md#installation-with-service-mesh) from the kserve repo for more.
+4. Create a KfDef that includes the KServe components and runtimes.
+
+```
+apiVersion: kfdef.apps.kubeflow.org/v1
+kind: KfDef
+metadata:
+  name: opendatahub
+  namespace: opendatahub
+spec:
+  applications:
+    - kustomizeConfig:
+        repoRef:
+          name: manifests
+          path: odh-common
+      name: odh-common
+    - kustomizeConfig:
+        repoRef:
+          name: manifests
+          path: kserve
+      name: kserve
+  repos:
+    - name: manifests
+      uri: https://api.github.com/repos/opendatahub-io/odh-manifests/tarball/master
+  version: master
+```
+
+5. You can now create a new project.
+
+6. Make sure that you have a runtime defined in your target namespace (you can use a [template](https://github.com/opendatahub-io/odh-dashboard/blob/main/manifests/modelserving/ovms-ootb.yaml) in ODH).
+
+```yaml
+apiVersion: serving.kserve.io/v1alpha1
+kind: ServingRuntime
+metadata:
+  name: example-runtime
+spec:
+...
+```
+More information in the [KServe docs](https://kserve.github.io/website/0.10/modelserving/servingruntimes/).
+
+7. Create an `InferenceService` CR in your target namespace.
+
+
+## Using KServe in ODH
+
+You can use the `InferenceService` examples from KServe. Make sure to include the additional annotation for OpenShift Service Mesh:
+
+```yaml
+metadata:
+  annotations:
+    sidecar.istio.io/inject: "true"
+    sidecar.istio.io/rewriteAppHTTPProbers: "true"
+    serving.knative.openshift.io/enablePassthrough: "true"
+```
+
+Example:
+
+```yaml
+apiVersion: "serving.kserve.io/v1beta1"
+kind: "InferenceService"
+metadata:
+  name: "sklearn-iris"
+  namespace: kserve-demo
+  annotations:
+    sidecar.istio.io/inject: "true"
+    sidecar.istio.io/rewriteAppHTTPProbers: "true"
+    serving.knative.openshift.io/enablePassthrough: "true"
+spec:
+  predictor:
+    model:
+      runtime: <your-runtime>
+      modelFormat:
+        name: sklearn
+      storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"
+```
+

kserve/base/inferenceservice-config-patch.yaml

@@ -0,0 +1,60 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: inferenceservice-config
+  namespace: kserve
+data:
+  explainers: |-
+    {
+        "alibi": {
+            "image" : "$(kserve-alibi-explainer)",
+            "defaultImageVersion": "$(kserve-alibi-explainer-version)"
+        },
+        "art": {
+            "image" : "$(kserve-art-explainer)",
+            "defaultImageVersion": "$(kserve-art-explainer-version)"
+        }
+    }
+  storageInitializer: |-
+    {
+        "image" : "$(kserve-storage-initializer)",
+        "memoryRequest": "100Mi",
+        "memoryLimit": "1Gi",
+        "cpuRequest": "100m",
+        "cpuLimit": "1",
+        "storageSpecSecretName": "storage-config",
+        "enableDirectPvcVolumeMount": false
+    }
+  logger: |-
+    {
+        "image" : "$(kserve-agent)",
+        "memoryRequest": "100Mi",
+        "memoryLimit": "1Gi",
+        "cpuRequest": "100m",
+        "cpuLimit": "1",
+        "defaultUrl": "http://default-broker"
+    }
+  batcher: |-
+    {
+        "image" : "$(kserve-agent)",
+        "memoryRequest": "1Gi",
+        "memoryLimit": "1Gi",
+        "cpuRequest": "1",
+        "cpuLimit": "1"
+    }
+  agent: |-
+    {
+        "image" : "$(kserve-agent)",
+        "memoryRequest": "100Mi",
+        "memoryLimit": "1Gi",
+        "cpuRequest": "100m",
+        "cpuLimit": "1"
+    }
+  router: |-
+    {
+        "image" : "$(kserve-router)",
+        "memoryRequest": "100Mi",
+        "memoryLimit": "1Gi",
+        "cpuRequest": "100m",
+        "cpuLimit": "1"
+    }

kserve/base/kserve-controller-manager-patch.yaml

@@ -0,0 +1,19 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: kserve-controller-manager
+  namespace: kserve
+spec:
+  template:
+    spec:
+      containers:
+        # Change the value of image field below to your controller image URL
+        - image: $(kserve-controller)
+          name: manager
+          env:
+            - name: POD_NAMESPACE
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.namespace
+            - name: SECRET_NAME
+              value: kserve-webhook-server-cert

kserve/base/kustomization.yaml

@@ -0,0 +1,79 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - ../kserve-built
+
+namespace: opendatahub
+
+patches:
+  - path: kserve-controller-manager-patch.yaml
+  - path: inferenceservice-config-patch.yaml
+
+configMapGenerator:
+- envs:
+  - params.env
+  name: kserve-parameters
+
+generatorOptions:
+  disableNameSuffixHash: true
+
+vars:
+- fieldref:
+    fieldpath: data.kserve-controller
+  name: kserve-controller
+  objref:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kserve-parameters
+- fieldref:
+    fieldpath: data.kserve-alibi-explainer
+  name: kserve-alibi-explainer
+  objref:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kserve-parameters
+- fieldref:
+    fieldpath: data.kserve-alibi-explainer-version
+  name: kserve-alibi-explainer-version
+  objref:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kserve-parameters
+- fieldref:
+    fieldpath: data.kserve-art-explainer
+  name: kserve-art-explainer
+  objref:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kserve-parameters
+- fieldref:
+    fieldpath: data.kserve-art-explainer-version
+  name: kserve-art-explainer-version
+  objref:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kserve-parameters
+- fieldref:
+    fieldpath: data.kserve-storage-initializer
+  name: kserve-storage-initializer
+  objref:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kserve-parameters
+- fieldref:
+    fieldpath: data.kserve-agent
+  name: kserve-agent
+  objref:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kserve-parameters
+- fieldref:
+    fieldpath: data.kserve-router
+  name: kserve-router
+  objref:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kserve-parameters
+
+configurations:
+  - params.yaml

kserve/base/params.env

@@ -0,0 +1,8 @@
+kserve-controller=quay.io/opendatahub/kserve-controller:v0.10.2
+kserve-alibi-explainer-version=v0.10.2
+kserve-alibi-explainer=quay.io/opendatahub/kserve-alibi-explainer
+kserve-art-explainer-version=latest
+kserve-art-explainer=quay.io/opendatahub/kserve-art-explainer
+kserve-agent=quay.io/opendatahub/kserve-agent:v0.10.2
+kserve-router=quay.io/opendatahub/kserve-router:v0.10.2
+kserve-storage-initializer=quay.io/opendatahub/kserve-storage-initializer:v0.10.2

kserve/base/params.yaml

@@ -0,0 +1,5 @@
+varReference:
+  - path: spec/template/spec/containers/image
+    kind: Deployment
+  - path: data
+    kind: ConfigMap

kserve/hack/build-kserve-manifests.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+
+# This script is used to fetch KServe manifests from github.com/opendatahub-io/kserve
+# and bundles them into one big assembled `kserve-built-yaml` file.
+# To update the version, update `hack/kustomization.yaml` and re-run the script with:
+#
+# $ hack/build-kserve-manifests.sh
+
+set -Eeuo pipefail
+
+echo "Building KServe manifests"
+script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+output_dir="$script_dir/../kserve-built"
+
+command -v kustomize >/dev/null 2>&1 || echo >&2 "kustomize is not installed. Please install kustomize in order to proceed"
+
+kustomize build "$script_dir" > "$output_dir"/kserve-built.yaml
+
+echo "KServe manifests fetched from upstream and assembled into $output_dir/kserve-built.yaml"

kserve/hack/kustomization.yaml

@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - github.com/opendatahub-io/kserve/config/default?ref=release-0.10