OCI KServe Custom Storage Initializer

Have you ever wondered to deploy an ML model by directly referencing an OCI artifact that contains your model?

This repository provides an example of KServe custom storage initializer that showcases how users can automate the deployment using Kserve of an ML model that is stored as OCI artifact.

Users can then create an InferenceService like this one and the job is done 😄

   apiVersion: "serving.kserve.io/v1beta1"
   kind: "InferenceService"
   metadata:
     name: "sklearn-iris"
   spec:
     predictor:
       model:
         modelFormat:
           name: sklearn
         storageUri: "oci-artifact://quay.io/<path-to-your-greatest-model>"

Note

This repository was mainly intended for demoing purposes

The implementation was inspired by the default KServe storage container, this means that its integration should be pretty straightforward if required at some point.

Quickstart

Please take a look at Get Started guide for a very simple step-by-step example on how to use this OCI custom storage initializer to deploy an ML model stored as OCI artifact using KServe.

Development

Local development

Prerequisites:

• Python
• Poetry

Note

I would suggest to use a Python virtual environment as development environment

Install the dependencies:

make install-deps

Build the python wheel:

make build

Install the python package:

make install

Image build

Build the container image:

make image-build

Push the container image:

make image-push