You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
I store the trained model (I use mnist-svm.joblib in my case) on a PVC, and I have some extra logics to handle the trained model after it is loaded. Therefore, I need to write a custom ServingRuntime to handle it.
To Reproduce
It works well when I follow the doc: Deploy a model stored on a Persistent Volume Claim, I can see the model file mnist-svm.joblib and model-settings.json under the folder of /models/_mlserver_models/, showing as below:
However, I want to write a custom ServingRuntime, so I go on to follow the doc: Python-Based Custom Runtime with MLServer, then create a new ServingRuntime and create an InferenceService for this ServingRuntime. After all these actions, everything is okay, except the inference service can not be True due to "NOT_FOUND" error in inference service, showing below:
Based on the comparison, I think there should be something wrong with the Python-Based Custom Runtime with MLServer when using Persistent Volume Claim to store a trained model.
Expected behavior
Hope that there is an explicit demo to show how to support the use case of python-based custom runtime with MLServer with a model stored on a Persistent Volume Claim. Thanks a lot if it's possible!
The text was updated successfully, but these errors were encountered:
#### Motivation
I'm very happy to quickly install `kserve modelmesh` by following
[quickstart](https://github.com/kserve/modelmesh-serving/blob/main/docs/quickstart.md),
but I encountered the problem when I want to write a python-based custom
`ServingRuntime` for model `mnist-svm.joblib` which is also used in many
guides and docs, and I also opened an [issue
](#494) for it.
However, I can hardly find the complete process which is available
online, including the community. So I pieced together some information
to make it's easier for user to do it.
#### Modifications
1. Add a REAMD for describing the complete process
2. Add a folder for custom ServingRuntime, including the python code,
Dockerfile and required library configuration file.
#### Result
The process for writing a python-based custom `ServingRuntime` is
completed, hope it can be a easy start for fresh user.
---------
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>
Describe the bug
I store the trained model (I use
mnist-svm.joblib
in my case) on a PVC, and I have some extra logics to handle the trained model after it is loaded. Therefore, I need to write a custom ServingRuntime to handle it.To Reproduce
It works well when I follow the doc: Deploy a model stored on a Persistent Volume Claim, I can see the model file
mnist-svm.joblib
andmodel-settings.json
under the folder of/models/_mlserver_models/
, showing as below:However, I want to write a custom ServingRuntime, so I go on to follow the doc: Python-Based Custom Runtime with MLServer, then create a new ServingRuntime and create an InferenceService for this ServingRuntime. After all these actions, everything is okay, except the inference service can not be
True
due to "NOT_FOUND" error in inference service, showing below:Based on the comparison, I think there should be something wrong with the
Python-Based Custom Runtime with MLServer
when usingPersistent Volume Claim
to store a trained model.Expected behavior
Hope that there is an explicit demo to show how to support the use case of python-based custom runtime with MLServer with a model stored on a Persistent Volume Claim. Thanks a lot if it's possible!
The text was updated successfully, but these errors were encountered: