-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MLServer model-settings.json uri conversion #40
Comments
Hello @xvnyv. Thanks for including all of these details in the issue! That is strange behavior, it seems that the path is being replicated for some reason, and I don't see how the name of the file should matter 🤔 The model files in
|
Hi @tjohnson31415, thanks for the quick response! I have tried out your suggestions and this is what I have found:
|
Yep, everything there checks out. I was able to reproduce it as well and the adapter code logs the incorrect rewrite directly:
The relevant code in the MLServer adapter is here. I did some additional testing and think that I understand the problem: Since you found the bug, would you like to contribute a fix? |
@tjohnson31415 ohh thanks for finding the issue, that makes sense. Sure, I'll do up a fix once I have some time available. |
#### Motivation As documented in [this issue](#40), there is currently a bug in the conversion of the `uri` provided in the MLServer `model-settings.json` file if the model files are symlinked before the uri conversion occurs. This PR is intended to fix the issue by ensuring that `model-settings.json` will always be processed first. #### Modifications The function `adaptModelLayoutForRuntime` in the MLServer adapter `server.go` was modified to ensure that, if present, the `model-settings.json` file will be placed at the head of the list of files to be processed. This makes sure that the config file will always be processed before any symlinks are created for the model files. A unit test was also added to ensure that the bug was fixed. #### Result Incorrect uri conversion bug no longer occurs even when the model files alphabetically precede the filename`model-settings.json`. Signed-off-by: xinyitan <xin.yi.tan@sap.com> Co-authored-by: xinyitan <xin.yi.tan@sap.com>
Resolved by #43 |
…ahub-io#43) #### Motivation As documented in [this issue](kserve#40), there is currently a bug in the conversion of the `uri` provided in the MLServer `model-settings.json` file if the model files are symlinked before the uri conversion occurs. This PR is intended to fix the issue by ensuring that `model-settings.json` will always be processed first. #### Modifications The function `adaptModelLayoutForRuntime` in the MLServer adapter `server.go` was modified to ensure that, if present, the `model-settings.json` file will be placed at the head of the list of files to be processed. This makes sure that the config file will always be processed before any symlinks are created for the model files. A unit test was also added to ensure that the bug was fixed. #### Result Incorrect uri conversion bug no longer occurs even when the model files alphabetically precede the filename`model-settings.json`. Signed-off-by: xinyitan <xin.yi.tan@sap.com> Co-authored-by: xinyitan <xin.yi.tan@sap.com>
I am experiencing some strange behaviour regarding the uri conversion of the
parameters.uri
field in themodel-settings.json
file. I have created amodel-settings.json
file that is placed in the same S3 folder as my model. In most cases, the uri conversion works as expected.For example, with my model filename as
my-model.joblib
, I provided the followingmodel-settings.json
fileand the
name
andparameters.uri
fields were correctly overwritten into these values when I checked the file from within the serving runtime podHowever, the peculiar thing is that if I were to name my model file as
mnist.joblib
instead and change mymodel-settings.json
to the following,then my model would fail to load due to this error in the
mlserver
container:I have checked the
model-settings.json
file in the serving runtime pod and it appears that theparameters.uri
value has indeed been set to/models/_mlserver_models/sklearn-mnist-2__isvc-a6929dd134/models/sklearn-mnist-2__isvc-a6929dd134/sklearn-model/mnist.joblib
.I am currently using MLServer v1.2.3 and kserve v0.9.0.
May I know if anyone is able to reproduce this? If so, are there any clues as to what the issue might be? And if not, what might I be doing wrong that is causing this error?
The text was updated successfully, but these errors were encountered: