refactor(ml): model sessions #10559
Merged
refactor(ml): model sessions #10559
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
linting formatting
mertalev
force-pushed
the
refactor/ml-model-session
branch
from
aa65409 to
1eac415
Compare
|
Great refactoring, I'll give this a spin on ARM in few days |
jrasm91
approved these changes
Jun 23, 2024
fyfrey
approved these changes
Jun 25, 2024
zackpollard
approved these changes
Jun 25, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The ML service has some coupling with ONNX Runtime: much of the logic in the base inference class is only relevant to this backend, while handling for other backends is treated as a special case. This PR abstracts ONNX Runtime into a
ModelSessionwrapper similar to how ARM NN is handled.The catalyst for this change is that facial recognition recently broke for ARM NN due to a change that only applied to ONNX Runtime. As part of this, it also adds relevant tests for facial recognition, as well as
AnnSessionin general as its test coverage was quite low.How Has This Been Tested?
Tested that the endpoint still works for each model task using ONNX Runtime. I'm unable to test ARM NN.