-
Notifications
You must be signed in to change notification settings - Fork 162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add sample code for GenAI model inferencing #1131
Conversation
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
olive/engine/packaging/sample_code/GenAIOnnxModel/cs/olive-genai-cs-sample.sln
Fixed
Show fixed
Hide fixed
f799ca4
to
84d842a
Compare
Can we also update unit tests for this? |
installed_packages = [ | ||
pkg | ||
for pkg in pkg_resources.working_set | ||
if pkg.key.startswith("onnxruntime-genai") or pkg.project_name.startswith("onnxruntime-genai") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any difference between key and project_name?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found some context here https://github.com/recipy/recipy/blob/d8f8fe8ace3659f1d700bb454e68a8db453e84f4/integration_test/environment.py#L125 which gives the impression that they could be different.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a big deal for now, we could be more consistent in our code.
Also, update the packaging logic to include sample code and required runtime binaries. NOTE: Native (both CPP & CS) binaries are not yet being published by the GenAI team and needs revisit once those binaries are available for download.
I will revisit the unit test once the runtime headers/binaries are available from genai repo. There is also some cleanup work in the packaging unit test to make packaging and sample code testing possible in CI. There is an open task on Dashboard from grab. |
installed_packages = [ | ||
pkg | ||
for pkg in pkg_resources.working_set | ||
if pkg.key.startswith("onnxruntime-genai") or pkg.project_name.startswith("onnxruntime-genai") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a big deal for now, we could be more consistent in our code.
Add sample code for GenAI model inferencing
Also, update the packaging logic to include sample code and required runtime binaries.
NOTE: Native (both CPP & CS) binaries are not yet being published by the GenAI team and needs revisit once those binaries are available for download.
Checklist before requesting a review
lintrunner -a
(Optional) Issue link