Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dynamic quantization + minor improvements in inference APIs #1433

Merged
merged 1 commit into from
Aug 22, 2022
Merged

Dynamic quantization + minor improvements in inference APIs #1433

merged 1 commit into from
Aug 22, 2022

Conversation

dskhudia
Copy link
Contributor

@dskhudia dskhudia commented Aug 19, 2022 •
edited
Loading

Adds

  1. Helper wrapper for dynamic quantization
  2. If scripting fails, we were printing an exception stack trace but still continuing with tracing. Fix this.
  3. Add a test for dynamic quantization.

closes https://mosaicml.atlassian.net/browse/CO-798

Test:
python -m pytest -k "test_dynamic_quantize" -s

@dskhudia dskhudia requested review from nik-mosaic and hanlint August 19, 2022 22:11
@dskhudia dskhudia changed the title Dynamic quantization + minor improvements inference APIs Dynamic quantization + minor improvements in inference APIs Aug 19, 2022
@dskhudia dskhudia force-pushed the inf_api_improvements branch from 7fbcae4 to d7b5732 Compare August 19, 2022 22:54
lupesko
lupesko reviewed Aug 21, 2022
View reviewed changes
tests/utils/test_inference.py Outdated Show resolved Hide resolved
composer/utils/inference.py Show resolved Hide resolved
@dskhudia dskhudia force-pushed the inf_api_improvements branch from d7b5732 to 63185cc Compare August 22, 2022 20:27
@lupesko
Copy link
Contributor

lupesko commented Aug 22, 2022

Looks good! Thanks for addressing the comments.

nik-mosaic
nik-mosaic approved these changes Aug 22, 2022
View reviewed changes
Copy link
Contributor

@nik-mosaic nik-mosaic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved!

@dskhudia dskhudia merged commit 962b3b1 into mosaicml:dev Aug 22, 2022
@dskhudia dskhudia deleted the inf_api_improvements branch August 22, 2022 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lupesko lupesko lupesko left review comments

@nik-mosaic nik-mosaic nik-mosaic approved these changes

@hanlint hanlint Awaiting requested review from hanlint

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dskhudia @lupesko @nik-mosaic