Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quantize_static fail with HistogramCalibrater. #17619

Closed
guotuofeng opened this issue Sep 20, 2023 · 1 comment
Closed

quantize_static fail with HistogramCalibrater. #17619

guotuofeng opened this issue Sep 20, 2023 · 1 comment
Labels
platform:windows issues related to the Windows platform quantization issues related to quantization

Comments

@guotuofeng
Copy link
Contributor

Describe the issue

In quantize_static, the TensorsData is created by using

                if len(v) == 4:
                    self.data[k] = TensorData(lowest=v[0], highest=v[1], histogram=v[2], bins=v[3])

However, in TensorData.__init__, the following logic is check the keyword args.

class TensorData:
    _allowed = frozenset(["avg", "std", "lowest", "highest", "hist", "hist_edges"])

    def __init__(self, **kwargs):
        for k, v in kwargs.items():
            if k not in TensorData._allowed:
                raise ValueError(f"Unexpected value {k!r} not in {TensorData._allowed}.")
            setattr(self, k, v)

    @property
    def range_value(self):
        if not hasattr(self, "lowest") or not hasattr(self, "highest"):
            raise AttributeError(f"Attributes 'lowest' and/or 'highest' missing in {dir(self)}.")
        return (self.lowest, self.highest)

    @property
    def avg_std(self):
        if not hasattr(self, "avg") or not hasattr(self, "std"):
            raise AttributeError(f"Attributes 'avg' and/or 'std' missing in {dir(self)}.")
        return (self.avg, self.std)

We can see the histogram key is not in the allowed keyword, therefore the following errors are raised.

Traceback (most recent call last):
  File "/mnt/vss/_work/1/s/***/passes/onnx/quantization.py", line 408, in _run_for_config
    quantize_static(
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/onnxruntime/quantization/quantize.py", line 436, in quantize_static
    tensors_range = calibrator.compute_data()
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/onnxruntime/quantization/calibrate.py", line 504, in compute_data
    return TensorsData(cal, self.collector.compute_collection_result())
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/onnxruntime/quantization/calibrate.py", line 58, in __init__
    self.data[k] = TensorData(lowest=v[0], highest=v[1], histogram=v[2], bins=v[3])
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/onnxruntime/quantization/calibrate.py", line 30, in __init__
    raise ValueError(f"Unexpected value {k!r} not in {TensorData._allowed}.")
ValueError: Unexpected value 'histogram' not in frozenset({'hist', 'lowest', 'hist_edges', 'avg', 'std', 'highest'}).

To reproduce

See https://aiinfra.visualstudio.com/PublicPackages/_build/results?buildId=357286&view=logs&jobId=35ea891e-7d20-5395-2445-be2fa0298764&j=5f33dc1f-8f24-5b97-62d3-f49845ddf6c9&t=d902c85d-ad1e-5c9a-ae64-6b490d39ff57.

Urgency

This bug break the onnxruntime quantization. it is a blocker.

Platform

Windows

OS Version

10

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.6.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

@github-actions github-actions bot added platform:windows issues related to the Windows platform quantization issues related to quantization labels Sep 20, 2023
guotuofeng added a commit to microsoft/Olive that referenced this issue Sep 20, 2023
## Describe your changes
- move the onnxruntime import for calibrate to vitis execution to fix
the onnxruntime 1.6 compatibility issue with openvino
- pin resnet ORT version to wait
microsoft/onnxruntime#17619 to be fixed.
- disable vitis test for ORT 1.16 since the calibrator API is changed.
- pin ort extension to 0.8.0 since the check_model will fail with latest
version.
- copy quant_pre_process code to olive to work-around the Windows temp
folder clean permission error.
microsoft/onnxruntime#17627

## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [x] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [x] Format your code by running `pre-commit run --all-files`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.

## (Optional) Issue link
yufenglee pushed a commit that referenced this issue Sep 25, 2023
### Description
One quantization case was not covered by the current list of unit tests.
This PR adds a unit test to cover that case with the fix. It fixes the
issue #17619.



### Motivation and Context
trajepl added a commit to microsoft/Olive that referenced this issue Sep 26, 2023
## Describe your changes
1. Only_onnxruntime to false when opt_level > 1
2. set quantization calibration_method to minmax as the bug from ort
1.16.0 microsoft/onnxruntime#17619

## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Format your code by running `pre-commit run --all-files`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.

## (Optional) Issue link
snnn pushed a commit that referenced this issue Sep 29, 2023
### Description
One quantization case was not covered by the current list of unit tests.
This PR adds a unit test to cover that case with the fix. It fixes the
issue #17619.



### Motivation and Context
kleiti pushed a commit to kleiti/onnxruntime that referenced this issue Mar 22, 2024
…t#17649)

### Description
One quantization case was not covered by the current list of unit tests.
This PR adds a unit test to cover that case with the fix. It fixes the
issue microsoft#17619.



### Motivation and Context
@LightSun
Copy link

LightSun commented Sep 10, 2024

ubuntu: 18.04. cuda 11.4
same to me . how to fix it ??

    raise ValueError(f"Unexpected value {k!r} not in {TensorData._allowed}.")
ValueError: Unexpected value 'histogram' not in frozenset({'std', 'hist_edges', 'highest', 'avg', 'lowest', 'hist'}).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform:windows issues related to the Windows platform quantization issues related to quantization
Projects
None yet
Development

No branches or pull requests

3 participants