[Bugfix][Relay][Strategy] Enable compile time transformation of weights matrix for arm_cpu NHWC quantized conv2d #15584
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ts matrix for arm_cpu NHWC quantized conv2d Fixed arm_cpu strategy bug which was causing tensorization errors when using the `AlterOpLayout` pass for the quantized NHWC conv2d schedules, as discovered in apache#10724. Therefore, we can now also enable the usage of `AlterOpLayout` for these schedules in order to transform the weight matrix at compile time, instead of runtime as before. I also modified the padding in `Conv2DGemmWeightTransformRel` and `interleave_transpose_weights` to reflect the changes made in apache#13669 and updated the AlterOpLayout tests accordingly.
|
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment. Generated by tvm-bot |
lhutton1
reviewed
Aug 18, 2023
There was a problem hiding this comment.
Thanks @Anndrey24, this looks like a great improvement! It seems we have test coverage for the alter op layout pass which is great, although, it would be good to also have a test to check no weight transform happens at runtime. Is it possible to extend
to check the schedule that gets selected is "conv2d_NHWC_quantized_*_without_transform.arm_cpu"?Updated the int8 conv2d implementation selection test to include more targets and to run the AlterOpLayout pass such that its respective changes of schedule are captured.
lhutton1
approved these changes
Aug 22, 2023
ekalda
approved these changes
Aug 22, 2023
|
Thanks @Anndrey24 @ekalda! |
Anndrey24
added a commit
to Anndrey24/tvm
that referenced
this pull request
Nov 6, 2023
Refactored out a piece of common functionality from the `conv2d_gemm_weight_transform` and `interleave_transpose_weights` functions, which has previously led to bugs stemming from changes made to only one but not the other, like in apache#15584. Determining the necessary padding for the interleaved and transposed weights matrix has now been separated into a new utility function, allowing future changes to be reflected in both callers.
Hzfengsy
pushed a commit
that referenced
this pull request
Nov 7, 2023
Refactored out a piece of common functionality from the `conv2d_gemm_weight_transform` and `interleave_transpose_weights` functions, which has previously led to bugs stemming from changes made to only one but not the other, like in #15584. Determining the necessary padding for the interleaved and transposed weights matrix has now been separated into a new utility function, allowing future changes to be reflected in both callers.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixed arm_cpu strategy bug which was causing tensorization errors when using the
AlterOpLayoutpass for the quantized NHWC conv2d schedules, as discovered in #10724. Therefore, we can now also enable the usage ofAlterOpLayoutfor these schedules in order to transform the weight matrix at compile time, instead of runtime as before.I also modified the padding in
Conv2DGemmWeightTransformRelandinterleave_transpose_weightsto reflect the changes made in #13669 and updated the AlterOpLayout tests accordingly.cc @ekalda @lhutton1 @leandron