add new post-quant methods #32208
Merged
add new post-quant methods #32208
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
|
请为新增代码补充下单测。 |
juncaipeng
reviewed
Apr 13, 2021
| @@ -138,8 +138,10 @@ def __init__(self, | |||
| batch_size=10, | |||
| batch_nums=None, | |||
| algo="KL", | |||
| hist_perc=0.99999, | |||
| quantizable_op_type=["conv2d", "depthwise_conv2d", "mul"], | ||
| is_full_quantize=False, | ||
| bias_correct=False, |
python/paddle/fluid/contrib/slim/quantization/post_training_quantization.py
Outdated
Show resolved
Hide resolved
| @@ -373,14 +389,17 @@ def quantize(self): | |||
| batch_id += 1 | |||
| if self._batch_nums and batch_id >= self._batch_nums: | |||
| break | |||
|
|
|||
| if self._algo == 'avg': | |||
| if self._algo == "abs_max": | ||
| self._sample_abs_max() | ||
| if self._algo in ["avg", "abs_max"]: | ||
| self._sample_abs_max_avg() |
| if mse_loss <= self._best_mse_loss[var_name]: | ||
| self._best_mse_loss[var_name] = mse_loss | ||
| best_scale = scale | ||
| if best_scale > 0.0: |
There was a problem hiding this comment.
这个判断没有必要，self._quantized_threshold[var_name] = best_scale 可以放到if mse_loss <= self._best_mse_loss[var_name]:中
| save_info( | ||
| op_node, out_var_name, self._quantized_threshold, | ||
| argname_index[0] + str(argname_index[1]) + "_threshold", | ||
| "post_absmax") |
| if self._algo == 'avg': | ||
| if (var_name not in self._quantized_var_avg): | ||
| self._quantized_var_avg[var_name] = [] | ||
| abs_avg_value = float(np.mean(np.max(np.abs(var_tensor.reshape(var_tensor.shape[0], -1)), axis=(1)))) |
| @@ -1154,7 +1158,35 @@ def apply(self, graph): | |||
| else: | |||
| quant_axis = 0 | |||
| quantized_param_v = self._quant( | |||
| param_v, scale_v, self._weight_bits, quant_axis) | |||
| param_v.copy(), scale_v, self._weight_bits, quant_axis) | |||
| if self._bias_correct == True: | |||
There was a problem hiding this comment.
将bias_correction功能独立为一个函数实现
| if isinstance(scale_v, list): | ||
| if quant_axis == 0: | ||
| for i, s in enumerate(scale_v): | ||
| quantized_param_v[i] = quantized_param_v[i] * s / bnt |
wanghaoshuang
approved these changes
Apr 14, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Performance optimization
PR changes
APIs
Describe
1.Add new methods: 'mse', 'hist', 'avg' of getting threshold values of activations for post-training quantization in 'post_training_quantization.py'.
2.Add bias correction method in of https://arxiv.org/abs/1810.05723 for post-training quantization in 'quantization_pass.py'. Bias correction changes the quantized weights by the following formulation:
Experimental results on 6-bit MobileNetV1 which calibrated by a batch of 32 images:
abs_max: 44.99 abs_max+bias_correction: 47.91
avg: 56.34 avg+bias_correction: 56.17
mse: 61.32 mse+bias_correction: 61.42
hist(0.9999): 60.72 hist+bias_correction: 62.44
KL: 53.11 KL+bias_correction: 58.46
Time cost: Abs_max and avg cost about 1 minute; hist and KL cost about 6 minutes; mse cost about 10 minutes. Bias correction cost little time.