Support dynamic export for dynabert #3549

LiuChiachi · 2022-10-25T05:33:47Z

PR types

New features

PR changes

APIs

Description

Support dynamic export for dynabert

PYTHONPATH=/liujiaqi/PaddleNLP:/liujiaqi/PaddleSlim python compress_seq_cls.py    \
    --dataset   "clue cluewsc2020"     \
    --model_name_or_path ernie-3.0-tiny-nano-v2-zh   \
    --per_device_train_batch_size 32   \
    --output_dir ./test  \
    --per_device_eval_batch_size 32    \
    --num_train_epochs 5   \
    --width_mult_list 2/3   \
    --batch_size_list 4   \
    --algo_list 'abs_max'   \
    --strategy 'dynabert ptq embeddings'  \
    --onnx_format False

wawltor · 2022-10-25T14:01:43Z

paddlenlp/trainer/trainer_compress.py

+def load_parameters(dynabert_model, ori_state_dict):
+    dynabert_state_dict = dynabert_model.state_dict()
+    for key in ori_state_dict.keys():
+        dynabert_key = key.replace(".fn", "")


这块的通用性如何？是否要做一些限制了？

这块的通用性如何？是否要做一些限制了？

这里不太明白为什么对fn的参数做一个替换

ofa model的参数被改写过，模型的state_dict的key中都有个'.fn'，所以在set参数之前，需要把'.fn'去掉

codecov · 2022-12-19T10:22:30Z

Codecov Report

Merging #3549 (7229c0d) into develop (240f817) will increase coverage by 0.00%.
The diff coverage is 7.14%.

❗ Current head 7229c0d differs from pull request most recent head 434151e. Consider uploading reports for the commit 434151e to get more accurate results

@@           Coverage Diff            @@
##           develop    #3549   +/-   ##
========================================
  Coverage    36.33%   36.33%           
========================================
  Files          419      419           
  Lines        59226    59221    -5     
========================================
- Hits         21520    21519    -1     
+ Misses       37706    37702    -4

Impacted Files	Coverage Δ
paddlenlp/trainer/trainer_compress.py	`9.01% <7.14%> (+0.30%)`	⬆️
paddlenlp/transformers/roberta/modeling.py	`89.85% <0.00%> (-0.37%)`	⬇️
paddlenlp/trainer/trainer.py	`11.24% <0.00%> (-0.10%)`	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

wawltor · 2022-12-22T03:08:20Z

paddlenlp/trainer/trainer_compress.py

-        if "ptq" in args.strategy:
-            self.args.input_filename_prefix = "pruned_model"
+        if "ptq" in args.strategy or "qat" in args.strategy:
+            self.args.input_filename_prefix = "model"


这里将前缀改变的原因是啥了？看起来其他的model zoo模型也要改这个前缀

export_model API 保存的模型前缀是model，不支持自定义前缀名称

wawltor · 2022-12-22T03:12:22Z

paddlenlp/trainer/trainer_compress.py

+def load_parameters(dynabert_model, ori_state_dict):
+    dynabert_state_dict = dynabert_model.state_dict()
+    for key in ori_state_dict.keys():
+        dynabert_key = key.replace(".fn", "")


这块的通用性如何？是否要做一些限制了？

这里不太明白为什么对fn的参数做一个替换

wawltor · 2022-12-22T03:13:17Z

paddlenlp/trainer/trainer_compress.py

+            if len(dynabert_shape) == 2:
+                dynabert_state_dict[dynabert_key] = ori_state_dict[key][: dynabert_shape[0], : dynabert_shape[1]]
+            else:
+                dynabert_state_dict[dynabert_key] = ori_state_dict[key][: dynabert_shape[0]]


建议这里对shape不为1，2的抛一个报错，防止用户对其他模型使用有问题

感谢提醒，已经增加了对异常处理的逻辑

…nto support-dynamic-dynabert

wawltor

LGTM

…nto support-dynamic-dynabert

wawltor

LGTM

support dynamic export for dynabert

7e9695d

wawltor reviewed Oct 25, 2022

View reviewed changes

LiuChiachi added 3 commits October 26, 2022 03:21

save diff

1f35898

solve conflicts

90f4231

solve conficts

21f498c

wawltor reviewed Dec 22, 2022

View reviewed changes

LiuChiachi added 2 commits December 27, 2022 03:36

add error message for input check

b892e68

Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…

ce309be

…nto support-dynamic-dynabert

LiuChiachi requested a review from wawltor December 28, 2022 12:00

wawltor previously approved these changes Dec 29, 2022

View reviewed changes

LiuChiachi added 2 commits December 29, 2022 08:55

update input model name prefix

290abbd

solve conficts

7229c0d

LiuChiachi dismissed wawltor’s stale review via 7229c0d December 29, 2022 09:30

LiuChiachi added 2 commits December 29, 2022 11:16

solve conficts and update compress strategy

d79cc04

Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…

434151e

…nto support-dynamic-dynabert

wawltor approved these changes Dec 29, 2022

View reviewed changes

LiuChiachi merged commit ff3c0c9 into PaddlePaddle:develop Dec 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support dynamic export for dynabert #3549

Support dynamic export for dynabert #3549

LiuChiachi commented Oct 25, 2022 •

edited

Loading

wawltor Oct 25, 2022 •

edited

Loading

wawltor Dec 22, 2022

LiuChiachi Dec 27, 2022

codecov bot commented Dec 19, 2022 •

edited

Loading

wawltor Dec 22, 2022

LiuChiachi Dec 27, 2022

wawltor Dec 22, 2022

wawltor Dec 22, 2022

LiuChiachi Dec 27, 2022

wawltor left a comment

wawltor left a comment

Support dynamic export for dynabert #3549

Support dynamic export for dynabert #3549

Conversation

LiuChiachi commented Oct 25, 2022 • edited Loading

PR types

PR changes

Description

wawltor Oct 25, 2022 • edited Loading

Choose a reason for hiding this comment

wawltor Dec 22, 2022

Choose a reason for hiding this comment

LiuChiachi Dec 27, 2022

Choose a reason for hiding this comment

codecov bot commented Dec 19, 2022 • edited Loading

Codecov Report

wawltor Dec 22, 2022

Choose a reason for hiding this comment

LiuChiachi Dec 27, 2022

Choose a reason for hiding this comment

wawltor Dec 22, 2022

Choose a reason for hiding this comment

wawltor Dec 22, 2022

Choose a reason for hiding this comment

LiuChiachi Dec 27, 2022

Choose a reason for hiding this comment

wawltor left a comment

Choose a reason for hiding this comment

wawltor left a comment

Choose a reason for hiding this comment

LiuChiachi commented Oct 25, 2022 •

edited

Loading

wawltor Oct 25, 2022 •

edited

Loading

codecov bot commented Dec 19, 2022 •

edited

Loading