-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[prompt] update prompt api & add prefix template #3724
Merged
Merged
Changes from 18 commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
e5336a7
[prompt] update template & add prefix template
LemonNoel f52527d
Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…
LemonNoel a8c348c
[prompt] update verbalizer and trainer
LemonNoel f1f1153
[prompt] fix incompatible implementations based on multi-class
LemonNoel 8dfbdbb
[prompt] update document
LemonNoel bc42645
[prompt] fix bugs encountered while exporting static model.
LemonNoel aa87ecb
[few-shot] update prompt apis in text_classification application
LemonNoel 2ad7625
Merge branch 'develop' into prompt
LemonNoel 3dea39d
[few-shot] update python deploy in multi-class application
LemonNoel dddf439
Merge remote-tracking branch 'lemon/prompt' into prompt
LemonNoel 5771b21
[few-shot] update python deploy in hierachical application
LemonNoel a14b14c
[few-shot] update python deploy in multi-class application
LemonNoel 4834efd
Merge remote-tracking branch 'origin/develop' into prompt
LemonNoel ae4e3f3
[few-shot] update readmes in text_classification application
LemonNoel 22d7a50
Merge branch 'develop' into prompt
LemonNoel 8da482e
[prompt] fix bug while creating attention masks
LemonNoel 8ac676b
Merge remote-tracking branch 'lemon/prompt' into prompt
LemonNoel 5b525d9
[prompt] update the template example in comments
LemonNoel f34d2a9
[few-shot] add requirements
LemonNoel 9ed5eeb
[prompt] add reference in document
LemonNoel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -65,9 +65,9 @@ | |
|
||
内存: 630 GB | ||
|
||
3. PaddlePaddle 版本:2.3.1 | ||
3. PaddlePaddle 版本:2.4rc | ||
|
||
4. PaddleNLP 版本:2.3.5 (develop) | ||
4. PaddleNLP 版本:2.4.2 (develop) | ||
|
||
5. 评估设置 | ||
|
||
|
@@ -91,7 +91,7 @@ | |
| model_name | 训练方式 | Micro F1分数 | Macro F1分数 | | ||
| ---------- | ------- | ----------- | ----------- | | ||
| ernie-3.0-base-zh | 微调学习 | 0.7172 | 0.3821 | | ||
| ernie-3.0-base-zh | 提示学习 | 0.8855 | 0.8443 | | ||
| ernie-3.0-base-zh | 提示学习 | 0.8945 | 0.8516 | | ||
|
||
|
||
<a name="定制训练"></a> | ||
|
@@ -102,10 +102,10 @@ | |
<a name="运行环境"></a> | ||
### 3.1 运行环境 | ||
|
||
- python >= 3.6 | ||
- paddlepaddle > 2.3(2.4版本发布前推荐安装[develop版本](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html)) | ||
- paddlenlp >= 2.3.5 | ||
- paddle2onnx >= 1.0.0rc3 | ||
- python >= 3.7 | ||
- paddlepaddle >= 2.4rc | ||
- paddlenlp >= 2.4.3 | ||
- paddle2onnx >= 1.0.3 | ||
|
||
<a name="代码结构"></a> | ||
### 3.2 代码结构 | ||
|
@@ -222,12 +222,12 @@ python train.py \ | |
--do_export \ | ||
--num_train_epochs 100 \ | ||
--logging_steps 5 \ | ||
--save_total_limit 1 \ | ||
--per_device_eval_batch_size 32 \ | ||
--per_device_train_batch_size 8 \ | ||
--metric_for_best_model macro_f1_score \ | ||
--load_best_model_at_end \ | ||
--evaluation_strategy epoch \ | ||
--save_strategy epoch | ||
--eval_steps 100 | ||
``` | ||
**多卡训练** | ||
|
||
|
@@ -247,12 +247,12 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \ | |
--do_export \ | ||
--num_train_epochs 100 \ | ||
--logging_steps 5 \ | ||
--save_total_limit 1 \ | ||
--per_device_eval_batch_size 32 \ | ||
--per_device_train_batch_size 8 \ | ||
--metric_for_best_model macro_f1_score \ | ||
--load_best_model_at_end \ | ||
--evaluation_strategy epoch \ | ||
--save_strategy epoch | ||
--eval_steps 100 | ||
``` | ||
|
||
可配置参数说明: | ||
|
@@ -273,6 +273,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \ | |
- `do_export`: 是否在运行结束时将模型导出为静态图,保存路径为`output_dir/export`。 | ||
- `num_train_epochs`: 训练的最大轮数。 | ||
- `max_steps`: 训练的最大步数。此设置将会覆盖`num_train_epochs`。 | ||
- `save_total_limit`: 模型检查点保存数量。 | ||
- `device`: 使用的设备,默认为`gpu`。 | ||
- `eval_steps`: 评估模型的间隔步数。 | ||
- `logging_steps`: 打印日志的间隔步数。 | ||
|
@@ -352,9 +353,9 @@ python infer.py --model_path_prefix checkpoints/export/model --data_dir ./data - | |
可配置参数说明: | ||
|
||
- `model_path_prefix`: 导出的静态图模型路径及文件前缀。 | ||
- `model_name_or_path`: 内置预训练模型名,或者模型参数配置目录路径,用于加载tokenizer。默认为`ernie-3.0-base-zh`。 | ||
- `model_name`: 内置预训练模型名,用于加载tokenizer。默认为`ernie-3.0-base-zh`。 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 这里为什么model_name_or_path-> model_name ,不可以直接传入一个path进去? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 这里 |
||
- `data_dir`: 待推理数据所在路径,数据应存放在该目录下的`data.txt`文件。 | ||
- `max_seq_length`: 最大句子长度,超过该长度的文本将被截断,不足的以Pad补全。提示文本不会被截断。 | ||
- `max_length`: 最大句子长度,超过该长度的文本将被截断,不足的以Pad补全。提示文本不会被截断。 | ||
- `batch_size`: 每次预测的样本数量。 | ||
- `device`: 选择推理设备,包括`cpu`和`gpu`。默认为`gpu`。 | ||
- `device_id`: 指定GPU设备ID。 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个要么直接2.4.3版本? 后面也可以直接2.4.3版本
最好能依赖关系放在requirements.txt里面
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已修改为2.4.3版本。相关依赖放在了
requirements_cpu.txt
和requirements_gpu.txt
文件。