-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
merge docs #9657
merge docs #9657
Conversation
Thanks for your contribution! |
llm/docs/merge_model.md
Outdated
|
||
## 2.大模型融合算法介绍 | ||
### 2.1 Linear | ||
[Linear](https://arxiv.org/abs/2203.05482):融合方法通过对多个微调模型的权重进行线性加权平均实现模型的合并。这种方法基于以下假设:微调后的模型通常位于低误差盆地,线性组合权重能够保持其性能和特性。Linear 方法简单高效。 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已按照要求完成修改
llm/docs/merge_model.md
Outdated
--output_path ../checkpoints/model_merge | ||
|
||
``` | ||
<summary>  脚本参数介绍</summary><div> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里有点多余,和下面重复了
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已按照要求完成修改
llm/docs/merge_model.md
Outdated
|
||
- PaddlePaddle 3.0-beta | ||
- PaddleNLP 3.0.0b2 | ||
- PaddleSlim develop |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
不需要slilm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已按照要求完成修改
llm/docs/merge_model.md
Outdated
- **缓解对齐代价**:在对预训练模型进行微调 sft 后得到微调模型,之后我们通常进行 RLHF。RLHF 被证明有效提升了 LLMs 的用户友好性,但它会引入一个对齐税(alignment tax),即在对齐人类偏好后模型的性能可能有所下降。模型融合可以缓解对齐税。 | ||
|
||
## 2.大模型融合算法介绍 | ||
### 2.1 Linear |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
llm/docs/merge_model.md
Outdated
### 3.1 环境准备 | ||
|
||
- PaddlePaddle 3.0-beta | ||
- PaddleNLP 3.0.0b2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
把快速开始放在part 2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已按照要求完成修改
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #9657 +/- ##
========================================
Coverage 52.79% 52.79%
========================================
Files 718 718
Lines 112241 112241
========================================
Hits 59262 59262
Misses 52979 52979 ☔ View full report in Codecov by Sentry. |
llm/docs/mergekit.md
Outdated
#### 2.2.1 环境准备 | ||
|
||
- PaddlePaddle 3.0-beta | ||
- PaddleNLP 3.0.0b2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PaddleNLP develop
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已按照要求完成修改
llm/docs/mergekit.md
Outdated
<summary>  模型参数(Model Parameters)</summary><div> | ||
|
||
- `model_path_list`: 融合模型的路径或名称列表,需至少包含两个路径。默认为 `None`。 | ||
- `model_path_str`: 融合模型名称或路径字符串,以逗号分隔,默认为 `None`。 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
去掉model_path_str,这个主要是单测用的
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已按照要求完成修改
llm/docs/mergekit.md
Outdated
- **dare**: [DARE](https://arxiv.org/abs/2311.03099)论文中提出稀疏方式,根据设定概率,随机选择保留原始权重或设为0。可选超参 `reserve_p`、`rescale`。 | ||
- **magprune**:[DELLA](https://arxiv.org/abs/2406.11617)论文中提出稀疏方式,根据权重绝对值大小给定不同保留概率,随机选择保留原始权重或设为0。可选超参 `reserve_p`、`rescale`、`epsilon`。 | ||
|
||
### 2.2 快速开始 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
调换一下2.1和2.2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已按照要求完成修改
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR types
PR changes
docs
Description
merge docs