Add fused linear for the LLAMA MLP block and multi-head attention block #6425
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
ZHUI
reviewed
Jul 18, 2023
| gather_output=False, | ||
| ) | ||
| if self.fuse_attn_qkv: | ||
| self.qkv_proj = mpu.ColumnParallelLinear( |
There was a problem hiding this comment.
预训练参数加载转换，看看能不能根据fuse 搞成自动的。
littsk
force-pushed
the
fused_linear_feature
branch
3 times, most recently
from
a460685 to
3672383
Compare
…improved readability and consistency.
littsk
force-pushed
the
fused_linear_feature
branch
from
3672383 to
178e63b
Compare
Codecov Report
@@ Coverage Diff @@
## develop #6425 +/- ##
===========================================
- Coverage 63.18% 63.14% -0.05%
===========================================
Files 529 529
Lines 77214 77315 +101
===========================================
+ Hits 48789 48817 +28
- Misses 28425 28498 +73
|
sijunhe
reviewed
Jul 21, 2023
sijunhe
requested changes
Jul 21, 2023
littsk
force-pushed
the
fused_linear_feature
branch
from
9495473 to
32c36cb
Compare
sijunhe
reviewed
Jul 24, 2023
…he fused linear implementation.
littsk
force-pushed
the
fused_linear_feature
branch
from
32c36cb to
6412892
Compare
ZHUI
reviewed
Jul 25, 2023
ZHUI
previously approved these changes
Jul 25, 2023
| gather_output=False, | ||
| has_bias=False, | ||
| ) | ||
| # 为了减少张量并行的通信量，将两个linear合并成一个 |
| has_bias=False, | ||
| ) | ||
| # 为了减少张量并行的通信量，将两个linear合并成一个 | ||
| if config.fuse_mlp_linear: |
There was a problem hiding this comment.
这个 fuse_mlp_linear 是不是换个名字更好？
fuse_gate_up_proj ? 或者其他？
littsk
force-pushed
the
fused_linear_feature
branch
2 times, most recently
from
e1e2083 to
7927e54
Compare
littsk
force-pushed
the
fused_linear_feature
branch
from
7927e54 to
394608f
Compare
sijunhe
approved these changes
Jul 26, 2023
triple-Mu
pushed a commit
to triple-Mu/PaddleNLP
that referenced
this pull request
Aug 3, 2023
…ck (PaddlePaddle#6425) * Add fused linear for the LLAMA MLP block and multi-head attention block * Refactor the config name 'fuse_attn_qkv' to 'fuse_attention_qkv' for improved readability and consistency. * Added a switch for fuse_mlp_linear and improved the organization of the fused linear implementation. * add tensor parallel mappings for fused linears
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
Models
Description
Add fused linear operation for MLP and multi-head attention in LLAMA.