[bf16] add bf16 kernel: layer_norm p_norm reduce_sum #39843
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
OPs
Describe
添加
layer_normp_normreduce_sumbf16 kernel.对LayerNorm做性能测试，在embed_dim=1024的场景下分析call_1024_kernel在bf16数据类型下的计算性能：（call_1024_kernel的优化策略见PR39247）
layer_norm前向及反向耗时：
cost_time fp32: 0.01763439178466797s
cost_time bf16 use 1024 kernel: 0.007885456085205078s
cost_time bf16 no use 1024 kernel: 0.008244991302490234s