Enable input_embeds for ChatGLM / ChatGLMForConditionalGeneration #5775
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix typo and minor bugs to enable the input_embeds input rather than input_ids
|
Thanks for your contribution! |
wawltor
previously approved these changes
Apr 24, 2023
| else: | ||
| raise ValueError("You have to specify either input_ids or inputs_embeds") | ||
|
|
||
| if inputs_embeds is None: | ||
| inputs_embeds = self.word_embeddings(input_ids) | ||
| inputs_embeds = inputs_embeds.transpose([1, 0, 2]) | ||
| inputs_embeds = inputs_embeds.transpose([1, 0, 2]) |
There was a problem hiding this comment.
这里的一些小坑是在于input_ids是作为默认输出，在export和inference逻辑是默认使用input_ids
There was a problem hiding this comment.
其它transformer模型（比如ERNIE）对input_ids都有默认值None。同时PretrainedModel实例也都会检查，要求input_ids和embeds不同时为None，不同时有值。所以这里应该还好
Codecov Report
@@ Coverage Diff @@
## develop #5775 +/- ##
===========================================
- Coverage 61.60% 61.57% -0.03%
===========================================
Files 489 489
Lines 68500 68540 +40
===========================================
+ Hits 42197 42205 +8
- Misses 26303 26335 +32
|
LemonNoel
reviewed
Apr 24, 2023
| @@ -826,7 +826,7 @@ def update_model_kwargs_for_generation( | |||
|
|
|||
| def forward( | |||
| self, | |||
| input_ids, | |||
| input_ids=None, | |||
There was a problem hiding this comment.
这里 input_ids 默认设为 None 时，需要修改下 ChatGLMModel 中 position_ids is None 和 attention_mask is None 的分支逻辑，因为这两个分支都依赖 input_ids。
1）可支持为None 的，在 input_ids is None 时将 input_ids 相关的参数改为从 input_embeds 获取。
2) 不支持为 None 的，需要显式抛出异常，说明原因。
There was a problem hiding this comment.
按照逻辑2)修改了，要求input_ids与attention_mask / position_ids不同时为None
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix typo and minor bugs to enable the input_embeds input rather than input_ids
PR types
Bug fixes
PR changes
Models / APIs
Description
See changelog / Commit