Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix loading chatglm3 #1937

Merged
merged 4 commits into from
Aug 19, 2024
Merged

Conversation

moetayuko
Copy link
Contributor

@moetayuko moetayuko commented Aug 12, 2024

fixes #1909

@binary-husky
Copy link
Owner

looking into it

@moetayuko
Copy link
Contributor Author

the pr is two-fold:

  1. using quantization_config is due to upstream API change, the previously used argument is marked deprecated. also, BitsAndBytesConfig implicitly moves data to cuda after quantization so there's no need to specify device. removing device also fixes cuda oom for me even if quantization is enabled
  2. the latest transformers library cannot load chatglm3 model. they pin transformers==4.40.0 but 4.41.x works fine for me so I pin <4.42 instead

@binary-husky binary-husky changed the base branch from master to frontier August 19, 2024 15:31
@binary-husky binary-husky merged commit a95b3da into binary-husky:frontier Aug 19, 2024
1 check passed
binary-husky added a commit that referenced this pull request Sep 8, 2024
* update welcome svg

* fix loading chatglm3 (#1937)

* update welcome svg

* update welcome message

* fix loading chatglm3

---------

Co-authored-by: binary-husky <qingxu.fu@outlook.com>
Co-authored-by: binary-husky <96192199+binary-husky@users.noreply.github.com>

* begin rag project with llama index

* rag version one

* rag beta release

* add social worker (proto)

* fix llamaindex version

---------

Co-authored-by: moetayuko <loli@yuko.moe>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: 可能是由于transformers库版本导致的chatglm无法使用问题
2 participants