Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问最低的资源要求是什么,单张24G卡似乎并不能运行。或者可以给一个支持多卡运行的版本吗,谢谢 #3

Open
qinyongchun opened this issue Aug 1, 2024 · 3 comments
Labels

Comments

@qinyongchun
Copy link

No description provided.

@heroding77
Copy link
Collaborator

您好,网页版本不支持多卡,单卡需要至少40G以上的卡支持。 多卡的解决方案如下:
先运行parse_pdf对pdf进行解析得到mmd文件,然后运行inference代码进行review。

@zhhtu
Copy link

zhhtu commented Aug 18, 2024

您好,再问个following up的小问题,我看是7b的model,单张24G卡也不能跑起来吗?是因为,input token太长的原因吗?

@heroding77
Copy link
Collaborator

您好,再问个following up的小问题,我看是7b的model,单张24G卡也不能跑起来吗?是因为,input token太长的原因吗?

是的,一篇paper截断后输入的token有12-16k,显存占用接近40G,因此24G的单卡推理不起来。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants