Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Number of V100 needed in pre-training stage and fine-tuning stage #10

Open
Yang-bug-star opened this issue Mar 4, 2024 · 4 comments
Open

Number of V100 needed in pre-training stage and fine-tuning stage #10

Yang-bug-star opened this issue Mar 4, 2024 · 4 comments

Comments

@Yang-bug-star
Copy link

Is it possible to train on 2 v100 gpus due to the small language model used ?
@monjurulkarim
Copy link

monjurulkarim commented Mar 20, 2024 •
edited
Loading

@zhuyiche @JLM-Z
@Yang-bug-star
Were you able to conduct training on your GPUs? I'm also interested in know about it.

@Yang-bug-star
Copy link
Author

Sorry, I haven't tried it.

@feiyu12138
Copy link

I found the training time is very close to original LLaVA: llama with clip 336 batch 14 except the pretrain is 1-hour shorter, is that normal?

@hunarbatra
Copy link

@feiyu12138 how many gpu's / which gpu did you use?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@hunarbatra @monjurulkarim @Yang-bug-star @feiyu12138