The default model repository of openllm

This repo (on main branch) is already included by openllm by default.

If you want more up-to-date untested models, please add our nightly branch.

openllm repo add nightly https://github.com/bentoml/openllm-models@nightly

Supported Models

Table of Contents

• Llama-3.2
• Qwen-2.5
• Pixtral
• Phi-3
• Mistral
• Gemma-2
• Mixtral
• Mistral-Large
• Codestral
• Llama-3
• Qwen-2
• Llama-3.1
• Llama-2
• Gemma

---

Llama-3.2

Model	Version	Huggingface Link
llama3.2	11b-vision-instruct	HF Link
llama3.2	1b-instruct-fp16	HF Link
llama3.2	3b-instruct-fp16	HF Link

---

Qwen-2.5

Model	Version	Huggingface Link
qwen2.5	0.5b-instruct-fp16	HF Link
qwen2.5	1.5b-instruct-fp16	HF Link
qwen2.5	14b-instruct-fp16	HF Link
qwen2.5	32b-instruct-fp16	HF Link
qwen2.5	3b-instruct-fp16	HF Link
qwen2.5	72b-instruct-fp16	HF Link
qwen2.5	7b-instruct-fp16	HF Link

---

Pixtral

Model	Version	Huggingface Link
pixtral	12b-240910	HF Link

---

Phi-3

Model	Version	Huggingface Link
phi3	3.8b-instruct-fp16	HF Link
phi3	3.8b-instruct-ggml-q4	HF Link

---

Mistral

Model	Version	Huggingface Link
mistral	24b-instruct-nemo	HF Link
mistral	7b-instruct-awq-4bit	HF Link
mistral	7b-instruct-fp16	HF Link

---

Gemma-2

Model	Version	Huggingface Link
gemma2	27b-instruct-fp16	HF Link
gemma2	9b-instruct-fp16	HF Link

---

Mixtral

Model	Version	Huggingface Link
mixtral	8x7b-instruct-v0.1-awq-4bit	HF Link
mixtral	8x7b-instruct-v0.1-fp16	HF Link

---

Mistral-Large

Model	Version	Huggingface Link
mistral-large	123b-instruct-awq-4bit	HF Link
mistral-large	123b-instruct-fp16	HF Link

---

Codestral

Model	Version	Huggingface Link
codestral	22b-v0.1-fp16	HF Link

---

Llama-3

Model	Version	Huggingface Link
llama3	70b-instruct-awq-4bit	HF Link
llama3	70b-instruct-fp16	HF Link
llama3	8b-instruct-awq-4bit	HF Link
llama3	8b-instruct-fp16	HF Link

---

Qwen-2

Model	Version	Huggingface Link
qwen2	0.5b-instruct-fp16	HF Link
qwen2	1.5b-instruct-fp16	HF Link
qwen2	57b-a14b-instruct-fp16	HF Link
qwen2	72b-instruct-awq-4bit	HF Link
qwen2	72b-instruct-fp16	HF Link
qwen2	7b-instruct-awq-4bit	HF Link
qwen2	7b-instruct-fp16	HF Link

---

Llama-3.1

Model	Version	Huggingface Link
llama3.1	405b-instruct-awq-4bit	HF Link
llama3.1	70b-instruct-awq-4bit	HF Link
llama3.1	70b-instruct-fp16	HF Link
llama3.1	8b-instruct-awq-4bit	HF Link
llama3.1	8b-instruct-fp16	HF Link

---

Llama-2

Model	Version	Huggingface Link
llama2	13b-chat-fp16	HF Link
llama2	70b-chat-fp16	HF Link
llama2	7b-chat-awq-4bit	HF Link
llama2	7b-chat-fp16	HF Link

---

Gemma

Model	Version	Huggingface Link
gemma	2b-instruct-fp16	HF Link
gemma	7b-instruct-awq-4bit	HF Link
gemma	7b-instruct-fp16	HF Link

---