本地RAG问答系统

解析用户输入： Web URL：html、git、jira、 文件：PDF、TXT文件 Image图像

• 问答机器人🤖
• 代码理解
• 图片识别
• 图片搜索🔍

Setup

Ollama

官网：https://ollama.com/

Windows系统查看ollama 帮助命令

ollama.exe serve --help

模型下载到本地默认目录：

• On Mac, the models will be download to ~/.ollama/models
• On Linux (or WSL), the models will be stored at /usr/share/ollama/.ollama/models

Ollama supports a list of models available on ollama.com/library

Here are some example models that can be downloaded:

Model	Parameters	Size	Download
Llama 2	7B	3.8GB	ollama run llama2
Mistral	7B	4.1GB	ollama run mistral
Dolphin Phi	2.7B	1.6GB	ollama run dolphin-phi
Phi-2	2.7B	1.7GB	ollama run phi
Neural Chat	7B	4.1GB	ollama run neural-chat
Starling	7B	4.1GB	ollama run starling-lm
Code Llama	7B	3.8GB	ollama run codellama
Llama 2 Uncensored	7B	3.8GB	ollama run llama2-uncensored
Llama 2 13B	13B	7.3GB	ollama run llama2:13b
Llama 2 70B	70B	39GB	ollama run llama2:70b
Orca Mini	3B	1.9GB	ollama run orca-mini
Vicuna	7B	3.8GB	ollama run vicuna
LLaVA	7B	4.5GB	ollama run llava
Gemma	2B	1.4GB	ollama run gemma:2b
Gemma	7B	4.8GB	ollama run gemma:7b

Note: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.

常用命令

#Pull a model
ollama pull llama2
#Remove a model
ollama rm llama2
#Copy a model
ollama cp llama2 my-llama2
#List models on your computer
ollama list
#查看模型信息
ollama show --modelfile mistral

导出Ollama本地Model（格式为gguf）

❯ ollama show --modelfile bakllava
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM bakllava:latest

FROM /Users/liushihao/.ollama/models/blobs/sha256:deb26e54ccebc9ce0665bf35dfdc73f2989b7308ecf9b08bd897a9a1ec9cb455
TEMPLATE """{{ .System }}
USER: {{ .Prompt }}
ASSSISTANT: """
PARAMETER num_ctx 4096
PARAMETER stop "</s>"
PARAMETER stop "USER:"

❯ cp /Users/liushihao/.ollama/models/blobs/sha256:deb26e54ccebc9ce0665bf35dfdc73f2989b7308ecf9b08bd897a9a1ec9cb455 bakllava.gguf

❯ git lfs track  bakllava.gguf
Tracking "bakllava.gguf"

❯ git add -A .

❯ git commit -m "bakllava.gguf"

❯ git push

如何导入Model

Ollama supports importing GGUF models in the Modelfile:

#1.Create a file named Modelfile, with a FROM instruction with the local filepath to the model you want to import.
FROM ./vicuna-33b.Q4_0.gguf
#2.Create the model in Ollama
ollama create example -f Modelfile
#3.Run the model
ollama run example

❯ ollama show --modelfile mistral
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM mistral:latest

FROM /path/.ollama/models/blobs/sha256:e8a35b5937a5e6d5c35d1f2a15f161e07eefe5e5bb0a3cdd42998ee79b057730
TEMPLATE """[INST] {{ .System }} {{ .Prompt }} [/INST]"""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
(base) 


❯ ollama show --modelfile llava:latest
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM llava:latest

FROM /path/.ollama/models/blobs/sha256:170370233dd5c5415250a2ecd5c71586352850729062ccef1496385647293868
TEMPLATE """[INST] {{ if .System }}{{ .System }} {{ end }}{{ .Prompt }} [/INST]"""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
(base) 



❯ ollama show --modelfile llama2      
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM llama2:latest

FROM /path/.ollama/models/blobs/sha256:8934d96d3f08982e95922b2b7a2c626a1fe873d7c3b06e8e56d7bc0a1fef9246
TEMPLATE """[INST] <<SYS>>{{ .System }}<</SYS>>

{{ .Prompt }} [/INST]
"""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER stop "<<SYS>>"
PARAMETER stop "<</SYS>>"

❯ ollama show bakllava:latest  --modelfile
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM bakllava:latest

FROM /Users/liushihao/.ollama/models/blobs/sha256:deb26e54ccebc9ce0665bf35dfdc73f2989b7308ecf9b08bd897a9a1ec9cb455
TEMPLATE """{{ .System }}
USER: {{ .Prompt }}
ASSSISTANT: """
PARAMETER num_ctx 4096
PARAMETER stop "</s>"
PARAMETER stop "USER:"

REST API

Ollama has a REST API for running and managing models.

Generate a response

curl http://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt":"Why is the sky blue?"
}'

Chat with a model

curl http://localhost:11434/api/chat -d '{
  "model": "mistral",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

国内魔塔Ollama Model地址：https://modelscope.cn/models/liush99/ollama_models/files