ChatKNML

Getting started

First setup

1. Clone the repository:

   git clone git@github.com:knmlprz/ChatKNML.git

2. Navigate to the project directory:

   cd ChatKNML

3. Create a new branch for your development work:

   git checkout -b {your_branch_name}

4. Make sure you are working on the correct branch:

    git status

Starting app development

1. Copy the .env.example file:

   cp .env.example .env

   Modify the environment variables to suit your requirements.

2. Launching services using the "dev" profile:

   docker compose --profile dev up

Starting app production

Starting app

docker compose --profile prod up

Starting llm and embedding

1. Download model (must have for service llm-embedding to work!!!)

   Download model (size of file 3.6GB ):

   curl -o ./llm/models/llama-2-7b.Q3_K_L.gguf -L https://huggingface.co/TheBloke/Llama-2-7B-GGUF/resolve/main/llama-2-7b.Q3_K_L.gguf

   or

   wget -P ./llm/models/llama-2-7b.Q3_K_L.gguf https://huggingface.co/TheBloke/Llama-2-7B-GGUF/resolve/main/llama-2-7b.Q3_K_L.gguf

2. Launching llm and embedding

   2.1. Running on cpu

   docker compose --profile cpu up

   2.2. Running on gpu

   docker compose --profile gpu up

LLM and embedding api swagger

Swegger with EP for completions(llm + embedding) and only embedding is here