Skip to content

Commit

Permalink
docs: Update MODEL_SPEC.md
Browse files Browse the repository at this point in the history
  • Loading branch information
wsxiaoys authored Nov 1, 2023
1 parent 36ffeb6 commit 90e446b
Showing 1 changed file with 1 addition and 4 deletions.
5 changes: 1 addition & 4 deletions MODEL_SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ The minimal Tabby model directory should include the following contents:
```
ggml/
tabby.json
tokenizer.json
```

### tabby.json
Expand All @@ -27,10 +26,8 @@ One example for the **prompt_template** is `<PRE>{prefix}<SUF>{suffix}<MID>`. In

The **chat_template** field is optional. When it is present, it is assumed that the model supports an instruct/chat-style interaction, and can be passed to `--chat-model`.

### tokenizer.json
This is the standard fast tokenizer file created using [Hugging Face Tokenizers](https://github.com/huggingface/tokenizers). Most Hugging Face models already come with it in repository.

### ggml/

This directory contains binary files used by the [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. Tabby utilizes ggml for inference on `cpu`, `cuda` and `metal` devices.

Currently, only `q8_0.v2.gguf` in this directory is in use. You can refer to the instructions in llama.cpp to learn how to acquire it.

0 comments on commit 90e446b

Please sign in to comment.