New mechanism for running GGUF files directly, using llm -m gguf
. Example:
llm -m gguf \
-o path una-cybertron-7b-v2-bf16.Q8_0.gguf \
'Instruction: Five reasons to get a pet walrus
Response:'
This makes it much easier to try out new GGUF files, for example those released by TheBloke on Hugging Face. #26