v0.1.0-rc.0

📦 Artifacts

• Binary for Mac OSX M1/M2, with Metal support provided by llama.cpp: tabby_aarch64-apple-darwin.
• Binary for Linux x86/64, CPU only: tabby_x86_64-unknown-linux-gnu.
• Docker image with NVIDIA CUDA Support: tabbyml/tabby:0.1.0-rc.0.

🚀 Features

llama.cpp on Apple M1/M2 for metal inference backend

• feat: llama.cpp for metal support [TAB-146] by @wsxiaoys in #391
• feat: support cancellation in llama backend [TAB-146] by @wsxiaoys in #392
• feat: tune llama metal backend performance by @wsxiaoys in #393
• fix: ensure default suffix to be non-empty by @wsxiaoys in #400
• feat: turn on metal device by default on macosx / aarch64 devices by @wsxiaoys in #398
• feat: implement input truncation with options.max_input_length by @wsxiaoys in #415
• feat: implement input truncation for llama-cpp-bindings by @wsxiaoys in #416

Experimental support of http api backend.

• feat: add vertex api bindings by @wsxiaoys in #410
• feat: add support vertex-ai http bindings by @wsxiaoys in #419
• feat: add support fastchat http bindings by @leiwen83 in #421

🧰 Improvements

• Improve default suffix handling in FIM inference. #400

💫 New Contributors

• @sunny0826 made their first contribution in #399
• @leiwen83 made their first contribution in #421

Full Changelog: v0.0.1...v0.1.0-rc.0