Nicol

Nicol is an open-source web service, developed using the Kotlin programming language,

that enables streaming Server Stream Events and supports CPU inference for the port of llama2.c model.

🌟Features

🤖 Local Model: A Kotlin port of llama2.c by @karpathy .
🎢 Server Stream Event: Implemented a streaming inference interface similar to OpenAI's style .
🔒 Authorization: Provides basic interface token validation functionality based on the Spring Framework.

Showcasing in Web Demo	Server Stream Event Restful Api

Note
中国大陆 Ip 请保证网络能够正常访问 Github && Hugging face.
China Ip should make sure your network can access Github && Hugging face properly.

git clone https://github.com/WackyGem/Nicol.git

cd Nicol

docker-compose up

// web demo
http://127.0.0.1:8005/

// openapi
http://127.0.0.1:9005/swagger-ui/index.html

🤖 llama2.c : Inference a baby Llama 2 model in pure C .
💡 jimmer : A revolutionary ORM framework for both java and kotlin, and a complete integrated solution.