cl100kbase

Here are 2 public repositories matching this topic...

SharpToken is a C# library for tokenizing natural language text. It's based on the tiktoken Python library and designed to be fast and accurate.

csharp tokenizer openai gpt gpt-3 gpt-4 cl100kbase

This project implements token calculation for OpenAI's gpt-4 and gpt-3.5-turbo model, specifically using `cl100k_base` encoding.

encoding ai csharp tokens openai gpt4 chatgpt langchain tiktoken gpt35turbo cl100kbase tiktoken-sharp p50kbase langchain-dotnet

Add a description, image, and links to the cl100kbase topic page so that developers can more easily learn about it.

To associate your repository with the cl100kbase topic, visit your repo's landing page and select "manage topics."