perf:reduce allocations in EncodingLoader #60
Conversation
- Avoid garbage strings from splitting on space. - Avoid allocations from using LINQ in a hot loop. Signed-off-by: Bradley Grainger <bgrainger@gmail.com>
WalkthroughThe changes in this pull request involve optimizations in the Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Outside diff range and nitpick comments (4)
src/libs/Tiktoken.Encodings.Abstractions/EncodingLoader.cs (2)
59-59: Suggestion: Ensure thebytesbuffer size is sufficient for decoded dataThe
bytesbuffer is currently allocated with a fixed size of 256 bytes. If the Base64-decoded data exceeds this size,Convert.TryFromBase64Charswill fail. Consider dynamically sizing the buffer based on the maximum possible decoded length of the Base64 input to prevent conversion failures.For example:
- Span<byte> bytes = stackalloc byte[256]; + int maxDecodedLength = (line.Length * 3) / 4; + Span<byte> bytes = maxDecodedLength <= 1024 + ? stackalloc byte[maxDecodedLength] + : new byte[maxDecodedLength];
58-58: Nitpick: Adjust the size of thetokensbufferThe
tokensbuffer is allocated with a size of 3, but only 2 tokens are expected per line. Reducing the size to 2 conserves stack space.Apply this diff:
- Span<Range> tokens = stackalloc Range[3]; + Span<Range> tokens = stackalloc Range[2];src/libs/Tiktoken.Core/CoreBPE.cs (2)
Line range hint
222-222: Incorrect initialization ofList<byte>The initialization
List<byte> accumulatedBytes = [];is invalid in C#. You should usenew List<byte>();to instantiate the list.Apply this fix:
-List<byte> accumulatedBytes = []; +List<byte> accumulatedBytes = new List<byte>();
Line range hint
253-271: Inefficient UTF-8 validation in older .NET versionsUsing exception handling for UTF-8 validation in the
#elseblock can impact performance due to the overhead of exceptions.Consider using
Encoding.UTF8.GetDecoder().Convertto validate UTF-8 sequences without exceptions:#if NET8_0_OR_GREATER bool isValid = Utf8.IsValid(accuArr); #else - bool isValid = true; - - try - { - var _ = encoding.GetString(accuArr); - } - catch (ArgumentException) - { - isValid = false; - } + Decoder utf8Decoder = Encoding.UTF8.GetDecoder(); + int charsUsed, bytesUsed; + bool completed; + char[] charBuffer = new char[accuArr.Length]; + utf8Decoder.Convert(accuArr, 0, accuArr.Length, charBuffer, 0, charBuffer.Length, true, out bytesUsed, out charsUsed, out completed); + bool isValid = completed; #endifEnsure to test this change thoroughly for correctness and performance improvements.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (2)
src/libs/Tiktoken.Core/CoreBPE.cs(1 hunks)src/libs/Tiktoken.Encodings.Abstractions/EncodingLoader.cs(3 hunks)
|
LGTM. Thank you for your contribution! |
|
Released as 2.1.0 on NuGet |
|
Thanks for the quick update! |
Improve performance of
ModelToEncoder.For("gpt-4o")by reducing allocations:Summary by CodeRabbit
New Features
Bug Fixes
Documentation
EncodingLoaderclass.