Performance Optimization: Expanding binary search window

[benbrandt]

Optimizes the binary search to not be too greedy when checking the size of sections.

Previously, I relied on the fact that we'd find a semantic level that didn't fit in the chunk size. However, it is quite likely there won't be one, and then the reliance on having this to scope the sections we are binary searching over means that in worst case, we are binary searching over the entire rest of the document, which might be large search area, and requires tokenizing huge chunks of text, which is the real bottleneck.

Now, we increase the search area by a factor of 2 each successful attempt, and also keep track of updating our starting low value for the binary search if we already know it is too low.

[codecov]

Codecov Report

Attention: Patch coverage is 98.26087% with 4 lines in your changes missing coverage. Please review.

Project coverage is 99.50%. Comparing base (b1b39d1) to head (26998f1).

❗ Current head 26998f1 differs from pull request most recent head a38ded3

Please upload reports for the commit a38ded3 to get more accurate results.

Files	Patch %	Lines
src/splitter.rs	96.33%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #231      +/-   ##
==========================================
- Coverage   99.65%   99.50%   -0.15%     
==========================================
  Files          11       11              
  Lines        2047     2033      -14     
==========================================
- Hits         2040     2023      -17     
- Misses          7       10       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.