New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[tokenizer] Updates huggingface tokenizer to 0.15.2 #3071

Merged

frankfliu merged 1 commit into master from rust

Apr 7, 2024

Contributor

frankfliu commented Apr 7, 2024

Description

Brief description of what this PR is about

If this change is a backward incompatible change, why must this change be made?
Interesting edge cases to note here


          [tokenizer] Updates huggingface tokenizer to 0.15.2

48baf4c

frankfliu requested review from zachgk and a team as code owners

April 7, 2024 18:57

siddvenk approved these changes

View reviewed changes

frankfliu merged commit fa9bfbd into master

15 checks passed

frankfliu deleted the rust branch

April 7, 2024 19:26

frankfliu added a commit that referenced this pull request


          [tokenizer] Updates huggingface tokenizer to 0.15.2 (#3071)

7dbdab0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet