Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tokenization - Prevent detecting plus and ampersand as hard word boundary #655

Merged
merged 4 commits into from
May 29, 2024
Merged

Tokenization - Prevent detecting plus and ampersand as hard word boundary #655

merged 4 commits into from
May 29, 2024

Conversation

sabieber
Copy link
Member

Description

Also migrates the tokenizer tests to kotlin.

Additional Notes

  • This PR fixes or works on following ticket(s): SIRI-906

Checklist

  • Code change has been tested and works locally
  • Code was formatted via IntelliJ and follows SonarLint & best practices
  • Patch Tasks: Is local execution of Patch Tasks necessary? If so, please also mark the PR with the tag.

sabieber added 4 commits May 24, 2024 15:02
@sabieber
Migrate tests to kotlin
cc20538 
Fixes: SIRI-906
@sabieber
Prevent detecting plus and ampersand as hard word boundary
c36cc3a 
Also include a test to ensure this assumption in the future.

Fixes: SIRI-906
@sabieber
Improve code style slightly
1e69703 
Fixes: SIRI-906
@sabieber
Improve performance by turning a pattern compilation constant
d7c1809 
Fixes: SIRI-906
@sabieber sabieber added 🧬 Enhancement Contains new features 🛠️ Maintenance Translations, Code Cleanup, ... labels May 24, 2024
@sabieber sabieber merged commit 6a27652 into develop May 29, 2024
5 checks passed
@sabieber sabieber deleted the feature/sbi/SIRI-906 branch May 29, 2024 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@scireum-mbo scireum-mbo scireum-mbo approved these changes

@mkeckmkeck mkeckmkeck mkeckmkeck approved these changes

@jakobvogel jakobvogel jakobvogel approved these changes

@andyHa andyHa andyHa approved these changes

Assignees
No one assigned
Labels
🧬 Enhancement Contains new features 🛠️ Maintenance Translations, Code Cleanup, ...
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@sabieber @andyHa @jakobvogel @mkeckmkeck @scireum-mbo