Add apostrophe to word tokenization list allowing possessive forms #149
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #148
I would like to propose adding the apostrophe symbol for word tokenization, so that the linter does not complain on possessive forms.
The integration tests don't run locally for me (due to missing API access, happy to help fix that if someone can guide me). The local tests after my addition appear unchanged (no additional failures):
In order to clearly separate the added test from the tokenization change, I have not squashed local commits. Apologies if this PR does violate the Contribution Guidelines for that. I'm happy to help in the future and happy to learn from this issue to submit better PRs in the future.