Feature/nn and fo language extensions

[lise-brinck]

Description

Added language extensions for Faroese and Norwegian Nynorsk.

Types of change

This is an enhancement as it enhances the support of the spaCy language extensions.

Checklist

• I confirm that I have the right to submit this contribution under the project's MIT license.
• I ran the tests, and all new and existing tests passed.
• My changes don't require a change to the documentation, or if they do, I've added all required information.

[adrianeboyd]

Thanks for the nice PR, especially for including all the citations and tests!

I'm a little concerned about the duplication between the nb and nn tokenizer settings. Are there many practical differences, e.g. with removing 's as a suffix for one or the other? (nb doesn't remove the kind of English-specific suffixes, but nn does.)

And do you mind if I push directly to this branch to add these languages to the website docs?

[adrianeboyd]

Is "juli" missing on purpose?

[lise-brinck]

Yes. The abbreviation "jul(.)" for "juli" also means "Christmas" i Norwegian Nynorsk.

[lise-brinck]

Thanks for the nice PR, especially for including all the citations and tests!

I'm a little concerned about the duplication between the nb and nn tokenizer settings. Are there many practical differences, e.g. with removing 's as a suffix for one or the other? (nb doesn't remove the kind of English-specific suffixes, but nn does.)

And do you mind if I push directly to this branch to add these languages to the website docs?

The Norwegian Nynorsk tokenizer is a mix of the Norwegian Bokmål and Danish tokenizers with the addition of some language-specific abbreviations. I'm honestly not too certain about the exact differences between the two variations of Norwegian, and I'm sure there's room for improvements by someone who knows Norwegian Nynorsk.

And sure, go ahead and push to this branch :)