[ENH] New language detection

[djukicn]

Issue

Implements the new approach to language detection in the add-on.
Fixes #583

Description of changes

• The function for language detection and dict of supported languages (all languages supported by any method in Addon)
• Adoption of corpus to store language setting. Corpus can have the language set to the iso code of the language or None (for languages that are not in the list of supported languages - language-dependent methods do not work, but all other methods work)
• Adopting "input" widgets to set the language of the Corpus
• Adopting widgets which uses language setting (removing language controls, handling not supported languages)
• Removed Stafornford POS-tagging method since we do not use it and it is deprecated in NLTK

Includes

• Code changes
• Tests
• Documentation

Comments to the reviewer

• In the Create Corpus widget user must select a language manually
• Twitter widget use language set by the user; if language is not set and all tweets have the same language, it is set as Corpus'es language. When tweets are in different languages, language is set to None. I also changed the Twitter widget that variables are initialized when Corpus is built. Before, the same variable instance was used multiple times, and those settings were shared (values, ....).
• UDPipe offers various variants of models for the same language. Should we still support those variants or use the basic one for each language? Are they used in practice?

[codecov-commenter]

Codecov Report

Merging #874 (e061cd7) into master (23da347) will decrease coverage by 0.73%.
The diff coverage is 89.97%.

❗ Current head e061cd7 differs from pull request most recent head 501c135. Consider uploading reports for the commit 501c135 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #874      +/-   ##
==========================================
- Coverage   77.82%   77.10%   -0.73%     
==========================================
  Files          87       86       -1     
  Lines       12338    12014     -324     
  Branches     1624     1570      -54     
==========================================
- Hits         9602     9263     -339     
- Misses       2434     2459      +25     
+ Partials      302      292      -10     

[ajdapretnar]

I agree with

In place of a language dropdown, we can add a checkbox Use default stop words or something similar

[ajdapretnar]

I think it is not an error caused by this PR.

You are right, I cannot reproduce this anymore. Ignore the comment.

[VesnaT]

There is some strange behaviour regarding corpus.language property and Corpus ContexHandler:

1. saved setting probably should not overwrite the input corpus.language (widget Corpus (1))
2. the corpus.language has been changed globally (widget Python Script (6))
   

[VesnaT]

nams -> names

[VesnaT]

A new line is missing at the end of the file. The same goes for all *.tab.metadata files.

[VesnaT]

Language is missing in the docstring.

[VesnaT]

Placing the Wikipedia widget on canvas results in an error:

Traceback (most recent call last):
  File "/Users/vesna/orange-canvas-core/orangecanvas/scheme/widgetmanager.py", line 236, in __add_widget_for_node
    w = self.create_widget_for_node(node)
  File "/Users/vesna/orange-widget-base/orangewidget/workflow/widgetsscheme.py", line 300, in create_widget_for_node
    widget = self.create_widget_instance(node)
  File "/Users/vesna/orange-widget-base/orangewidget/workflow/widgetsscheme.py", line 413, in create_widget_instance
    widget.__init__()
  File "/Users/vesna/orange3-text/orangecontrib/text/widgets/owwikipedia.py", line 64, in __init__
    language_edit = ComboBox(self, 'language', tuple(sorted(LANG2ISO.items())))
TypeError: '<' not supported between instances of 'NoneType' and 'str'