Add language support to the OCR manager #7465
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ocr-manager-language-support
…ocr-manager-language-support
…ocr-manager-language-support
daneryl
approved these changes
Dec 2, 2024
daneryl
approved these changes
Dec 2, 2024
Yes, let me know if you want me to create a new issue addressing that. Update: new issue addressing that #7514 |
RafaPolit
pushed a commit
that referenced
this pull request
Dec 11, 2024
* add LanguageCodeMapper * add ukrainian language to elastic search * simplify languages support * update languageSchema * revert changes * use legacy elastic mapping * fix language code * fix test case * add other language to available languages array * fix test errors --------- Co-authored-by: Joan Gallego Girona <daneryl@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #7439
It's not too clear in code, but the Uwazi context and OCR context are using different language code:
Uwazi = ISO639_3
OCR = ISO639_1
We were using getLanguage that I believe it was used to convert Uwazi language codes to Elastic search languages codes, as Ukranian is not supported in Elastic search, it was retuning
nulland not executing the OCR service.I've created LanguageCodeMapper that can convert between language code.
A better solution, it would be to make OCR models, properties more explicit in code so we can make a well defined boundary that can tell what belongs to Uwazi and what belongs to OCR
With that well defined Models we can create a Anti-Corruption Layer and whenever we need to use some feature from OCR we use this Anti-Corruption Layer to get the models from OCR.