Merge pull request #55 from Aristotle-Metadata-Enterprises/refactor

Minor for fallbacks refactor - Fixes #41 Fixes #49

.github/workflows/ci.yml

@@ -47,7 +47,10 @@ jobs:
 
       - name: Install test tools
         run: pip install tox coverage
-
+
+      - name: Flake8
+        run: pip install flake8 && flake8 --ignore=E501
+
       - name: Run SQLite tests
         run: tox -e dj-31-db-sqlite
 

Procfile

@@ -0,0 +1 @@
+web: bash ./run.sh

README.md

@@ -1,6 +1,14 @@
 # django-garnett
 
-Django Garnett is a field level translation library that allows you to store strings in multiple languages for fields in Django - with minimal changes to your models and without having to rewrite your code (mostly).
+Django Garnett is a field level translation library that allows you to store strings in multiple languages for fields in Django - with minimal changes to your models and without having to rewrite your code.
+
+Want a demo? https://django-garnett.herokuapp.com/
+
+<a href="https://www.aristotlemetadata.com"
+    style="text-decoration: none">
+Made with <i style="color:#d63384">♥</i> by
+<img src="https://brand.aristotlemetadata.com/images/svgs/thick.svg" height=20> Aristotle Metadata
+</a>
 
 In summary it allows you to do this:
 
@@ -97,7 +105,7 @@ Tested on:
 Pros:
 * Fetching all translations for a models requires a single query
 * Translations are stored in a single database field with the model
-* Translations act like regular a field `Model.field_name = "some string"` and `print(Model.field_name)` work as you'd expect
+* Translations act like regular a field `Model.field_name = "some string"` and `print(Model.field_name)` work as you would expect
 * Includes a configurable middleware that can set the current language context based on users cookies, query string or HTTP headers
 * Works nicely with Django Rest Framework
 
@@ -108,7 +116,7 @@ Cons:
 
 A few reasons:
 * Most existing django field translation libraries are static, and add separate database columns per translation.
-* We needed a library that could be added in without requiring a rewrite of a large code base.
+* We needed a library that could be added in without requiring a rewrite of a very large code base.
 
 Note: Field language is different to the django display language. Django can be set up to translate your pages based on the users browser and serve them with a user interface in their preferred language.
 
@@ -128,7 +136,7 @@ Garnett *does not* use the browser language by design - a user with a French bro
 5. Re-run `django makemigrations` & `django migrate` for any apps you've updated.
 6. Thats mostly it.
 
-You can also add a few value adds:
+You can also add a few optional value adds:
 
 7. (Optional) Add a garnett middleware to take care of field language handling:
 
@@ -146,21 +154,37 @@ You can also add a few value adds:
 
     * Install `garnett.context_processors.languages` this will add `garnett_languages` (a list of available `Language`s) and `garnett_current_language` (the currently selected language).
 
+10. (Optional) Add a custom translation fallback:
+
+    By default, if a language isn't available for a field, Garnett will show a mesage like:
+    > No translation of this field available in English
+
+    You can override this either by creating a custom fallback method:
+    ```
+    Translated(CharField(max_length=150), fallback=my_fallback_method))
+    ```
+    Where `my_fallback_method` takes a dictionary of language codes and corresponding strings, and returns the necessary text.
+
+    Additionally, you can customise how django outputs text in templates by creating a new
+    `TranslationStr` class, and overriding the [`__html__` method][dunder-html].
+
+
+
 ## `Language` vs language
 
-Django Garnett uses the python `langcodes` to determine more information about the languages being used - including the full name and local name of the language being used. This is stored as a `Language` object.
+Django Garnett uses the python `langcodes` library to determine more information about the languages being used - including the full name and local name of the language being used. This is stored as a `Language` object.
 
 
 ## Django Settings options:
 
 * `GARNETT_DEFAULT_TRANSLATABLE_LANGUAGE`
     * Stores the default language to be used for reading and writing fields if no language is set in a context manager or by a request.
     * By default it is 'en-AU' the [language code][term-language-code] for 'Strayan, the native tongue of inhabitants of 'Straya (or more commonly known as Australia). 
-    * Can also be callable that returns list of language codes
+    * Can also be callable that returns default language code
     * default: `'en-AU'`
 * `GARNETT_TRANSLATABLE_LANGUAGES`:
     * Stores a list of [language codes][term-language-code] that users can use to save against TranslatableFields.
-    * Can also be callable that returns default language code
+    * Can also be callable that returns list of language codes
     * default `[GARNETT_DEFAULT_TRANSLATABLE_LANGUAGE]`
 * `GARNETT_REQUEST_LANGUAGE_SELECTORS`:
     * A list of string modules that determines the order of options used to determine the language selected by the user. The first selector found is used for the language for the request, if none are found the DEFAULT_LANGUAGE is used. These can any of the following in any order:
@@ -190,18 +214,21 @@ Advanced Settings (you probably don't need to adjust these)
 ## Why call it Garnett?
 
 * Libraries need a good name.
-* Searching for "Famous Translators" will tell you about [Constnace Garnett](https://en.wikipedia.org/wiki/Constance_Garnett).
-* Searching for "Garnett Django" shows there was no library with this name. It did however talk about [Garnet Clark](https://en.wikipedia.org/wiki/Garnet_Clark) (also spelled Garnett), a jazz pianist who played with Django Reinhart - the namesake of the Django Web Framework.
+* Searching for "Famous Translators" will tell you about [Constance Garnett](https://en.wikipedia.org/wiki/Constance_Garnett).
+* Searching for "Django Garnett" showed there was no python library with this name.
+* It did however talk about [Garnet Clark](https://en.wikipedia.org/wiki/Garnet_Clark) (also spelled Garnett), a jazz pianist who played with Django Reinhart - the namesake of the Django Web Framework.
 * Voila - a nice name
 
 ## Warnings
 
-* `contains == icontains` For cross database compatibility reasons this library treats `contains` like `icontains`. I don't know why - https://www.youtube.com/watch?v=PgGNWRtceag
+* `contains == icontains` - On SQLite only, when doing a contains query
+ it does a case insensitive search. I don't know why - https://www.youtube.com/watch?v=PgGNWRtceag
 * Due to how django sets admin form fields you will not get the admin specific widgets like
   `AdminTextAreaWidget` on translated fields in the django admin site by default. They can however
-  be specified explicitly on the corresponding model form
+  be specified explicitly on the corresponding admin model form.
 
 need to run tests like this for now: PYTHONPATH=../ ./manage.py shell
 
 [term-language-code]: https://docs.djangoproject.com/en/3.1/topics/i18n/#term-language-code
 [django-how]: https://docs.djangoproject.com/en/3.1/topics/i18n/translation/#how-django-discovers-language-preference
+[dunder-html]: https://code.djangoproject.com/ticket/7261

dev/docker-compose.yml

@@ -18,6 +18,9 @@ services:
     build:
       context: ..
       dockerfile: ./dev/Dockerfile
+    environment:
+      - PYTHONPATH=/usr/src/app:/usr/src/app/tests/
+      - DJANGO_SETTINGS_MODULE=library_app.settings
     image: python
     # env_file:
     #   - default.env

garnett/context.py

@@ -1,13 +1,17 @@
 from contextlib import ContextDecorator
 import contextvars
+from langcodes import Language
 
 # Internal context var should be set via set_field_language and get via get_current_language
 _ctx_language = contextvars.ContextVar("garnett_language")
 
 
 class set_field_language(ContextDecorator):
     def __init__(self, language):
-        self.language = language
+        if isinstance(language, Language):
+            self.language = language
+        else:
+            self.language = Language.get(language)
         self.token = None
 
     def __enter__(self):

garnett/context_processors.py

@@ -1,10 +1,8 @@
 from .utils import get_languages, get_current_language
-from langcodes import Language
 
 
 def languages(request):
-    languages = [Language.make(language=l) for l in get_languages()]
     return {
-        "garnett_languages": languages,
+        "garnett_languages": get_languages(),
         "garnett_current_language": get_current_language(),
     }

garnett/fields.py

@@ -1,64 +1,23 @@
 from django.conf import settings
-from django.contrib.admin import widgets
-from django.contrib.admin.options import FORMFIELD_FOR_DBFIELD_DEFAULTS
 from django.core import exceptions
-from django.db.models import Model, JSONField
+from django.db.models import JSONField
 from django.db.models.fields.json import KeyTransform
-from django.utils.translation import gettext as _
 from dataclasses import make_dataclass
 from functools import partial
-from langcodes import Language
 import logging
 from typing import Callable, Dict, Union
 
-from garnett.utils import get_current_language, get_property_name, get_languages
+from garnett.translatedstr import TranslatedStr, VerboseTranslatedStr
+from garnett.utils import (
+    get_current_language_code,
+    get_property_name,
+    get_languages,
+    is_valid_language,
+)
 
 logger = logging.getLogger(__name__)
 
 
-class TranslatedStr(str):
-    def __new__(cls, content, fallback=False, fallback_language=""):
-        instance = super().__new__(cls, content)
-        instance.is_fallback = fallback
-        instance.fallback_language = fallback_language
-        return instance
-
-
-def translation_fallback(field: "TranslatedField", obj: Model) -> str:
-    """Default fallback function that returns an error message"""
-    all_ts = getattr(obj, f"{field.name}_tsall")
-    language = get_current_language()
-    lang_name = Language.make(language=language).display_name(language)
-    lang_en_name = Language.make(language=language).display_name()
-
-    return all_ts.get(
-        language,
-        _(
-            "No translation of %(field)s available in %(lang_name)s"
-            " [%(lang_en_name)s]."
-        )
-        % {
-            "field": field.name,
-            "lang_name": lang_name,
-            "lang_en_name": lang_en_name,
-        },
-    )
-
-
-def next_language_fallback(field: "TranslatedField", obj: Model) -> TranslatedStr:
-    """Fallback that checks each language consecutively"""
-    all_ts = getattr(obj, f"{field.name}_tsall")
-    for lang in get_languages():
-        if lang in all_ts:
-            return TranslatedStr(all_ts[lang], fallback=True, fallback_language=lang)
-
-    return TranslatedStr("", fallback=True)
-
-
-def blank_fallback(field, obj):
-    return ""
-
-
 def validate_translation_dict(all_ts: dict) -> None:
     """Validate that translation dict maps valid lang code to string
 
@@ -68,20 +27,19 @@ def validate_translation_dict(all_ts: dict) -> None:
         raise exceptions.ValidationError("Invalid value assigned to translatable field")
 
     # Check language codes
-    languages = set(get_languages())
     for code, value in all_ts.items():
-        if not isinstance(code, str) or code not in languages:
-            raise exceptions.ValidationError(f'"{code}" is not a valid language code')
-
         if not isinstance(value, str):
             raise exceptions.ValidationError(f'Invalid value for language "{code}"')
 
+        if not is_valid_language(code):
+            raise exceptions.ValidationError(f'"{code}" is not a valid language code')
+
 
 def translatable_default(
     inner_default: Union[str, Callable[[], str]]
 ) -> Dict[str, str]:
     """Return default from inner field as dict with current language"""
-    lang = get_current_language()
+    lang = get_current_language_code()
     if callable(inner_default):
         return {lang: inner_default()}
 
@@ -96,11 +54,14 @@ class TranslatedField(JSONField):
 
     def __init__(self, field, *args, fallback=None, **kwargs):
         self.field = field
+        self._fallback = fallback
 
-        if fallback:
+        if type(fallback) is type and issubclass(fallback, TranslatedStr):
             self.fallback = fallback
+        elif callable(fallback):
+            self.fallback = partial(TranslatedStr, fallback=fallback)
         else:
-            self.fallback = translation_fallback
+            self.fallback = VerboseTranslatedStr
 
         # Move some args to outer field
         outer_args = [
@@ -142,7 +103,7 @@ def value_from_object(self, obj):
             )
             return str(all_ts)
 
-        language = get_current_language()
+        language = get_current_language_code()
         return all_ts.get(language, None)
 
     def get_attname_column(self):
@@ -154,22 +115,12 @@ def get_attname_column(self):
     def contribute_to_class(self, cls, name, private_only=False):
         super().contribute_to_class(cls, name, private_only)
 
-        # We use ego to diferentiate scope here as this is the inner self
-        # Maybe its not necessary, but it is funny
+        # We use `ego` to differentiate scope here as this is the inner self
+        # Maybe its not necessary, but it is funny.
 
         @property
         def translator(ego):
-            """Getter for main field (without _tsall)"""
-            value = self.value_from_object(ego)
-            if value is not None:
-                return TranslatedStr(value, False)
-
-            fallback_value = self.fallback(self, ego)
-            # If fallback function didn't return a TranslatedStr wrap it in one
-            if not isinstance(fallback_value, TranslatedStr):
-                fallback_value = TranslatedStr(fallback_value, fallback=True)
-
-            return fallback_value
+            return self.fallback(getattr(ego, f"{self.name}_tsall"))
 
         @translator.setter
         def translator(ego, value):
@@ -188,7 +139,8 @@ def translator(ego, value):
                 all_ts = {}
 
             if isinstance(value, str):
-                all_ts[get_current_language()] = value
+                language_code = get_current_language_code()
+                all_ts[language_code] = value
             elif isinstance(value, dict):
                 all_ts = value
             else:
@@ -231,7 +183,7 @@ def available_languages(ego):
             langs = set()
             for field in ego.translatable_fields:
                 langs |= getattr(ego, f"{field.name}_tsall", {}).keys()
-            return [l for l in get_languages() if l in langs]
+            return [lang for lang in get_languages() if lang.language in langs]
 
         setattr(cls, "available_languages", available_languages)
 
@@ -265,7 +217,7 @@ def get_transform(self, name):
     def deconstruct(self):
         name, path, args, kwargs = super().deconstruct()
         args.insert(0, self.field)
-        kwargs["fallback"] = self.fallback
+        kwargs["fallback"] = self._fallback
         return name, path, args, kwargs
 
 
@@ -289,4 +241,4 @@ def __call__(self, *args, **kwargs):
 
 
 # Import lookups here so that they are registered by just importing the field
-from garnett import lookups
+from garnett import lookups  # noqa: F401, E402

garnett/lookups.py

@@ -1,5 +1,6 @@
-from django.db.models.fields import json
 from django.db.models import lookups
+from django.db.models.fields import json, CharField
+from django.db.models.functions import Cast
 from django.contrib.postgres.fields.jsonb import KeyTextTransform
 from django.contrib.postgres.lookups import SearchLookup, TrigramSimilar
 from django.contrib.postgres.search import TrigramSimilarity
@@ -111,6 +112,8 @@ def process_rhs(self, compiler, connection):
         return super().process_rhs(compiler, connection)
 
 
+# Override contains lookup for after a key lookup i.e. title__en__contains="thing"
+@TranslatedKeyTransform.register_lookup
 class KeyTransformContains(json.KeyTransformTextLookupMixin, lookups.Contains):
     lookup_name = "contains"
 
@@ -121,8 +124,15 @@ def process_rhs(self, compiler, connection):
         return super().process_rhs(compiler, connection)
 
 
-# Override contains lookup for after a key lookup i.e. title__en__contains="thing"
-TranslatedKeyTransform.register_lookup(KeyTransformContains)
+@TranslatedKeyTransform.register_lookup
+class KeyTransformExact(json.KeyTransformExact):
+    def process_lhs(self, compiler, connection):
+        self.lhs = Cast(self.lhs, CharField())
+        return super().process_lhs(compiler, connection)
+
+    def process_rhs(self, compiler, connection):
+        self.rhs = Cast(self.lhs, CharField())
+        return super().process_lhs(compiler, connection)
 
 
 @TranslatedField.register_lookup