-
Notifications
You must be signed in to change notification settings - Fork 113
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Add Vinoo's l10n docs to the repository This is copy pasted + some formatting from https://docs.google.com/document/d/1KUjR9GpQy4kpUsBu0RuMvauNpnJO4k66b2GzaDSNQTc/edit Vinoo is the author here, not me. * Add extras.xliff documentation * Add a bunch of new words to .dictionary * Address spellchecks
- Loading branch information
Showing
2 changed files
with
232 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,202 @@ | ||
| # Localization Overview of Mozilla VPN | ||
|
|
||
| ## Introduction | ||
|
|
||
| Mozilla VPN localizes its UI for various languages and regions. Each locale, which combines a language and its corresponding region, has its own set of strings in the application’s resources. To change the locale, users can use a language menu in the app. When the language is switched, the UI is updated with the specific set of strings for that locale, and locale-specific formatting is used for fields like date, time and currency. | ||
|
|
||
| When a new string is introduced to the app, it starts in English. Afterwards, the string is translated into supported locales by a global community of volunteers using the [Pontoon](https://pontoon.mozilla.org/) translation management system. After translation, strings are added to each locale’s file of localized strings and those files are added to the app binary’s resources. | ||
|
|
||
| VPN uses Qt framework classes for loading a localized string for a given locale, and for locale-specific formatting of dates, time, currency etc. | ||
|
|
||
| ## Using Localized Strings | ||
|
|
||
| A localized string is referenced in the code base using a string ID. This indirection allows the locale-specific string corresponding to that ID to be loaded. | ||
|
|
||
| VPN has two different sets of localized strings. Legacy strings use an older approach. All other strings use a new approach. Here is how they are obtained in the code: | ||
|
|
||
| - Legacy strings are obtained using Qt’s [**qsTrId**](https://doc.qt.io/qt-6/linguist-id-based-i18n.html)**(_stringID_)** in QML or [**qtTrId**](https://doc.qt.io/qt-5/qtglobal.html#qtTrId)**(_stringID_)** in C++. | ||
| - C++ example: | ||
| ```cpp | ||
| QString str = qtTrId("vpn.toggle.on") | ||
| ``` | ||
|
|
||
| - QML example: | ||
| ```js | ||
| qsTrId("vpn.toggle.on") | ||
| ``` | ||
| - All other strings are obtained from a partially generated VPN class, **I18nStrings**, which is a map from a string ID key to the localized string value. **I18nStrings** internally uses **qtTrId(_stringID_)** to get the localized string: | ||
| - C++ example: | ||
| ```cpp | ||
| I18nStrings::instance()->id(I18nStrings::NotificationsUnsecuredNetworkMessage); | ||
| ``` | ||
|
|
||
| - QML example, where **I18nStrings** is exposed to QML as **MZI18n**: | ||
| ```js | ||
| MZI18n.NotificationsUnsecuredNetworkMessage | ||
| ``` | ||
|
|
||
| ## Creating Localized Strings | ||
|
|
||
| VPN’s string localization has evolved from a legacy approach to a new approach. New strings should be added using the new approach, which is the recommended method. Legacy strings have not been converted to the new approach to avoid additional work for Pontoon’s volunteer translation community. | ||
|
|
||
| - Legacy Strings: | ||
|
|
||
| - Legacy strings are added by preceding one of the qsTrid or qtTrid calls with a comment that starts with `//%`. For example: | ||
|
|
||
| ``` | ||
| //% "VPN is off" | ||
| text: qsTrId("vpn.controller.deactivated") | ||
| ``` | ||
|
|
||
| In this case, the comment provides the English version of the string in quotes, and the ID is the argument passed to qsTrId. These are extracted by Qt tools during the build process. | ||
|
|
||
| Legacy strings have disadvantages because the ID and strings are scattered throughout the code base, and moving a string to a different file will trigger a new translation request in Pontoon. | ||
|
|
||
| - New Strings: | ||
| - New strings are added in the **‘src\translations\strings.yaml’** file. For example: | ||
| ```yaml | ||
| notifications: | ||
| unsecuredNetworkMessage: | ||
| value: "%1 is not secure. Click here to turn on VPN and secure your device." | ||
| comment: "%1 is the Wi-Fi network name" | ||
| ``` | ||
| In this example, the string is added to a section called **‘notifications’**. Its ID is ‘**vpn.notifications.unsecuredNetworkMessage’**, formed by concatenating ‘vpn’, the section name and the string’s section ID. The **‘value’** field has the English string and the **‘comment’** field provides guidance for translation. | ||
| During the build process, **‘./scripts/utils/generate_strings.py’** processes **‘strings.yaml’** and generates part of the **I18nStrings** class in **‘build\translations\generated\i18nstrings_p.cpp’** and **‘i18nstrings.h’**. The corresponding string ID in **I18nStrings** is **‘NotificationsUnsecuredNetworkMessage’**, formed by concatenating the section name and string’s section ID and converting it to Pascal case. | ||
| ## Creating Localized Strings: Best Practices | ||
| When adding new English strings in ‘strings.yaml’ that need to be localized, ensure the following: | ||
| - **Avoid String Concatenation:** Don’t combine localized sub-strings in the app. Instead, use positional parameters like %1, %2 in the string and include a comment to clarify the meaning of these parameters. This allows for flexibility across different languages where the order of phrases may vary. | ||
| - **Don’t reuse string IDs:** If an existing string needs to be changed, create a new string ID for it. This ensures that a new translation request can be properly managed by Pontoon. Do not reuse the previous string ID. The old string ID can be deleted along with the old string. | ||
| - **Additional Best Practices:** See [Localization best practices for developers](https://mozilla-l10n.github.io/documentation/localization/dev_best_practices.html). | ||
| ## Qt’s Support for Localization | ||
| Mozilla VPN uses the following Qt Localization features: | ||
| - **Translation Source Files (TS files):** Qt uses XML files called Translation Source files (TS files) for each locale. A file contains the set of localized strings for a locale, where each entry includes an ID, the original string in the source language, a comment guiding the translation, and the translated string. | ||
| **mozillavpn_fr.ts** | ||
| ```xml | ||
| <message id="vpn.notifications.unsecuredNetworkMessage"> | ||
| <source>%1 is not secure. Click here to turn on VPN and secure your device.</source> | ||
| <extracomment>%1 is the Wi-Fi network name</extracomment> | ||
| <translation type="unfinished">%1 n’est pas sécurisé. Cliquez ici pour activer le VPN et sécuriser votre appareil.</translation> | ||
| </message> | ||
| ``` | ||
|
|
||
| In this example, the ID is **'vpn.notifications.unsecuredNetworkMessage'**, the source language string is **'%1 is not secure. Click here to turn on VPN and secure your device.'**, the translated string is **'%1 n’est pas sécurisé. Cliquez ici pour activer le VPN et sécuriser votre appareil.'**, and the comment provides additional information for the translation. | ||
|
|
||
| Mozilla VPN generates TS files during the build process from Pontoon’s XLIFF files (details provided below). | ||
|
|
||
| - **Compiling to Binary (QM Files):** Each TS file is compiled into a QM binary file and then added to the app’s binary using the [Qt Resource System](https://doc.qt.io/qt-6/resources.html). For instance, **‘mozillavpn_fr.ts’** is compiled and added to the app’s binary as a **‘qrc:/i18n/mozillavpn_fr.qm’** resource. | ||
|
|
||
| * **Localized String Loading:** Qt’s[ QTranslator](https://doc.qt.io/qt-6/qtranslator.html) is used to get a localized string when **qsTrId(_stringID_)** or **qtTrId(_stringID_)** are called. For example **qsTrId("vpn.notifications.unsecuredNetworkMessage")** fetches the translated string from the currently loaded translators. | ||
|
|
||
| - **Formatting with QLocale:** Qt’s [QLocale ](https://doc.qt.io/qt-6/qlocale.html)is used for formatting data, such as date, time and currency. | ||
|
|
||
|
|
||
| ## VPN’s Localizer Class | ||
|
|
||
| VPN’s **Localizer** class uses Qt’s QTranslator to load a QM translation file resource corresponding to a given locale. Since some translations may not be available, a fallback mechanism is used: | ||
|
|
||
| - First, an English translator is loaded as the base, because it has all the strings. Then translators for any additional language fallbacks, as specified in ‘**src\translations\extras\translations_fallback.json’,** are loaded. Finally, the translator for the user’s selected locale is loaded. Translators are available only if a predefined threshold percentage of strings have been translated. | ||
|
|
||
| - When **qsTrId(_stringID_)**is used to get a localized string, Qt uses the most recently loaded translator first. If the string is not found, Qt searches backward through the ordered list of translators until it reaches the first translator. | ||
|
|
||
| The **Localizer** class uses the QLocale class for requests to format dates and currency. | ||
|
|
||
| ## Translating Strings with Pontoon | ||
|
|
||
| [Pontoon](https://pontoon.mozilla.org/) is a tool used by volunteers to translate strings. Periodically, a GitHub action extracts strings from the VPN repository and sends them to the Pontoon repository at [**https://github.com/mozilla-l10n/mozilla-vpn-client-l10n**](https://github.com/mozilla-l10n/mozilla-vpn-client-l10n). | ||
|
|
||
| The strings are sent in a XML Localization Interchange File Format ([XLIFF](http://docs.oasis-open.org/xliff/xliff-core/v2.1/xliff-core-v2.1.html)) file, for each locale. Here is an example for French: | ||
|
|
||
| **fr/mozillavpn.xliff:** | ||
|
|
||
| ```xml | ||
| <file original=**"generated/l18nstrings_p.cpp"** datatype="cpp" source-language="en" target-language="fr"> | ||
| <body> | ||
| <trans-unit id=**"vpn.notifications.unsecuredNetworkMessage"** xml:space="preserve"> | ||
| <note annotates="source" from="developer">%1 is the Wi-Fi network name</note> | ||
| <target>%1 n’est pas sécurisé. Cliquez ici pour activer le VPN et sécuriser votre appareil.</target> | ||
| <source>%1 is not secure. Click here to turn on VPN and secure your device.</source> | ||
| </trans-unit> | ||
| ``` | ||
|
|
||
| Each string entry contains the string ID, the English string, and a comment providing guidance for the translation. These entries are grouped into sections based on the file that created the string. For strings in strings.yaml, the string is created by the **‘generated\i18nstrings_p.cpp’** file**.** | ||
|
|
||
| Pontoon detects changes in the XLIFF file by checking attributes like **‘original’** in the **‘file’** tag, **‘id**’ in the **‘trans-unit**’ tag, and the **‘source’** text. If there are changes or new strings, Pontoon notifies volunteers about the translation work needed. Once volunteers complete the translation for a string in a specific locale, Pontoon updates the **'target'** text with the translation. | ||
|
|
||
| The Pontoon repository is a submodule of the VPN repository, allowing VPN to obtain the XLIFF file changes. During the build process, a corresponding translation source file (TS file) is generated for each locale from the XLIFF, compiled into a QM file, and added to the VPN binary as a resource. The TS and QM files can be found in the **“build\translations\generated”** directory.\ | ||
| The completeness of translations in a locale is provided in the generated **“build\translations\generated\translations.completeness"** file. The file provides a score from 0 to 1, where 0 means not done at all, and 1 means fully completed. For example: | ||
|
|
||
| ``` | ||
| es_MX:0.8287937743190662 | ||
| fr:1.0 | ||
| ``` | ||
|
|
||
| In this example, French has a score of 1.0, meaning that it is 100% translated. Spanish - Mexico has a score of about 0.83, indicating that around 83% of strings have been translated. | ||
|
|
||
| The completeness score is used to determine if TS files for that locale should be generated. | ||
|
|
||
| ## Special Cases: server and language names | ||
|
|
||
| Language name and server name strings are special cased in the Mozilla VPN localization pipeline. | ||
|
|
||
| Although these strings are still translated by volunteers in Pontoon, they are declared on the `src/translations/extras.xliff` file and the process to add, remove and edit these strings is not the same as for any other string in the application. | ||
|
|
||
| ### Language names | ||
|
|
||
| Language name strings are used in the languages chooser view of the application. This view shows both the language name translated in the currently active language and the native language name. | ||
|
|
||
| #### Native language name translation | ||
|
|
||
| In order to load the native language name translation without having to load the full `QTranslator` for each language whenever the language chooser menu is opened, the native language name translations are added to a map created at build time by the `generate_language_names_map.py` script. The resulting map contains a dictionary with the language code and the native language name translation: | ||
|
|
||
| ```cpp | ||
| namespace LanguageStrings { | ||
| const QMap<QString, QString> NATIVE_LANGUAGE_NAMES = { | ||
| {"sl", "slovenščina"}, | ||
| {"sk", "slovenčina"}, | ||
| {"pa_IN", "ਪੰਜਾਬੀ"}, | ||
| {"pl", "Polski"}, | ||
| ... | ||
| ``` | ||
|
|
||
| This map can be imported in C++ code through the header `i18nlanguagenames.h`. | ||
|
|
||
| #### Adding, removing and editing language names | ||
|
|
||
| ##### Adding | ||
|
|
||
| The `extras.xliff` file must always contain strings for all languages supported by the application. In order to know which languages are supported by the application, one can simply look at the folder list in the i18n submodule. To guarantee the `extras.xliff` file and the supported language list is always up to date, a linter workflow is executed on each commit that checks if the i18n submodule and the `extras.xliff` language names are up to date. | ||
|
|
||
| ##### Removing | ||
|
|
||
| The linter does not check if a language name string must be _removed_ from `extras.xliff`. When that is case, it is up to developers to do so and the process is as simple as deleting the XML node that includes the unwanted string. | ||
|
|
||
| ##### Editing | ||
|
|
||
| Just like strings in `strings.yaml` a string in `extras.xliff` should not be edited. If a language name base translation needs to be edited, it will need to be special cased in all scripts and a new XML node will need to be added to the `extras.xliff` file such e.g. `pt_BR2` in case the base translation for Brazilian Portuguese need to be changed. | ||
|
|
||
| **It is best not to edit these base translations and only do so in when strictly required**. | ||
|
|
||
| ### Server names | ||
|
|
||
| Server names are strings of country and city names where VPN servers are available. These strings are used in the server list view of the application. They are inherently dynamic, since servers may be added or removed from the server list at any time. | ||
|
|
||
| In order to keep these strings up to date a CI Script it run at 7AM UTC every Monday to fetch the latest server list from Mullvad and update the `extras.xliff` file as needed -- this script can be found at `.github/workflows/i18n_server_list_update.yaml`. If everything is working, developers should not have to manually update server names in the `extras.xliff` file, they would only ever need to approve and merge these automatic pull requests. | ||
|
|
||
| Just like the language names CI check, this script will never check for strings that need to be removed. It will only ever _add_ new strings and it is up to developers to remove strings when that is necessary. | ||
|
|
||
| Editing these strings is inadvisable for the same reasons as language name strings. |