Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation for Manage Translations #4470

Merged
merged 6 commits into from
Aug 23, 2018
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
229 changes: 229 additions & 0 deletions docs/guides/manage-translations.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
Manage Translations
===================

This guide walks through the process needed to manage translations of your documentation.
Once this work is done, you can setup your project under Read the Docs to build each language of your documentation by reading :doc:`/localization`.

Overview
--------

There are many different ways to manage documentation in multiple languages by using different tools or services.
All of them have their pros and cons depending on the context of your project or organization.

In this guide we will focus our efforts around two different methods: manual and using Transifex_.

In both methods, we need to follow these steps to translate our documentation:

#. Create translatable files (``.pot`` and ``.po`` extensions) from source language
#. Translate the text on those files from source language to target language
#. Build the documentation in *target language* using the translated texts

Besides these steps, once we have published our first translated version of our documentation,
we will want to keep it updated from the source language. At that time, the workflow would be:

#. Update our translatable files from source language
#. Translate only *new* and *modified* texts in source language into target language
#. Build the documentation using the most up to date translations


Create translatable files
-------------------------

To generate these ``.pot`` files it's needed to run this command from your ``docs/`` directory:

.. code-block:: console

$ sphinx-build -b gettext . _build/gettext

.. tip::

We recommend configuring Sphinx to use :ref:`sphinx:std:confval:gettext_uuid` as ``True``
and also :ref:`sphinx:std:confval:gettext_compact` as ``False`` to generate ``.pot`` files.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't make intersphinx work here.



This command will leave the generated files under ``_build/gettext``.


Translate text from source language
-----------------------------------

Manually
~~~~~~~~

We recommend using `sphinx-intl`_ tool for this workflow.

.. _sphinx-intl: https://pypi.org/project/sphinx-intl/

First, we need to install it:

.. code-block:: console

$ pip install sphinx-intl


As a second step, we want to create a directory with each translated file per target language
(in this example we are using Spanish/Argentina and Portuguese/Brazil).
This can be achieved with the following command:

.. code-block:: console

$ sphinx-intl update -p _build/gettext -l es_AR -l pt_BR

This command will create a directory structure similar to the following
(with one ``.po`` file per ``.rst`` file in your documentation)::

locale
├── es_AR
│ └── LC_MESSAGES
│ └── index.po
└── pt_BR
└── LC_MESSAGES
└── index.po


Now, you can just open those ``.po`` files with a text editor and translate them taking care of no breaking the reST notation.
Example:

.. code-block:: po

# b8f891b8443f4a45994c9c0a6bec14c3
#: ../../index.rst:4
msgid ""
"Read the Docs hosts documentation for the open source community."
"It supports :ref:`Sphinx <sphinx>` docs written with reStructuredText."
msgstr ""
"FILL HERE BY TARGET LANGUAGE FILL HERE BY TARGET LANGUAGE FILL HERE "
"BY TARGET LANGUAGE :ref:`Sphinx <sphinx>` FILL HERE."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a semi-manual process, it may be interesting to mention specialized (but non-cloud) tools such as poedit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it!



Using Transifex
~~~~~~~~~~~~~~~

Transifex_ is a platform that simplifies the manipulation of ``.po`` files and offers many useful features to make the translation process as smooth as possible.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess given we're about to make a nice advertising for Transifex services (which is fine by me), it would be fair to mention the competition and their relationship with open-source projects.

Transifex:

We offer Transifex for free to Open Source projects that have no funding, revenue, and/or commercialization model
https://docs.transifex.com/projects/open-source-project

Crowdin:
Free too, with conditions

Zanata:
Free, open source, self-hostable (by Red Hat), but as far as I know, less features
http://zanata.org/

And then there's PhraseApp, Smartling and many others.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

This was something that bothered to me while I was writing this guide. The question was: "Why we are suggesting a non-free solution?" I read that note that you mention here, but it still made me noise at that time.

I commented this to @agjohnson and he said that maybe it does worth to have multiples guides instead just one. Like:

  • How to use Transifex and Read the Docs?
  • How to use Crowdin and Read the Docs?
  • etc

I think this is the direction that I would like to follow, but I do not have experience with these other tools and this will take me some time to read, test the flow and write the docs. So, I'd say that we could have a initial guide (as this one) and work in the other guides soon.

Maybe, we could mention the other solutions (name and link) and add a note saying that we are working on writing other guides for them and contributions are welcomed :)

What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, you seem way more ambitious than me :D I would tend to just mention names and links as you said, and maybe a link to every project documentation.

If those projects have documentation, then they might be more precise and up to date than ours will be (given we're really following a classic use case here).

If those projects don't have documentation... Do we really want to link to them ? :D

That being said, I'm totally ok keeping the transifex doc, especially now it's been written, because if there's at least one doc, then it's easier to understand what we're trying to do and adapt it to another tool.

Lastly, there's another alternative if you want to explore it : a "logical" documentation that would explain the steps and you can see examples with different tools. Something like

Upload the sources:
with transifex | with crowdin | ...
[panel that would change depending on the click above]

This would maybe help people understand that whatever the tools, the steps are the same.

These features includes a great web based UI, `Translation Memory`_, collaborative translation, etc.

.. _Transifex: https://www.transifex.com/
.. _Translation Memory: https://docs.transifex.com/setup/translation-memory

You need to create an account in their service and a new project before start.

After that, you need to install the `transifex-client`_ tool which will help you in the process to upload source files, update them and also download translated files.
To do this, run this command:

.. _transifex-client: https://docs.transifex.com/client/introduction

.. code-block:: console

$ pip install transifex-client

After installing it, you need to configure your account.
For this, you need to create an API Token for your user to access this service through the command line.
This can be done under your `User's Settings`_.

.. _User's Settings: https://www.transifex.com/user/settings/api/


Now, you need to setup it to use this token:

.. code-block:: console

$ tx init --token $TOKEN --no-interactive


The next step is to map every ``.pot`` file you have created in the previous step to a resource under Transifex.
To achieve this, you need to run this command:

.. code-block:: console

$ tx config mapping-bulk --project $TRANSIFEX_PROJECT --file-extension '.pot' --source-file-dir docs/_build/gettext --source-lang en --type PO --expression 'docs/locale/<lang>/LC_MESSAGES/{filepath}/{filename}.po' --execute

This command will generate a file at ``.tx/config`` with all the information needed by the ``transifext-client`` tool to keep our translation synchronized.

Finally, you need to upload these files to Transifex platform so translators can start their work.
To do this, you can run this command:

.. code-block:: console

$ tx push --source


Now, you can go to your Transifex's project and check that there is one resource per ``.rst`` file of your documentation.

After the source files are translated using Transifex, you can download the all translations for all the languages by running:

.. code-block:: console

$ tx pull --all

This command will leave the ``.po`` files needed for building the documentation in the target language under ``locale/<lang>/LC_MESSAGES``.

.. warning::

It's important to use always the same method to translate the documentation and do not mix them.
Otherwise, it's very easy to end up with inconsistent translation or loosing already translated text.


Build the documentation in target language
------------------------------------------


Finally, to build our documentation in Spanish(Argentina) we need to tell Sphinx builder the target language with the following command:

.. code-block:: console

$ sphinx-build -b html -D language=es_AR . _build/html/es_AR

.. note::

There is no need to create a new ``conf.py`` to redefine the ``language`` for the Spanish version of this documentation.

After running this command, the Spanish(Argentina) version of your documentation will be under ``_build/html/es_AR``.


Summary
-------

Update sources to be translated
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Once we have done changes in our documentation, we want to make these additions/modifications available for translators so they can update it:

#. Create the ``.pot`` files:

.. code-block:: console

$ sphinx-build -b gettext . _build/gettext


.. For the manual workflow, we need to run this command
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm far (like, quite) from an rst expert, and I don't know enough of the syntax to understand the #. but the fact this one is the only one to begin differently seems noticeable (I bet I'm gonna learn something today :D )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

He he :)

The #. is to define a enumerated (automatically) list: http://docutils.sourceforge.net/docs/user/rst/quickref.html#enumerated-lists

When the line starts with .. it's just a comment that won't appear in the renderized (html, pdf, etc) version: http://docutils.sourceforge.net/docs/user/rst/quickref.html#comments

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But then I don't understand wyl this line in particular should be a comment ?

(Or was your "he he" a mark that it shouldn't ?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But then I don't understand wyl this line in particular should be a comment ?

I noted as a comment because I wanted to mention it but I thought that it breaks the summary that I was describing, which it was all based on Transifex since it's the one that we use under RTD.

(Or was your "he he" a mark that it shouldn't ?)

Nope, it didn't mean that. Just made me fun that there was a comment in the middle of nowhere without any kind of explanation :)

I'm not sure yet how to include it. I mean, maybe this section needs some more work to make the mention of these command in a proper way.


$ sphinx-intl update -p _build/gettext -l es_AR -l pt_BR


#. Push new files to Transifex

.. code-block:: console

$ tx push --sources


Build documentation from up to date translation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When translators have finished their job, we want to update the documentation by pulling the changes from Transifex:

#. Pull up to date translations from Transifex:

.. code-block:: console

$ tx pull --all

#. Commit and push these changes to our repo

.. code-block:: console

$ git add docs/locale/
$ git commit -m "Update translations"
$ git push

The last ``git push`` will trigger a build per translation defined as part of our project under Read the Docs and make it immediately available.