Machine translation option a la FB or Twitter etc

[ara4n]

[ara4n]

It's starting to jar that we miss this, especially when other fediversey apps have it (e.g. Mastalab).

Options could include Deepl.com, or stuff like Yandex. Needless to say, it would have to be opt-in per message to avoid a massive privacy leak.

[Half-Shot]

What would trigger the translation option in the first place, would messages need to start coming tagged with their language, or would we need to pass the body through some (thirdparty?) filter to check if a message needs translating?

This probably needs a a spec :)

[emorrp1]

I don't know about a spec, but I can flesh out a feature description based on my thoughts over the last couple of days.

Blue Sky Design

Assume that in 10 years time there's a libre self-hostable translation service and of course riot is the world # 1 chat client. As a poor monolingual user, I would like to view all messages in a e2ee room post-translation, in order to chat freely with a foreign language community.

To make this work, riot could have an account language preference and a homeserver preferred translation service url. Riot sends all messages through translation and displays the original if the service replies that it's already in the target language; otherwise displays the translated text, probably with an original language sticker and I can click View Source to see the original message json.

As a multi-lingual room owner, I want to compose an announcement in multiple languages, so that my community can view it in their native language, not machine translated. The published matrix message has content.languages = ["de", "en-AU", "ar-QA"] (ISO 639) and corresponding content.de.formatted_body etc. For multi-lingual users, select the best match to their account preferences and display that untranslated, otherwise display machine-translation.

Practical Solution

• Don't support e2ee rooms
• Add a button somewhere to copy the last N messages (<5000 characters) to clipboard so I can easily paste into a web translator (maybe something like mark start message -> mark end message -> copy IRC log style)
• Implement as an AppService with a configurable DeepL API key. Add "See Translation" button to message drop-down if the AppService is added to a room?

Rationale

#freenode_#play.it:matrix.org is a french/english bilingual channel and at the moment if the discussion is in french, I have to manually select and copy as much as I can to paste into the web translator. Rinse and repeat for longer chats, and then I understand the gist of the conversation and can reply (in english) if needed.

Another option would be to have a translating bridge, so it would create separate _en and _fr rooms for the corresponding monolingual users to join. That would leak the entire conversation and split the users into multiple independent rooms, so they wouldn't see each other as native matrix users. It's also not scalable, would be nice to do the same for native matrix rooms and ultimately it's an per-user preference, not a per-room one.

[emorrp1]

Some resources:

https://github.com/stom79/mytransl

MyTransL is a small library which allows to translate text using Yandex Translate or the DeepL API

You need at least to create a free Yandex translate API key or your DeepL API key.

https://engineering.fb.com/ml-applications/expanding-automatic-machine-translation-to-more-languages/
https://github.com/tomouchuu/mastodon-translate
https://fedilab.app/post/free-yandex-api-key/

[resynth1943]

@emorrp1 automatically translating messages via an online service is terrible for privacy.

Needless to say, it would have to be opt-in per message to avoid a massive privacy leak.

---

Something like this might work:

You need at least to create a free Yandex translate API key or your DeepL API key.

That would be up to individual Element hosts (e.g. app.element.io), not the Element team.

But generally speaking it would be better to provide a unified translation interface (a la /src/customisations) instead of hard-coding support for a few providers.

[emorrp1]

holy shit! https://libretranslate.com/

100% Self-Hosted. No Limits. No Ties to Proprietary Services.

$ http https://libretranslate.com/translate source=fr target=en \
    q='Je crois que Curse of Monkey Island (Monkey Island 3) propose un système de difficulté.' \
    | jq -r .translatedText
I think Curse of Monkey Island (Monkey Island 3) offers a system of difficulty.

No need for an api key, please, please implement a PoC!

[t3chguy]

please implement a PoC

Its open source, you're welcome to implement it behind Labs yourself.

[emorrp1]

MSC3554 Translatable Messages matrix-org/matrix-spec-proposals#3554