From 140c4581c680beea8829810456bb95a21503d38d Mon Sep 17 00:00:00 2001 From: Marcel Klehr Date: Fri, 27 Sep 2024 10:31:22 +0200 Subject: [PATCH 1/3] Admin docs: Add a note about VRAM usage Signed-off-by: Marcel Klehr --- admin_manual/ai/app_assistant.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/admin_manual/ai/app_assistant.rst b/admin_manual/ai/app_assistant.rst index 04859678741..abb5e00af08 100644 --- a/admin_manual/ai/app_assistant.rst +++ b/admin_manual/ai/app_assistant.rst @@ -38,13 +38,17 @@ Artificial intelligence at Nextcloud is built in a modular way, allowing you to **Note**: At Nextcloud we focus on creating on-premise AI apps that run fully self-hosted on your own servers in order to preserve your privacy and data sovereignty. However, you can also offload these resource-heavy tasks to an :ref:`"AI as a Service" provider`. +**Note**: When using our on-premise AI apps, make sure you have a GPU with enough VRAM that fits all the features you need. For each app documented here you will find its hardware requirements. + Machine translation ~~~~~~~~~~~~~~~~~~~ In order to make use of machine translation features in the assistant, you will need an app that provides a translation backend: +* :ref:`translate` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request) * :ref:`translate2 (ExApp)` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request) * *integration_deepl* - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported) +* *integration_libretranslate* - Integrates with the open source LibreTranslate API to provide translation functionality hosted commercially or on your own hardware (Only community supported) Speech-To-Text ~~~~~~~~~~~~~~ @@ -67,7 +71,7 @@ Text-To-Image In order to make use of Text-To-Image features, you will need an app that provides an image generation backend: -* text2image_stablediffusion (Customer support available upon request) +* text2image_stablediffusion2 (Customer support available upon request) * *integration_openai* - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service`) * *integration_replicate* - Integrates with the replicate API to provide AI functionality from replicate servers (see :ref:`AI as a Service`) From 80b467930f1c7b9574b60bc99fab91ac5ec78f31 Mon Sep 17 00:00:00 2001 From: Marcel Klehr Date: Fri, 27 Sep 2024 10:38:29 +0200 Subject: [PATCH 2/3] TaskProcessing docs: Add a note about multilingual prompts and testing Signed-off-by: Marcel Klehr --- .../digging_deeper/task_processing.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/developer_manual/digging_deeper/task_processing.rst b/developer_manual/digging_deeper/task_processing.rst index b602d78801e..3519d6f7bd3 100644 --- a/developer_manual/digging_deeper/task_processing.rst +++ b/developer_manual/digging_deeper/task_processing.rst @@ -87,6 +87,21 @@ The following built-in task types are available: * Output shape: * ``output``: ``ListOfImages`` +LLM Prompts and multilingual I/O +################################ + +When writing prompts for the TextToText task type in your apps, we recommend testing it with at least + +* OpenAI GPT-3.5 +* Llama 3.1 + +Also, make sure that you instruct the model to use the correct language in its output. By default most models will answer in English if the main prompt is in English, even though the source data is in another language. +A tweak to make sure of this is to instruct the model as follows: + +.. code-block:: php + + "Detect the language used in the text and make sure to answer in the same language without mentioning the language explicitly." + Input and output shapes ~~~~~~~~~~~~~~~~~~~~~~~ From d04343657d7fa4a40687c7fb9cf3aeaca87c1f29 Mon Sep 17 00:00:00 2001 From: Marcel Klehr Date: Fri, 27 Sep 2024 10:38:29 +0200 Subject: [PATCH 3/3] fix(AI/assistant): Don't mention old apps Signed-off-by: Marcel Klehr --- admin_manual/ai/app_assistant.rst | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/admin_manual/ai/app_assistant.rst b/admin_manual/ai/app_assistant.rst index abb5e00af08..24e93134e0f 100644 --- a/admin_manual/ai/app_assistant.rst +++ b/admin_manual/ai/app_assistant.rst @@ -44,11 +44,8 @@ Machine translation ~~~~~~~~~~~~~~~~~~~ In order to make use of machine translation features in the assistant, you will need an app that provides a translation backend: - -* :ref:`translate` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request) * :ref:`translate2 (ExApp)` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request) * *integration_deepl* - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported) -* *integration_libretranslate* - Integrates with the open source LibreTranslate API to provide translation functionality hosted commercially or on your own hardware (Only community supported) Speech-To-Text ~~~~~~~~~~~~~~ @@ -71,7 +68,7 @@ Text-To-Image In order to make use of Text-To-Image features, you will need an app that provides an image generation backend: -* text2image_stablediffusion2 (Customer support available upon request) +* text2image_stablediffusion (Customer support available upon request) * *integration_openai* - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service`) * *integration_replicate* - Integrates with the replicate API to provide AI functionality from replicate servers (see :ref:`AI as a Service`)