Update worker docs with recent enhancements #7969

erikjohnston · 2020-07-28T10:26:49Z

We also add some of the new config options to the sample config.

This also updates the wording to recommend using Redis and removing suggestion that it is experimental.

Apologies in advance that me bad at words.

docs/workers.md

clokep · 2020-07-28T15:52:05Z

synapse/config/federation.py

+        # If running with federation sender worker instances then they should
+        # be listed by their `worker_name` here, as well as setting
+        # `send_federation` to false.
+        #
+        #send_federation: false
+        #
+        #federation_sender_instances:
+        #  - federation_sender1


I'm not really sure what this means. I think it can be made clearer that this is what gets set in the shared config?

How about now? I'm not 100% convinced about the new wording

I think it is much clearer. Reading workers.md also helps a lot. :)

synapse/config/redis.py

synapse/config/workers.py

docs/workers.md

clokep · 2020-07-28T16:17:16Z

docs/workers.md

+Additionally, processes may make HTTP requests to each other. Typically this is
+used for operations which need to wait for a reply - such as sending an event.
+
+As of Synapse v1.13.0, it is possible to configure Synapse to send replication


It could be nice to have a snazzy ASCII art diagram here of the processes communicating...

does https://matrix.sw1v.org/_matrix/media/r0/download/sw1v.org/uhqIFIqONRWcRsKgVvbehsqF help? some of the links have been left out for clarity but I think it might help?

Possibly? I'm not really sure who it's aimed at.

Am happy to put it at the end of the doc, to be clear.

I'm not really sure who it's aimed at.

well, people trying to set up workers. The idea is that a picture helps people get an overview of how it works.

Ok, will add it to the end with a note at the start pointing to it 👍

clokep

Looks good! Can you make sure to link the various documentation issues with this PR?

erikjohnston · 2020-07-28T16:44:08Z

This needs repointing at release branch.

richvdh

some initial comments, but I need to go and make dinner

docs/sample_config.yaml

richvdh · 2020-07-28T17:29:52Z

docs/sample_config.yaml

+# Configuration for Redis when using workers.
+#
+redis:
+  # Uncomment the below to enable using Redis to replicate data between workers.


what happens if it's not enabled?

Hmm, I guess we should reword this to make it clear that it needs (unless using old style replication) to be enabled if using workers

richvdh · 2020-07-28T17:35:32Z

docs/workers.md

+Additionally, processes may make HTTP requests to each other. Typically this is
+used for operations which need to wait for a reply - such as sending an event.
+
+As of Synapse v1.13.0, it is possible to configure Synapse to send replication


docs/workers.md

richvdh

lgtm otherwise

docs/workers.md

richvdh · 2020-07-29T15:53:56Z

hrm, there was more that github has eaten

docs/workers.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

…bership_join_count * 'develop' of github.com:matrix-org/synapse: Update workers docs (#7990) Fix invite rejection when we have no forward-extremeties (#7980) Fix typo in docs/workers.md (#7992) Convert federation client to async/await. (#7975) Convert appservice to async. (#7973) Convert some of the data store to async. (#7976) Fix formatting of changelog and upgrade notes Ensure that remove_pusher is always async (#7981) Add deprecation warnings 1.18.0 Update worker docs with recent enhancements (#7969) Ensure the msg property of HttpResponseException is a string. (#7979) Remove from the event_relations table when purging historical events. (#7978) Add additional logging for SAML sessions. (#7971) Add MSC reference to changelog for #7736 Re-implement unread counts (#7736) Various improvements to the docs (#7899)

* commit 'a9631b7b4': 1.18.0 Update worker docs with recent enhancements (#7969)

erikjohnston force-pushed the erikj/worker_docs branch 2 times, most recently from f6b63fb to 226a0b6 Compare July 28, 2020 10:50

erikjohnston requested a review from a team July 28, 2020 12:44

clokep self-assigned this Jul 28, 2020

babolivier reviewed Jul 28, 2020

View reviewed changes

docs/workers.md Outdated Show resolved Hide resolved

clokep suggested changes Jul 28, 2020

View reviewed changes

clokep reviewed Jul 28, 2020

View reviewed changes

erikjohnston requested a review from a team July 28, 2020 16:37

clokep approved these changes Jul 28, 2020

View reviewed changes

This was linked to issues Jul 28, 2020

Document how to configure Synapse to use redis with workers #7833

Closed

Document that you can run multiple federation readers to handle /send/ #7832

Closed

Document how to run multiple federation sender workers #7831

Closed

richvdh reviewed Jul 28, 2020

View reviewed changes

erikjohnston force-pushed the erikj/worker_docs branch from d239cf5 to 5dc360e Compare July 29, 2020 09:25

erikjohnston requested a review from richvdh July 29, 2020 09:49

erikjohnston unassigned clokep Jul 29, 2020

richvdh reviewed Jul 29, 2020

View reviewed changes

erikjohnston requested a review from richvdh July 29, 2020 13:44

richvdh approved these changes Jul 29, 2020

View reviewed changes

docs/workers.md Show resolved Hide resolved

erikjohnston force-pushed the erikj/worker_docs branch from b745743 to bc951a2 Compare July 29, 2020 15:58

richvdh reviewed Jul 29, 2020

View reviewed changes

docs/workers.md Show resolved Hide resolved

erikjohnston and others added 7 commits July 29, 2020 23:20

Update worker documentation with latest additions

2278562

Add some of the new worker options to sample config

a880415

Newsfile

43df3fd

Update docs/workers.md

48ba564

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

Update synapse/config/redis.py

5494d0c

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Update synapse/config/redis.py

3e6af96

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Update synapse/config/workers.py

8bd0b08

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

erikjohnston and others added 19 commits July 29, 2020 23:20

Update docs/workers.md

b91e4fb

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Apply suggestions from code review

3dd4b73

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

s/master/main/

1fe0335

Add back some notes about redis dependencies

dad0be0

Clarify the 'federation_sender_instances' config

31191fa

sample config

5d8a970

Review suggestions

196e585

Add notes about installing Redis

21ef5cd

Move fedearation shard config to worker config

8741ac7

Note that redis must be enabled when using workers

c375b15

Fix sample config

63263ef

Apply suggestions from code review

6c78895

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Move config file explanation up

9d891f8

Move note about historical apps down

af26701

Doc worker_listeners

a236824

Specify what the room ID is in the path

753c4f3

Mention explicitly which streams can be moved off master

308b181

Move synctl worker config to seperate file

09aebbc

Add architectural diagram

49e804b

erikjohnston force-pushed the erikj/worker_docs branch from 7b76a03 to 49e804b Compare July 29, 2020 22:21

erikjohnston changed the base branch from develop to release-v1.18.0 July 29, 2020 22:21

erikjohnston merged commit 2c1b9d6 into release-v1.18.0 Jul 29, 2020

erikjohnston deleted the erikj/worker_docs branch July 29, 2020 22:22

erikjohnston mentioned this pull request Jul 30, 2020

Fixup worker doc (again) #8000

Merged

babolivier pushed a commit that referenced this pull request Sep 1, 2021

Merge commit 'a9631b7b4' into anoa/dinsic_release_1_18_x

0b2c404

* commit 'a9631b7b4': 1.18.0 Update worker docs with recent enhancements (#7969)

richvdh mentioned this pull request Aug 10, 2022

Fix worker doc for synapse.app.frontend_proxy #13451

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update worker docs with recent enhancements #7969

Update worker docs with recent enhancements #7969

erikjohnston commented Jul 28, 2020 •

edited

Loading

clokep Jul 28, 2020

erikjohnston Jul 28, 2020

clokep Jul 28, 2020

clokep Jul 28, 2020

richvdh Jul 28, 2020

richvdh Jul 29, 2020

erikjohnston Jul 29, 2020

erikjohnston Jul 29, 2020

richvdh Jul 29, 2020

erikjohnston Jul 29, 2020

clokep left a comment

erikjohnston commented Jul 28, 2020

richvdh left a comment

richvdh Jul 28, 2020

erikjohnston Jul 29, 2020

richvdh Jul 28, 2020

richvdh left a comment

richvdh commented Jul 29, 2020

Update worker docs with recent enhancements #7969

Update worker docs with recent enhancements #7969

Conversation

erikjohnston commented Jul 28, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clokep left a comment

Choose a reason for hiding this comment

erikjohnston commented Jul 28, 2020

richvdh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

richvdh commented Jul 29, 2020

erikjohnston commented Jul 28, 2020 •

edited

Loading