Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Handle inbound events from federation asynchronously #10272

Merged
merged 6 commits into from
Jun 29, 2021
Merged

Handle inbound events from federation asynchronously #10272

merged 6 commits into from
Jun 29, 2021

Conversation

erikjohnston
Copy link
Member

Fixes #9490

Based on #10269

This will break a couple of SyTest that are expecting failures to be added to the response of a federation /send, which obviously doesn't happen now that things are asynchronous.

Two drawbacks:

  1. Currently there is no logic to handle any events left in the staging area after restart, and so they'll only be handled on the next incoming event in that room. That can be fixed separately.
  2. We now only process one event per room at a time. This can be fixed up further down the line.

@erikjohnston erikjohnston changed the base branch from develop to erikj/async_federation_base_branch June 29, 2021 17:40
erikjohnston
erikjohnston commented Jun 29, 2021
View reviewed changes
synapse/federation/federation_server.py

# The common path is for the event we just received be the only event in
# the room, so instead of pulling the event out of the DB and parsing
# the event we just pull out the next event ID and check if that matches.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I don't know whether the extra complexity is worth saving fetching a single event from the DB, happy to delete this logic 🤷

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤷

@erikjohnston erikjohnston force-pushed the erikj/async_inbound_federation branch from f265174 to c2707da Compare June 29, 2021 17:42
richvdh
richvdh reviewed Jun 29, 2021
View reviewed changes
Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks generally about right

synapse/storage/schema/main/delta/59/16federation_inbound_staging.sql Outdated Show resolved Hide resolved
synapse/storage/schema/main/delta/59/16federation_inbound_staging.sql Outdated Show resolved Hide resolved
synapse/federation/federation_server.py Outdated Show resolved Hide resolved
synapse/storage/databases/main/event_federation.py Outdated Show resolved Hide resolved
synapse/storage/databases/main/event_federation.py Outdated Show resolved Hide resolved
synapse/storage/databases/main/event_federation.py Outdated Show resolved Hide resolved
erikjohnston and others added 6 commits June 29, 2021 19:32
@erikjohnston @richvdh
Handle inbound federation transactions asynchronously
0cc2816
@erikjohnston @richvdh
Newsfile
d541c74
@erikjohnston @clokep @richvdh
Apply suggestions from code review
3d096ce 
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
@erikjohnston @richvdh
Remove room version in table
57e1f96
@erikjohnston @richvdh
Update synapse/storage/schema/main/delta/59/16federation_inbound_stag…
5c94b10 
…ing.sql

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
@erikjohnston @richvdh
Update blacklist with now failing tests
f43cfe8
@richvdh richvdh force-pushed the erikj/async_inbound_federation branch from 280baea to f43cfe8 Compare June 29, 2021 18:32
@richvdh richvdh merged commit c54db67 into erikj/async_federation_base_branch Jun 29, 2021
@richvdh richvdh deleted the erikj/async_inbound_federation branch June 29, 2021 18:55
@richvdh richvdh mentioned this pull request Jun 29, 2021
Closed
9 tasks
aaronraimist added a commit to aaronraimist/synapse that referenced this pull request Jun 30, 2021
@aaronraimist
Merge tag 'v1.37.1rc1' into 2021-05
36fff27 
Synapse 1.37.1rc1 (2021-06-29)
==============================

Features
--------

- Handle inbound events from federation asynchronously. ([\matrix-org#10269](matrix-org#10269), [\matrix-org#10272](matrix-org#10272))
@erikjohnston erikjohnston mentioned this pull request Jun 30, 2021
Merged
@erikjohnston erikjohnston mentioned this pull request Jun 30, 2021
Merged
erikjohnston added a commit that referenced this pull request Jun 30, 2021
@erikjohnston
Fix the inbound PDU metric
7f96eff 
This broke in #10272
erikjohnston added a commit that referenced this pull request Jun 30, 2021
@erikjohnston
Fix the inbound PDU metric (#10279)
329ef5c 
This broke in #10272
aaronraimist added a commit to aaronraimist/synapse that referenced this pull request Jul 1, 2021
@aaronraimist
Merge tag 'v1.37.1' into 2021-05
a557b5b 
Synapse 1.37.1 (2021-06-30)
===========================

This release resolves issues (such as [matrix-org#9490](matrix-org#9490)) where one busy room could cause head-of-line blocking, starving Synapse from processing events in other rooms, and causing all federated traffic to fall behind. Synapse 1.37.1 processes inbound federation traffic asynchronously, ensuring that one busy room won't impact others. Please upgrade to Synapse 1.37.1 as soon as possible, in order to increase resilience to other traffic spikes.

No significant changes since v1.37.1rc1.

Synapse 1.37.1rc1 (2021-06-29)
==============================

Features
--------

- Handle inbound events from federation asynchronously. ([\matrix-org#10269](matrix-org#10269), [\matrix-org#10272](matrix-org#10272))
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jul 1, 2021
@0-wiz-0
matrix-synapse: update to 1.37.1.
916610a 
Synapse 1.37.1 (2021-06-30) ===========================

This release resolves issues (such as
[#9490](matrix-org/synapse#9490)) where
one busy room could cause head-of-line blocking, starving Synapse
from processing events in other rooms, and causing all federated
traffic to fall behind. Synapse 1.37.1 processes inbound federation
traffic asynchronously, ensuring that one busy room won't impact
others. Please upgrade to Synapse 1.37.1 as soon as possible, in
order to increase resilience to other traffic spikes.

No significant changes since v1.37.1rc1.


Synapse 1.37.1rc1 (2021-06-29) ==============================

Features --------

- Handle inbound events from federation asynchronously.
([\#10269](matrix-org/synapse#10269),
[\#10272](matrix-org/synapse#10272))
@richvdh richvdh mentioned this pull request Jul 2, 2021
Closed
erikjohnston added a commit to matrix-org/complement that referenced this pull request Jul 5, 2021
@erikjohnston
Fix race in MSC3083 TestRestrictedRoomsRemoteJoin
4e021cd 
This was broken by matrix-org/synapse#10272
@erikjohnston erikjohnston mentioned this pull request Jul 5, 2021
Merged
erikjohnston added a commit to matrix-org/complement that referenced this pull request Jul 7, 2021
@erikjohnston
Fix race in MSC3083 TestRestrictedRoomsRemoteJoin (#149)
76db9e5 
This was broken by matrix-org/synapse#10272
@erikjohnston erikjohnston mentioned this pull request Jul 26, 2021
Closed
babolivier added a commit to matrix-org/synapse-dinsic that referenced this pull request Sep 1, 2021
@babolivier
Merge tag 'v1.37.1' into babolivier/dinsic_1.41.0
6c9461e 
Synapse 1.37.1 (2021-06-30)
===========================

This release resolves issues (such as [#9490](matrix-org/synapse#9490)) where one busy room could cause head-of-line blocking, starving Synapse from processing events in other rooms, and causing all federated traffic to fall behind. Synapse 1.37.1 processes inbound federation traffic asynchronously, ensuring that one busy room won't impact others. Please upgrade to Synapse 1.37.1 as soon as possible, in order to increase resilience to other traffic spikes.

No significant changes since v1.37.1rc1.

Synapse 1.37.1rc1 (2021-06-29)
==============================

Features
--------

- Handle inbound events from federation asynchronously. ([\#10269](matrix-org/synapse#10269), [\#10272](matrix-org/synapse#10272))
Fizzadar pushed a commit to Fizzadar/synapse that referenced this pull request Oct 26, 2021
@bradtgmurray
Merge tag 'v1.37.1'
e33008a 
Synapse 1.37.1 (2021-06-30)
===========================

This release resolves issues (such as [matrix-org#9490](matrix-org#9490)) where one busy room could cause head-of-line blocking, starving Synapse from processing events in other rooms, and causing all federated traffic to fall behind. Synapse 1.37.1 processes inbound federation traffic asynchronously, ensuring that one busy room won't impact others. Please upgrade to Synapse 1.37.1 as soon as possible, in order to increase resilience to other traffic spikes.

No significant changes since v1.37.1rc1.

Synapse 1.37.1rc1 (2021-06-29)
==============================

Features
--------

- Handle inbound events from federation asynchronously. ([\matrix-org#10269](matrix-org#10269), [\matrix-org#10272](matrix-org#10272))
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@clokep clokep clokep left review comments

@richvdh richvdh richvdh left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

_room_pdu_linearizer starves other incoming transactions
3 participants
@erikjohnston @clokep @richvdh