Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Allow PUT/GET of aliases during faster join #14292

Merged
merged 4 commits into from
Nov 1, 2022
Merged

Allow PUT/GET of aliases during faster join #14292

merged 4 commits into from
Nov 1, 2022

Conversation

DMRobertson
Copy link
Contributor

@DMRobertson DMRobertson commented Oct 25, 2022 •
edited
Loading

Rather than block waiting for full state to list all servers int he room, we use the servers we got in the /send_join response as an appxoimation.

TODO:

  • fixup association tables when resync completes.
  • do we care about deleting aliases during a resync too?
    • Untested. It might work, but we do look up the deleter's power levels in the current state, and the room's canonical aliases too.

Resolves #13287.
Complement: matrix-org/complement#537

Allow PUT/GET of aliases during faster join
3d151cf 
Rather than block waiting for full state to list all servers int he
room, we use the servers we got in the /send_join response as an
appxoimation.
@DMRobertson DMRobertson mentioned this pull request Oct 25, 2022
Merged
@DMRobertson DMRobertson marked this pull request as ready for review October 25, 2022 17:13
@DMRobertson DMRobertson requested a review from a team as a code owner October 25, 2022 17:13
squahtx
squahtx reviewed Nov 1, 2022
View reviewed changes
synapse/handlers/directory.py Outdated
Comment on lines 97 to 111
async def _get_servers_known_to_be_in_room(self, room_id: str) -> Set[str]:
if await self.store.is_partial_state_room(room_id):
# We don't have the full list of hosts in the room yet, but we do have
# some of them from the `/send_join` response. Use those to best-effort
# approximate the full list of servers in the room.
servers = set(await self.store.get_partial_state_servers_at_join(room_id))

# We are also in the room.
servers.add(self.server_name)
else:
servers = await self._storage_controllers.state.get_current_hosts_in_room(
room_id
)
return servers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have an existing method, self._storage_controllers.state.get_current_hosts_in_room_or_partial_state_approximation that is more or less equivalent. Is there a reason we're avoiding it (apart from the awful name, which is my fault)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only that I didn't know it existed. Will update, thanks.

@DMRobertson DMRobertson self-assigned this Nov 1, 2022
@squahtx
Copy link
Contributor

squahtx commented Nov 1, 2022

fixup association tables when resync completes.
Actually I don't think this is needed, see reasoning in #13287 (comment)

I suppose it depends on whether we think it's okay to trust the server we joined off. Since we already rely on the remote server to provide accurate state, it can be argued that it's fine to rely on a good-faith servers_in_room too. Though I wish we didn't have to take the remote server's word on anything at all.

do we care about deleting aliases during a resync too?
Untested. It might work, but we do look up the deleter's power levels in the current state, and the room's canonical aliases too.

Is there any reason this wouldn't just work?

@DMRobertson
Copy link
Contributor Author

do we care about deleting aliases during a resync too?
Untested. It might work, but we do look up the deleter's power levels in the current state, and the room's canonical aliases too.

Is there any reason this wouldn't just work?

Haven't thought it through, tbh. (I was wondering if it would block waiting for the full state.) I can try to write a quick complement test when I get the chance.

squahtx
squahtx approved these changes Nov 1, 2022
View reviewed changes
Copy link
Contributor

@squahtx squahtx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@DMRobertson DMRobertson merged commit 2bd7f3e into develop Nov 1, 2022
@DMRobertson DMRobertson deleted the dmr/faster-joins/aliases branch November 1, 2022 15:02
@DMRobertson
Copy link
Contributor Author

I can try to write a quick complement test when I get the chance.

Added to matrix-org/complement#537.

bradtgmurray added a commit to beeper/synapse-legacy-fork that referenced this pull request Nov 22, 2022
@bradtgmurray
Merge tag 'v1.72.0' into merge-v1.72
844c458 
Synapse 1.72.0 (2022-11-22)
===========================

Please note that Synapse now only supports PostgreSQL 11+, because PostgreSQL 10 has reached end-of-life, c.f. our [Deprecation Policy](https://github.com/matrix-org/synapse/blob/develop/docs/deprecation_policy.md).

Bugfixes
--------

- Update forgotten references to legacy metrics in the included Grafana dashboard. ([\matrix-org#14477](matrix-org#14477))

Synapse 1.72.0rc1 (2022-11-16)
==============================

Features
--------

- Add experimental support for [MSC3912](matrix-org/matrix-spec-proposals#3912): Relation-based redactions. ([\matrix-org#14260](matrix-org#14260))
- Build Debian packages for Ubuntu 22.10 (Kinetic Kudu). ([\matrix-org#14396](matrix-org#14396))
- Add an [Admin API](https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/index.html) endpoint for user lookup based on third-party ID (3PID). Contributed by @ashfame. ([\matrix-org#14405](matrix-org#14405))
- Faster joins: include heroes' membership events in the partial join response, for rooms without a name or canonical alias. ([\matrix-org#14442](matrix-org#14442))

Bugfixes
--------

- Faster joins: do not block creation of or queries for room aliases during the resync. ([\matrix-org#14292](matrix-org#14292))
- Fix a bug introduced in Synapse 1.64.0rc1 which could cause log spam when fetching events from other homeservers. ([\matrix-org#14347](matrix-org#14347))
- Fix a bug introduced in 1.66 which would not send certain pushrules to clients. Contributed by Nico. ([\matrix-org#14356](matrix-org#14356))
- Fix a bug introduced in v1.71.0rc1 where the power level event was incorrectly created during initial room creation. ([\matrix-org#14361](matrix-org#14361))
- Fix the refresh token endpoint to be under /r0 and /v3 instead of /v1. Contributed by Tulir @ Beeper. ([\matrix-org#14364](matrix-org#14364))
- Fix a long-standing bug where Synapse would raise an error when encountering an unrecognised field in a `/sync` filter, instead of ignoring it for forward compatibility. ([\matrix-org#14369](matrix-org#14369))
- Fix a background database update, introduced in Synapse 1.64.0, which could cause poor database performance. ([\matrix-org#14374](matrix-org#14374))
- Fix PostgreSQL sometimes using table scans for queries against the `event_search` table, taking a long time and a large amount of IO. ([\matrix-org#14409](matrix-org#14409))
- Fix rendering of some HTML templates (including emails). Introduced in v1.71.0. ([\matrix-org#14448](matrix-org#14448))
- Fix a bug introduced in Synapse 1.70.0 where the background updates to add non-thread unique indexes on receipts could fail when upgrading from 1.67.0 or earlier. ([\matrix-org#14453](matrix-org#14453))

Updates to the Docker image
---------------------------

- Add all Stream Writer worker types to `configure_workers_and_start.py`. ([\matrix-org#14197](matrix-org#14197))
- Remove references to legacy worker types in the multi-worker Dockerfile. ([\matrix-org#14294](matrix-org#14294))

Improved Documentation
----------------------

- Upload documentation PRs to Netlify. ([\matrix-org#12947](matrix-org#12947), [\matrix-org#14370](matrix-org#14370))
- Add addtional TURN server configuration example based on [eturnal](https://github.com/processone/eturnal) and adjust general TURN server doc structure. ([\matrix-org#14293](matrix-org#14293))
- Add example on how to load balance /sync requests. Contributed by [aceArt](https://aceart.de). ([\matrix-org#14297](matrix-org#14297))
- Edit sample Nginx reverse proxy configuration to use HTTP/1.1. Contributed by Brad Jones. ([\matrix-org#14414](matrix-org#14414))

Deprecations and Removals
-------------------------

- Remove support for PostgreSQL 10. ([\matrix-org#14392](matrix-org#14392), [\matrix-org#14397](matrix-org#14397))

Internal Changes
----------------

- Run unit tests against Python 3.11. ([\matrix-org#13812](matrix-org#13812))
- Add TLS support for generic worker endpoints. ([\matrix-org#14128](matrix-org#14128), [\matrix-org#14455](matrix-org#14455))
- Switch to a maintained action for installing Rust in CI. ([\matrix-org#14313](matrix-org#14313))
- Add override ability to `complement.sh` command line script to request certain types of workers. ([\matrix-org#14324](matrix-org#14324))
- Enabling testing of [MSC3874](matrix-org/matrix-spec-proposals#3874) (filtering of `/messages` by relation type) in complement. ([\matrix-org#14339](matrix-org#14339))
- Concisely log a failure to resolve state due to missing `prev_events`. ([\matrix-org#14346](matrix-org#14346))
- Use a maintained Github action to install Rust. ([\matrix-org#14351](matrix-org#14351))
- Cleanup old worker datastore classes. Contributed by Nick @ Beeper (@Fizzadar). ([\matrix-org#14375](matrix-org#14375))
- Test against PostgreSQL 15 in CI. ([\matrix-org#14394](matrix-org#14394))
- Remove unreachable code. ([\matrix-org#14410](matrix-org#14410))
- Clean-up event persistence code. ([\matrix-org#14411](matrix-org#14411))
- Update docstring to clarify that `get_partial_state_events_batch` does not just give you completely arbitrary partial-state events. ([\matrix-org#14417](matrix-org#14417))
- Fix mypy errors introduced by bumping the locked version of `attrs` and `gitpython`. ([\matrix-org#14433](matrix-org#14433))
- Make Dependabot only bump Rust deps in the lock file. ([\matrix-org#14434](matrix-org#14434))
- Fix an incorrect stub return type for `PushRuleEvaluator.run`. ([\matrix-org#14451](matrix-org#14451))
- Improve performance of `/context` in large rooms. ([\matrix-org#14461](matrix-org#14461))
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@squahtx squahtx squahtx approved these changes

Assignees

@DMRobertson DMRobertson

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Faster joins: room alias server list may be incomplete
2 participants
@DMRobertson @squahtx