Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bust _membership_stream_cache cache when current state changes #17732

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from
Draft

Bust _membership_stream_cache cache when current state changes #17732

wants to merge 2 commits into from

Conversation

MadLittleMods
Copy link
Collaborator

@MadLittleMods MadLittleMods commented Sep 18, 2024 •
edited
Loading

Bust _membership_stream_cache cache when current state changes. This is particularly a problem in a state reset scenario where the membership might change without a corresponding event.

This is a general Synapse thing so by it's nature it helps out Sliding Sync.

Fix #17368

Dev notes

self._membership_stream_cache.entity_has_changed(state_key, stream_ordering)  # type: ignore[attr-defined]

self._membership_stream_cache.all_entities_changed(token)  # type: ignore[attr-defined]

get_sliding_sync_rooms_for_user

process_replication_rows
_process_event_stream_row

_account_data_stream_cache
_device_inbox_stream_cache
_device_federation_outbox_stream_cache
_user_signature_stream_cache
presence_stream_cache

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file. The entry should:
    • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
    • Use markdown where necessary, mostly for code blocks.
    • End with either a period (.) or an exclamation mark (!).
    • Start with a capital letter.
    • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
  • Code style is correct
    (run the linters)

MadLittleMods
MadLittleMods commented Sep 18, 2024
View reviewed changes
synapse/util/caches/stream_change_cache.py
Comment on lines +317 to +321
def all_entities_changed(self, stream_pos: int) -> None:
"""
Mark all entities as changed. This is useful when the cache is invalidated and
there may be some potential change for all of the entities.
"""
Copy link
Collaborator Author

@MadLittleMods MadLittleMods Sep 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: all_entities_changed(stream_pos): Does this concept make sense?

I don't think it makes sense to drop all of the keys as we're essentially not sure if something has changed. I think this is the way and is just "unfortunate for the membership caches"

MadLittleMods
MadLittleMods commented Sep 18, 2024
View reviewed changes
synapse/storage/databases/main/cache.py
@@ -219,6 +219,8 @@ def process_replication_rows(
room_id = row.keys[0]
members_changed = set(row.keys[1:])
self._invalidate_state_caches(room_id, members_changed)
for user_id in members_changed:
self._membership_stream_cache.entity_has_changed(user_id, token) # type: ignore[attr-defined]
Copy link
Collaborator Author

@MadLittleMods MadLittleMods Sep 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kinda weird to just stick this here (same with the others in process_replication_rows). Better way to organize this?

@MadLittleMods MadLittleMods marked this pull request as ready for review September 18, 2024 23:12
@MadLittleMods MadLittleMods requested a review from a team as a code owner September 18, 2024 23:12
@MadLittleMods MadLittleMods mentioned this pull request Sep 18, 2024
Merged
5 tasks
@MadLittleMods MadLittleMods marked this pull request as draft September 18, 2024 23:29
MadLittleMods
MadLittleMods commented Sep 18, 2024
View reviewed changes
synapse/storage/databases/main/cache.py
@@ -219,6 +219,8 @@ def process_replication_rows(
room_id = row.keys[0]
members_changed = set(row.keys[1:])
self._invalidate_state_caches(room_id, members_changed)
for user_id in members_changed:
self._membership_stream_cache.entity_has_changed(user_id, token) # type: ignore[attr-defined]
Copy link
Collaborator Author

@MadLittleMods MadLittleMods Sep 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, tested this out with the tests in #17725 and these cache busting spots for current state don't pick up a state reset. Will need to look into this more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
A-Sync
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Bust _membership_stream_cache cache when current state changes
1 participant
@MadLittleMods