Add `get_userinfo_by_id` method to `ModuleApi` #9581

jaywink · 2021-03-10T10:35:26Z

Makes it easier to fetch user details in for example spam checker modules, without needing to use api._store or figure out database interactions.

Signed-off-by: Jason Robinson jasonr@matrix.org

Makes it easier to fetch user details in for example spam checker modules, without needing to use api._store or figure out database interactions. Signed-off-by: Jason Robinson <jasonr@matrix.org>

synapse/module_api/__init__.py

# Conflicts: # synapse/module_api/__init__.py

Returned from ModuleApi.get_user_by_id Signed-off-by: Jason Robinson <jasonr@matrix.org>

Signed-off-by: Jason Robinson <jasonr@matrix.org>

jaywink · 2021-07-21T19:23:43Z

Hmm, interesting, the thing the linting is complaining about was actually formatted by ./scripts-dev/lint.sh -d 🤔

Signed-off-by: Jason Robinson <jasonr@matrix.org>

richvdh · 2021-07-28T10:28:31Z

Hmm, interesting, the thing the linting is complaining about was actually formatted by ./scripts-dev/lint.sh -d

It looks like you might be using the wrong version of black? It should be 21.6b0, as per setup.py.

Meanwhile it looks like mypy is flagging some valid problems.

richvdh

thanks for your work on this @jaywink! I've made a few suggestions below.

tests/test_types.py

synapse/types.py

synapse/module_api/__init__.py

synapse/types.py

Apply suggestions to usage of attrs and rename some of the attributes to be consistent. Remove unnecessary test case. Signed-off-by: Jason Robinson <jasonr@matrix.org>

Move logic to the registration store for building a UserInfo object. Makes appservice an ID reference to avoid neeting to build an ApplicationService object in the registration store. Drop some defaults on various UserInfo attributes. Signed-off-by: Jason Robinson <jasonr@matrix.org>

Signed-off-by: Jason Robinson <jasonr@matrix.org>

The fixes seem a bit hacky just to fix the type checks, as the return types are `Any` from the selects. Signed-off-by: Jason Robinson <jasonr@matrix.org>

richvdh

thanks for your perseverance here @jaywink! A couple more tweaks suggested here.

richvdh · 2021-08-04T06:00:48Z

synapse/storage/databases/main/registration.py

            appservice_id=user_data.get("appservice_id"),
            consent_server_notice_sent=user_data.get("consent_server_notice_sent"),
            consent_version=user_data.get("consent_version"),
-            creation_ts=user_data.get("creation_ts"),
+            creation_ts=user_data.get("creation_ts", 0),
            is_admin=bool(user_data.get("admin")),
            is_deactivated=bool(user_data.get("deactivated")),
            is_guest=bool(user_data.get("is_guest")),
            is_shadow_banned=bool(user_data.get("shadow_banned")),
-            user_id=UserID.from_string(user_data.get("name")),
+            user_id=UserID.from_string(str(user_data.get("name"))),
            user_type=user_data.get("user_type"),


The fixes seem a bit hacky just to fix the type checks, as the return types are Any from the selects.

It's because you're using user_data.get. .get returns None if the dict doesn't contain that property. In this case, we know that the dict will contain each of these properties, so you should do user_data["appservice_id"] instead, which I think will also keep mypy happy.

richvdh · 2021-08-04T06:03:38Z

synapse/storage/databases/main/registration.py

@@ -166,6 +166,42 @@ async def get_user_by_id(self, user_id: str) -> Optional[Dict[str, Any]]:
            desc="get_user_by_id",
        )

+    @cached()


it'd be swell if get_userinfo_by_id and get_user_by_id could share a cache. I think the easiest way to do that is by having one call the other.

richvdh · 2021-08-04T06:03:54Z

synapse/storage/databases/main/registration.py

@@ -166,6 +166,42 @@ async def get_user_by_id(self, user_id: str) -> Optional[Dict[str, Any]]:
            desc="get_user_by_id",
        )

+    @cached()
+    async def get_userinfo_by_id(self, user_id: str) -> Optional[UserInfo]:


please can you give this a docstring?

Also, please can you stick a comment or docstring on get_user_by_id to say it's deprecated and replaced by get_userinfo_by_id?

Probably best deprecate _store.get_user_by_id in a follow-up pr, I'm happy to do that?

Oh I just read this. I'd roll it into this PR, since it's simple enough.

Unless you were talking about actually replacing uses of get_user_by_id with get_userinfo_by_id? Which would be amazing if you've got time, and yes a separate PR would be best.

synapse/types.py

Signed-off-by: Jason Robinson <jasonr@matrix.org>

To make mypy happier. Signed-off-by: Jason Robinson <jasonr@matrix.org>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

jaywink · 2021-08-04T08:44:51Z

Review addressed, thanks for walking through! 🙏

Signed-off-by: Jason Robinson <jasonr@matrix.org>

richvdh

lgtm otherwise

changelog.d/9581.feature

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

@Bubu

Synapse 1.41.0rc1 (2021-08-18) ============================== Features -------- - Add `get_userinfo_by_id` method to ModuleApi. ([\#9581](#9581)) - Initial local support for [MSC3266](#10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\#10394](#10394)) - Experimental support for [MSC3288](matrix-org/matrix-spec-proposals#3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\#10435](#10435)) - Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. ([\#10475](#10475)) - Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\#10498](#10498)) - Add a configuration setting for the time a `/sync` response is cached for. ([\#10513](#10513)) - The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\#10518](#10518)) - Add support for new redaction rules for historical events specified in [MSC2716](matrix-org/matrix-spec-proposals#2716). ([\#10538](#10538)) - Add a setting to disable TLS when sending email. ([\#10546](#10546)) - Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10549](#10549), [\#10560](#10560), [\#10569](#10569), [\#10574](#10574), [\#10575](#10575), [\#10579](#10579), [\#10583](#10583)) - Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\#10558](#10558), [\#10628](#10628)) - Add support for routing `/createRoom` to workers. ([\#10564](#10564)) - Update the Synapse Grafana dashboard. ([\#10570](#10570)) - Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\#10578](#10578)) - Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\#10598](#10598)) - The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\#10602](#10602)) - Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\#10638](#10638)) Bugfixes -------- - Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\#10129](#10129)) - Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\#10532](#10532)) - Fix exceptions in logs when failing to get remote room list. ([\#10541](#10541)) - Fix longstanding bug which caused the user "status" to be reset when the user went offline. Contributed by @dklimpel. ([\#10550](#10550)) - Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10580](#10580)) - Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\#10592](#10592)) - Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](matrix-org/matrix-spec-proposals#2285) enabled. Contributed by @SimonBrandner. ([\#10606](#10606)) - Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\#10611](#10611)) - Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\#10623](#10623)) Improved Documentation ---------------------- - Add documentation for configuration a forward proxy. ([\#10443](#10443)) - Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\#10551](#10551)) - Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\#10599](#10599)) Deprecations and Removals ------------------------- - No longer build `.deb` packages for Ubuntu 20.10 LTS Groovy Gorilla, which has now EOLed. ([\#10588](#10588)) - The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\#10596](#10596)) Internal Changes ---------------- - Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\#10119](#10119)) - Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\#10504](#10504)) - Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\#10507](#10507)) - Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10527](#10527), [\#10530](#10530)) - Fix CI to not break when run against branches rather than pull requests. ([\#10529](#10529)) - Mark all events stemming from the [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint as historical. ([\#10537](#10537)) - Clean up some of the federation event authentication code for clarity. ([\#10539](#10539), [\#10591](#10591)) - Convert `Transaction` and `Edu` objects to attrs. ([\#10542](#10542)) - Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\#10552](#10552)) - Update contributing.md to warn against rebasing an open PR. ([\#10563](#10563)) - Remove the unused public rooms replication stream. ([\#10565](#10565)) - Clarify error message when failing to join a restricted room. ([\#10572](#10572)) - Remove references to BuildKite in favour of GitHub Actions. ([\#10573](#10573)) - Move `/batch_send` endpoint defined by [MSC2716](matrix-org/matrix-spec-proposals#2716) to the `/v2_alpha` directory. ([\#10576](#10576)) - Allow multiple custom directories in `read_templates`. ([\#10587](#10587)) - Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\#10590](#10590)) - Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10600](#10600)) - Build Debian packages for Debian 12 (Bookworm). ([\#10612](#10612)) - Fix up a couple of links to the database schema documentation. ([\#10620](#10620)) - Fix a broken link to the upgrade notes. ([\#10631](#10631))

@Bubu

Synapse 1.41.0 (2021-08-24) =========================== This release adds support for Debian 12 (Bookworm), but **removes support for Ubuntu 20.10 (Groovy Gorilla)**, which reached End of Life last month. Note that when using workers the `/_synapse/admin/v1/users/{userId}/media` must now be handled by media workers. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. Features -------- - Enable room capabilities ([MSC3244](matrix-org/matrix-spec-proposals#3244)) by default and set room version 8 as the preferred room version when creating restricted rooms. ([\matrix-org#10571](matrix-org#10571)) Synapse 1.41.0rc1 (2021-08-18) ============================== Features -------- - Add `get_userinfo_by_id` method to ModuleApi. ([\matrix-org#9581](matrix-org#9581)) - Initial local support for [MSC3266](matrix-org#10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\matrix-org#10394](matrix-org#10394)) - Experimental support for [MSC3288](matrix-org/matrix-spec-proposals#3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\matrix-org#10435](matrix-org#10435)) - Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\matrix-org#10596](matrix-org#10596)). ([\matrix-org#10475](matrix-org#10475)) - Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\matrix-org#10498](matrix-org#10498)) - Add a configuration setting for the time a `/sync` response is cached for. ([\matrix-org#10513](matrix-org#10513)) - The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\matrix-org#10518](matrix-org#10518)) - Add support for new redaction rules for historical events specified in [MSC2716](matrix-org/matrix-spec-proposals#2716). ([\matrix-org#10538](matrix-org#10538)) - Add a setting to disable TLS when sending email. ([\matrix-org#10546](matrix-org#10546)) - Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10549](matrix-org#10549), [\matrix-org#10560](matrix-org#10560), [\matrix-org#10569](matrix-org#10569), [\matrix-org#10574](matrix-org#10574), [\matrix-org#10575](matrix-org#10575), [\matrix-org#10579](matrix-org#10579), [\matrix-org#10583](matrix-org#10583)) - Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\matrix-org#10558](matrix-org#10558), [\matrix-org#10628](matrix-org#10628)) - Add support for routing `/createRoom` to workers. ([\matrix-org#10564](matrix-org#10564)) - Update the Synapse Grafana dashboard. ([\matrix-org#10570](matrix-org#10570)) - Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\matrix-org#10578](matrix-org#10578)) - Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\matrix-org#10598](matrix-org#10598)) - The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\matrix-org#10602](matrix-org#10602)) - Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\matrix-org#10638](matrix-org#10638)) Bugfixes -------- - Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\matrix-org#10129](matrix-org#10129)) - Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\matrix-org#10532](matrix-org#10532)) - Fix exceptions in logs when failing to get remote room list. ([\matrix-org#10541](matrix-org#10541)) - Fix longstanding bug which caused the user's presence "status message" to be reset when the user went offline. Contributed by @dklimpel. ([\matrix-org#10550](matrix-org#10550)) - Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10580](matrix-org#10580)) - Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\matrix-org#10592](matrix-org#10592)) - Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](matrix-org/matrix-spec-proposals#2285) enabled. Contributed by @SimonBrandner. ([\matrix-org#10606](matrix-org#10606)) - Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\matrix-org#10611](matrix-org#10611)) - Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\matrix-org#10623](matrix-org#10623)) Improved Documentation ---------------------- - Add documentation for configuring a forward proxy. ([\matrix-org#10443](matrix-org#10443)) - Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\matrix-org#10551](matrix-org#10551)) - Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\matrix-org#10599](matrix-org#10599)) Deprecations and Removals ------------------------- - No longer build `.deb` packages for Ubuntu 20.10 Groovy Gorilla, which has now EOLed. ([\matrix-org#10588](matrix-org#10588)) - The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\matrix-org#10596](matrix-org#10596)) Internal Changes ---------------- - Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\matrix-org#10119](matrix-org#10119)) - Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\matrix-org#10504](matrix-org#10504)) - Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\matrix-org#10507](matrix-org#10507)) - Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10527](matrix-org#10527), [\matrix-org#10530](matrix-org#10530)) - Fix CI to not break when run against branches rather than pull requests. ([\matrix-org#10529](matrix-org#10529)) - Mark all events stemming from the [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint as historical. ([\matrix-org#10537](matrix-org#10537)) - Clean up some of the federation event authentication code for clarity. ([\matrix-org#10539](matrix-org#10539), [\matrix-org#10591](matrix-org#10591)) - Convert `Transaction` and `Edu` objects to attrs. ([\matrix-org#10542](matrix-org#10542)) - Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\matrix-org#10552](matrix-org#10552)) - Update contributing.md to warn against rebasing an open PR. ([\matrix-org#10563](matrix-org#10563)) - Remove the unused public rooms replication stream. ([\matrix-org#10565](matrix-org#10565)) - Clarify error message when failing to join a restricted room. ([\matrix-org#10572](matrix-org#10572)) - Remove references to BuildKite in favour of GitHub Actions. ([\matrix-org#10573](matrix-org#10573)) - Move `/batch_send` endpoint defined by [MSC2716](matrix-org/matrix-spec-proposals#2716) to the `/v2_alpha` directory. ([\matrix-org#10576](matrix-org#10576)) - Allow multiple custom directories in `read_templates`. ([\matrix-org#10587](matrix-org#10587)) - Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\matrix-org#10590](matrix-org#10590)) - Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\matrix-org#10600](matrix-org#10600)) - Build Debian packages for Debian 12 (Bookworm). ([\matrix-org#10612](matrix-org#10612)) - Fix up a couple of links to the database schema documentation. ([\matrix-org#10620](matrix-org#10620)) - Fix a broken link to the upgrade notes. ([\matrix-org#10631](matrix-org#10631))

@Bubu

Synapse 1.41.0 (2021-08-24) =========================== This release adds support for Debian 12 (Bookworm), but **removes support for Ubuntu 20.10 (Groovy Gorilla)**, which reached End of Life last month. Note that when using workers the `/_synapse/admin/v1/users/{userId}/media` must now be handled by media workers. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. Features -------- - Enable room capabilities ([MSC3244](matrix-org/matrix-spec-proposals#3244)) by default and set room version 8 as the preferred room version when creating restricted rooms. ([\#10571](matrix-org/synapse#10571)) Synapse 1.41.0rc1 (2021-08-18) ============================== Features -------- - Add `get_userinfo_by_id` method to ModuleApi. ([\#9581](matrix-org/synapse#9581)) - Initial local support for [MSC3266](matrix-org/synapse#10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\#10394](matrix-org/synapse#10394)) - Experimental support for [MSC3288](matrix-org/matrix-spec-proposals#3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\#10435](matrix-org/synapse#10435)) - Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\#10596](matrix-org/synapse#10596)). ([\#10475](matrix-org/synapse#10475)) - Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\#10498](matrix-org/synapse#10498)) - Add a configuration setting for the time a `/sync` response is cached for. ([\#10513](matrix-org/synapse#10513)) - The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\#10518](matrix-org/synapse#10518)) - Add support for new redaction rules for historical events specified in [MSC2716](matrix-org/matrix-spec-proposals#2716). ([\#10538](matrix-org/synapse#10538)) - Add a setting to disable TLS when sending email. ([\#10546](matrix-org/synapse#10546)) - Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10549](matrix-org/synapse#10549), [\#10560](matrix-org/synapse#10560), [\#10569](matrix-org/synapse#10569), [\#10574](matrix-org/synapse#10574), [\#10575](matrix-org/synapse#10575), [\#10579](matrix-org/synapse#10579), [\#10583](matrix-org/synapse#10583)) - Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\#10558](matrix-org/synapse#10558), [\#10628](matrix-org/synapse#10628)) - Add support for routing `/createRoom` to workers. ([\#10564](matrix-org/synapse#10564)) - Update the Synapse Grafana dashboard. ([\#10570](matrix-org/synapse#10570)) - Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\#10578](matrix-org/synapse#10578)) - Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\#10598](matrix-org/synapse#10598)) - The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\#10602](matrix-org/synapse#10602)) - Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\#10638](matrix-org/synapse#10638)) Bugfixes -------- - Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\#10129](matrix-org/synapse#10129)) - Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\#10532](matrix-org/synapse#10532)) - Fix exceptions in logs when failing to get remote room list. ([\#10541](matrix-org/synapse#10541)) - Fix longstanding bug which caused the user's presence "status message" to be reset when the user went offline. Contributed by @dklimpel. ([\#10550](matrix-org/synapse#10550)) - Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10580](matrix-org/synapse#10580)) - Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\#10592](matrix-org/synapse#10592)) - Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](matrix-org/matrix-spec-proposals#2285) enabled. Contributed by @SimonBrandner. ([\#10606](matrix-org/synapse#10606)) - Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\#10611](matrix-org/synapse#10611)) - Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\#10623](matrix-org/synapse#10623)) Improved Documentation ---------------------- - Add documentation for configuring a forward proxy. ([\#10443](matrix-org/synapse#10443)) - Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\#10551](matrix-org/synapse#10551)) - Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\#10599](matrix-org/synapse#10599)) Deprecations and Removals ------------------------- - No longer build `.deb` packages for Ubuntu 20.10 Groovy Gorilla, which has now EOLed. ([\#10588](matrix-org/synapse#10588)) - The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\#10596](matrix-org/synapse#10596)) Internal Changes ---------------- - Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\#10119](matrix-org/synapse#10119)) - Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\#10504](matrix-org/synapse#10504)) - Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\#10507](matrix-org/synapse#10507)) - Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10527](matrix-org/synapse#10527), [\#10530](matrix-org/synapse#10530)) - Fix CI to not break when run against branches rather than pull requests. ([\#10529](matrix-org/synapse#10529)) - Mark all events stemming from the [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint as historical. ([\#10537](matrix-org/synapse#10537)) - Clean up some of the federation event authentication code for clarity. ([\#10539](matrix-org/synapse#10539), [\#10591](matrix-org/synapse#10591)) - Convert `Transaction` and `Edu` objects to attrs. ([\#10542](matrix-org/synapse#10542)) - Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\#10552](matrix-org/synapse#10552)) - Update contributing.md to warn against rebasing an open PR. ([\#10563](matrix-org/synapse#10563)) - Remove the unused public rooms replication stream. ([\#10565](matrix-org/synapse#10565)) - Clarify error message when failing to join a restricted room. ([\#10572](matrix-org/synapse#10572)) - Remove references to BuildKite in favour of GitHub Actions. ([\#10573](matrix-org/synapse#10573)) - Move `/batch_send` endpoint defined by [MSC2716](matrix-org/matrix-spec-proposals#2716) to the `/v2_alpha` directory. ([\#10576](matrix-org/synapse#10576)) - Allow multiple custom directories in `read_templates`. ([\#10587](matrix-org/synapse#10587)) - Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\#10590](matrix-org/synapse#10590)) - Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10600](matrix-org/synapse#10600)) - Build Debian packages for Debian 12 (Bookworm). ([\#10612](matrix-org/synapse#10612)) - Fix up a couple of links to the database schema documentation. ([\#10620](matrix-org/synapse#10620)) - Fix a broken link to the upgrade notes. ([\#10631](matrix-org/synapse#10631))

@Bubu

Synapse 1.41.0 (2021-08-24) =========================== This release adds support for Debian 12 (Bookworm), but **removes support for Ubuntu 20.10 (Groovy Gorilla)**, which reached End of Life last month. Note that when using workers the `/_synapse/admin/v1/users/{userId}/media` must now be handled by media workers. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. Features -------- - Enable room capabilities ([MSC3244](matrix-org/matrix-spec-proposals#3244)) by default and set room version 8 as the preferred room version when creating restricted rooms. ([\matrix-org#10571](matrix-org#10571)) Synapse 1.41.0rc1 (2021-08-18) ============================== Features -------- - Add `get_userinfo_by_id` method to ModuleApi. ([\matrix-org#9581](matrix-org#9581)) - Initial local support for [MSC3266](matrix-org#10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\matrix-org#10394](matrix-org#10394)) - Experimental support for [MSC3288](matrix-org/matrix-spec-proposals#3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\matrix-org#10435](matrix-org#10435)) - Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\matrix-org#10596](matrix-org#10596)). ([\matrix-org#10475](matrix-org#10475)) - Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\matrix-org#10498](matrix-org#10498)) - Add a configuration setting for the time a `/sync` response is cached for. ([\matrix-org#10513](matrix-org#10513)) - The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\matrix-org#10518](matrix-org#10518)) - Add support for new redaction rules for historical events specified in [MSC2716](matrix-org/matrix-spec-proposals#2716). ([\matrix-org#10538](matrix-org#10538)) - Add a setting to disable TLS when sending email. ([\matrix-org#10546](matrix-org#10546)) - Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10549](matrix-org#10549), [\matrix-org#10560](matrix-org#10560), [\matrix-org#10569](matrix-org#10569), [\matrix-org#10574](matrix-org#10574), [\matrix-org#10575](matrix-org#10575), [\matrix-org#10579](matrix-org#10579), [\matrix-org#10583](matrix-org#10583)) - Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\matrix-org#10558](matrix-org#10558), [\matrix-org#10628](matrix-org#10628)) - Add support for routing `/createRoom` to workers. ([\matrix-org#10564](matrix-org#10564)) - Update the Synapse Grafana dashboard. ([\matrix-org#10570](matrix-org#10570)) - Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\matrix-org#10578](matrix-org#10578)) - Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\matrix-org#10598](matrix-org#10598)) - The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\matrix-org#10602](matrix-org#10602)) - Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\matrix-org#10638](matrix-org#10638)) Bugfixes -------- - Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\matrix-org#10129](matrix-org#10129)) - Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\matrix-org#10532](matrix-org#10532)) - Fix exceptions in logs when failing to get remote room list. ([\matrix-org#10541](matrix-org#10541)) - Fix longstanding bug which caused the user's presence "status message" to be reset when the user went offline. Contributed by @dklimpel. ([\matrix-org#10550](matrix-org#10550)) - Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10580](matrix-org#10580)) - Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\matrix-org#10592](matrix-org#10592)) - Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](matrix-org/matrix-spec-proposals#2285) enabled. Contributed by @SimonBrandner. ([\matrix-org#10606](matrix-org#10606)) - Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\matrix-org#10611](matrix-org#10611)) - Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\matrix-org#10623](matrix-org#10623)) Improved Documentation ---------------------- - Add documentation for configuring a forward proxy. ([\matrix-org#10443](matrix-org#10443)) - Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\matrix-org#10551](matrix-org#10551)) - Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\matrix-org#10599](matrix-org#10599)) Deprecations and Removals ------------------------- - No longer build `.deb` packages for Ubuntu 20.10 Groovy Gorilla, which has now EOLed. ([\matrix-org#10588](matrix-org#10588)) - The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\matrix-org#10596](matrix-org#10596)) Internal Changes ---------------- - Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\matrix-org#10119](matrix-org#10119)) - Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\matrix-org#10504](matrix-org#10504)) - Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\matrix-org#10507](matrix-org#10507)) - Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10527](matrix-org#10527), [\matrix-org#10530](matrix-org#10530)) - Fix CI to not break when run against branches rather than pull requests. ([\matrix-org#10529](matrix-org#10529)) - Mark all events stemming from the [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint as historical. ([\matrix-org#10537](matrix-org#10537)) - Clean up some of the federation event authentication code for clarity. ([\matrix-org#10539](matrix-org#10539), [\matrix-org#10591](matrix-org#10591)) - Convert `Transaction` and `Edu` objects to attrs. ([\matrix-org#10542](matrix-org#10542)) - Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\matrix-org#10552](matrix-org#10552)) - Update contributing.md to warn against rebasing an open PR. ([\matrix-org#10563](matrix-org#10563)) - Remove the unused public rooms replication stream. ([\matrix-org#10565](matrix-org#10565)) - Clarify error message when failing to join a restricted room. ([\matrix-org#10572](matrix-org#10572)) - Remove references to BuildKite in favour of GitHub Actions. ([\matrix-org#10573](matrix-org#10573)) - Move `/batch_send` endpoint defined by [MSC2716](matrix-org/matrix-spec-proposals#2716) to the `/v2_alpha` directory. ([\matrix-org#10576](matrix-org#10576)) - Allow multiple custom directories in `read_templates`. ([\matrix-org#10587](matrix-org#10587)) - Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\matrix-org#10590](matrix-org#10590)) - Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\matrix-org#10600](matrix-org#10600)) - Build Debian packages for Debian 12 (Bookworm). ([\matrix-org#10612](matrix-org#10612)) - Fix up a couple of links to the database schema documentation. ([\matrix-org#10620](matrix-org#10620)) - Fix a broken link to the upgrade notes. ([\matrix-org#10631](matrix-org#10631))

Add get_user_by_id method to ModuleApi

ace5488

Makes it easier to fetch user details in for example spam checker modules, without needing to use api._store or figure out database interactions. Signed-off-by: Jason Robinson <jasonr@matrix.org>

richvdh suggested changes Mar 10, 2021

View reviewed changes

synapse/module_api/__init__.py Outdated Show resolved Hide resolved

synapse/module_api/__init__.py Outdated Show resolved Hide resolved

richvdh requested review from richvdh and removed request for richvdh April 9, 2021 10:54

erikjohnston added the X-Awaiting-Changes A contributed PR which needs changes and re-review before it can be merged label May 20, 2021

jaywink added 3 commits July 21, 2021 16:48

Merge branch 'develop' into jaywink/get-user-from-module-api

3ea20a7

# Conflicts: # synapse/module_api/__init__.py

Introduce a User type

598bbe6

Returned from ModuleApi.get_user_by_id Signed-off-by: Jason Robinson <jasonr@matrix.org>

Actually commit tests for User type

449c354

Signed-off-by: Jason Robinson <jasonr@matrix.org>

Appease linting

692aef9

Signed-off-by: Jason Robinson <jasonr@matrix.org>

jaywink requested a review from richvdh July 21, 2021 19:25

erikjohnston removed the X-Awaiting-Changes A contributed PR which needs changes and re-review before it can be merged label Jul 23, 2021

richvdh suggested changes Jul 28, 2021

View reviewed changes

jaywink added 5 commits August 3, 2021 11:45

Merge branch 'develop' into jaywink/get-user-from-module-api

af95fed

User -> UserInfo

6d225a0

Apply suggestions to usage of attrs and rename some of the attributes to be consistent. Remove unnecessary test case. Signed-off-by: Jason Robinson <jasonr@matrix.org>

Linter autofix

6dc1240

Signed-off-by: Jason Robinson <jasonr@matrix.org>

Make mypy happy

c0a9082

The fixes seem a bit hacky just to fix the type checks, as the return types are `Any` from the selects. Signed-off-by: Jason Robinson <jasonr@matrix.org>

jaywink requested a review from richvdh August 3, 2021 15:53

richvdh suggested changes Aug 4, 2021

View reviewed changes

jaywink and others added 5 commits August 4, 2021 11:18

Make get_userinfo_by_id re-use cache of get_user_by_id

a990355

Signed-off-by: Jason Robinson <jasonr@matrix.org>

Mark registration store get_user_by_id as deprecated

e28a6e3

Signed-off-by: Jason Robinson <jasonr@matrix.org>

Don't use .get() when compiling UserInfo

b41aede

To make mypy happier. Signed-off-by: Jason Robinson <jasonr@matrix.org>

Merge branch 'develop' into jaywink/get-user-from-module-api

bf8bbad

Update UserInfo docstring

71744f0

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

jaywink requested a review from richvdh August 4, 2021 08:44

Add docstring for get_userinfo_by_id

52c1a0a

Signed-off-by: Jason Robinson <jasonr@matrix.org>

richvdh approved these changes Aug 4, 2021

View reviewed changes

changelog.d/9581.feature Outdated Show resolved Hide resolved

Update changelog.d/9581.feature

c68f4a3

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

richvdh changed the title ~~Add get_user_by_id method to ModuleApi~~ Add get_userinfo_by_id method to ModuleApi Aug 4, 2021

richvdh enabled auto-merge (squash) August 4, 2021 10:32

richvdh merged commit c2000ab into develop Aug 4, 2021

richvdh deleted the jaywink/get-user-from-module-api branch August 4, 2021 10:40

callahad mentioned this pull request Aug 24, 2021

We should generate and publish documentation for the Module API using Sphinx #10687

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `get_userinfo_by_id` method to `ModuleApi` #9581

Add `get_userinfo_by_id` method to `ModuleApi` #9581

jaywink commented Mar 10, 2021

jaywink commented Jul 21, 2021

richvdh commented Jul 28, 2021

richvdh left a comment

richvdh left a comment

richvdh Aug 4, 2021

richvdh Aug 4, 2021

richvdh Aug 4, 2021

richvdh Aug 4, 2021

jaywink commented Aug 4, 2021

richvdh left a comment

Add get_userinfo_by_id method to ModuleApi #9581

Add get_userinfo_by_id method to ModuleApi #9581

Conversation

jaywink commented Mar 10, 2021

jaywink commented Jul 21, 2021

richvdh commented Jul 28, 2021

richvdh left a comment

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

richvdh Aug 4, 2021

Choose a reason for hiding this comment

richvdh Aug 4, 2021

Choose a reason for hiding this comment

richvdh Aug 4, 2021

Choose a reason for hiding this comment

richvdh Aug 4, 2021

Choose a reason for hiding this comment

jaywink commented Aug 4, 2021

richvdh left a comment

Choose a reason for hiding this comment

Add `get_userinfo_by_id` method to `ModuleApi` #9581

Add `get_userinfo_by_id` method to `ModuleApi` #9581