Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Improve register performance #8009

Merged
merged 7 commits into from
Aug 6, 2020
Merged

Improve register performance #8009

merged 7 commits into from
Aug 6, 2020

Conversation

clokep
Copy link
Member

@clokep clokep commented Jul 31, 2020

This has a few tweaks to the register (and modify password) endpoints that should improve performance a bit:

  1. Pre-stores whether registration is enabled instead of checking the config each time.
  2. Only hash passwords when necessary:
    1. If UI auth fails (and might be completed in a subsequent call).
    2. Right before registration, if a password was provided.

It also has a couple of other changes:

  1. Re-organizes some code in the register endpoint and clarifies comments.
  2. Renames AuthHandler.check_auth to AuthHandler.check_ui_auth to avoid clashes with UserInteractiveAuthChecker.

This should be reviewable commit-by-commit.

Fixes #7956

@clokep clokep force-pushed the clokep/simplify-register branch from c83f58c to c1a2f5f Compare July 31, 2020 17:11
@clokep clokep requested a review from a team July 31, 2020 18:30
@richvdh richvdh self-assigned this Aug 3, 2020
Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm I think.

reminder that we should test this stuff carefully from a range of clients after deploying it, as it's fragile and the sort of thing that we can accidentally break and not notice until our usage stats drop off a few days later.

Comment on lines 499 to 507
# If a password hash was previously stored we will not attempt to
# re-hash and store it.
#
# Note that if the password changes throughout the authentication
# flow this might break, but the data is meant to be consistent
# throughout the flow.
password_hash = await self.auth_handler.get_session_data(
session_id, "password_hash", None
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I struggled for a while to understand what was going on here, and tbh the comment didn't do much to help.

I think this comment - especially the bit about what happens if the password changes mid-flow - should move down to inside the except InteractiveAuthIncompleteError block at line 533ish, and we can just say "extract the previously-hashed password from the session" here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I struggled with where to put this, moving it further up sounds like it would be an improvement.

Is there a particular piece of knowledge that should be documented in comments?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I struggled with where to put this, moving it further up sounds like it would be an improvement.

I'm mostly suggesting the code gets left where it is, and the comment moves down.

Is there a particular piece of knowledge that should be documented in comments?

I don't think so, tbh. I think this code speaks for itself, but I only realised that after I stopped trying to understand the comment, which doesn't really describe what is happening here :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-arranged and re-worded the comments in e05118b, I'm hoping that helps?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, lgtm.

@clokep
Copy link
Member Author

clokep commented Aug 3, 2020

reminder that we should test this stuff carefully from a range of clients after deploying it, as it's fragile and the sort of thing that we can accidentally break and not notice until our usage stats drop off a few days later.

I think I've previously written tests for some of the assumptions in various client applications, but definitely agree this needs careful testing!

@clokep
Copy link
Member Author

clokep commented Aug 5, 2020

I did some testing with this and Element iOS and the registration behavior seems to be the same as with develop, which is not ideal frankly, but unrelated to this PR.

I was using the following configuration:

  1. Enable email notifications (email block) and require email for registration (registrations_require_3pid)
  2. Patch the email so it just prints instead

I then went through the following steps:

  1. Register via the iOS client
  2. Copy the link
  3. Visit the link

This works and the registration completes OK.

The behavior that isn't great is that if the link is visited on my desktop (so both the iOS app and web app are "running" at the same time) then sometimes only one of the apps completes registration successfully and the other just spins. Other times both complete registration and you immediately get a prompt to verify the other session. I think the two apps essentially race since we delete the threepid validation information after it is validated. This doesn't seem to match the rest of the UI Auth information which we purposefully leave around as "valid" for a period of time.

Note that this behavior is not due to this PR.

@clokep
Copy link
Member Author

clokep commented Aug 5, 2020

And...I was testing the wrong branch. 🤦 Some of the behavior on this branch does seem a little different. On develop I seem to always end up at the case where both sessions end up valid (e.g. the race isn't triggered). I'm going to look a bit more into this.

@clokep
Copy link
Member Author

clokep commented Aug 5, 2020

After some investigation it seems that Element iOS is requesting to be sent back to Element Web (via a next link link https://app.element.io/#/register?client_secret=98BC6BD7-CBA4-43AE-83A1-D968C292F5BC-44237-0000FDA97FBAB43A&hs_url=http%3A%2F%2Flocalhost%3A8081&session_id=gTGEIrTURJBbxcATejjmHsJL&is_url=https%3A%2F%2Fvector.im&sid=xDzfSQZUKctJQsRZ on the requestToken API for validating email addresses). This seems wrong, but is not a Synapse issue.

@clokep clokep merged commit 66f2444 into develop Aug 6, 2020
@clokep clokep deleted the clokep/simplify-register branch August 6, 2020 12:09
reivilibre added a commit that referenced this pull request Aug 13, 2020
Synapse 1.19.0rc1 (2020-08-13)
==============================

Removal warning
---------------

As outlined in the [previous release](https://github.com/matrix-org/synapse/releases/tag/v1.18.0), we are no longer publishing Docker images with the `-py3` tag suffix. On top of that, we have also removed the `latest-py3` tag. Please see [the announcement in the upgrade notes for 1.18.0](https://github.com/matrix-org/synapse/blob/develop/UPGRADE.rst#upgrading-to-v1180).

Features
--------

- Add option to allow server admins to join rooms which fail complexity checks. Contributed by @lugino-emeritus. ([\#7902](#7902))
- Add an option to purge room or not with delete room admin endpoint (`POST /_synapse/admin/v1/rooms/<room_id>/delete`). Contributed by @dklimpel. ([\#7964](#7964))
- Add rate limiting to users joining rooms. ([\#8008](#8008))
- Add a `/health` endpoint to every configured HTTP listener that can be used as a health check endpoint by load balancers. ([\#8048](#8048))
- Allow login to be blocked based on the values of SAML attributes. ([\#8052](#8052))
- Allow guest access to the `GET /_matrix/client/r0/rooms/{room_id}/members` endpoint, according to MSC2689. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#7314](#7314))

Bugfixes
--------

- Fix a bug introduced in Synapse v1.7.2 which caused inaccurate membership counts in the room directory. ([\#7977](#7977))
- Fix a long standing bug: 'Duplicate key value violates unique constraint "event_relations_id"' when message retention is configured. ([\#7978](#7978))
- Fix "no create event in auth events" when trying to reject invitation after inviter leaves. Bug introduced in Synapse v1.10.0. ([\#7980](#7980))
- Fix various comments and minor discrepencies in server notices code. ([\#7996](#7996))
- Fix a long standing bug where HTTP HEAD requests resulted in a 400 error. ([\#7999](#7999))
- Fix a long-standing bug which caused two copies of some log lines to be written when synctl was used along with a MemoryHandler logger. ([\#8011](#8011), [\#8012](#8012))

Updates to the Docker image
---------------------------

- We no longer publish Docker images with the `-py3` tag suffix, as [announced in the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/UPGRADE.rst#upgrading-to-v1180). ([\#8056](#8056))

Improved Documentation
----------------------

- Document how to set up a client .well-known file and fix several pieces of outdated documentation. ([\#7899](#7899))
- Improve workers docs. ([\#7990](#7990), [\#8000](#8000))
- Fix typo in `docs/workers.md`. ([\#7992](#7992))
- Add documentation for how to undo a room shutdown. ([\#7998](#7998), [\#8010](#8010))

Internal Changes
----------------

- Reduce the amount of whitespace in JSON stored and sent in responses. Contributed by David Vo. ([\#7372](#7372))
- Switch to the JSON implementation from the standard library and bump the minimum version of the canonicaljson library to 1.2.0. ([\#7936](#7936), [\#7979](#7979))
- Convert various parts of the codebase to async/await. ([\#7947](#7947), [\#7948](#7948), [\#7949](#7949), [\#7951](#7951), [\#7963](#7963), [\#7973](#7973), [\#7975](#7975), [\#7976](#7976), [\#7981](#7981), [\#7987](#7987), [\#7989](#7989), [\#8003](#8003), [\#8014](#8014), [\#8016](#8016), [\#8027](#8027), [\#8031](#8031), [\#8032](#8032), [\#8035](#8035), [\#8042](#8042), [\#8044](#8044), [\#8045](#8045), [\#8061](#8061), [\#8062](#8062), [\#8063](#8063), [\#8066](#8066), [\#8069](#8069), [\#8070](#8070))
- Move some database-related log lines from the default logger to the database/transaction loggers. ([\#7952](#7952))
- Add a script to detect source code files using non-unix line terminators. ([\#7965](#7965), [\#7970](#7970))
- Log the SAML session ID during creation. ([\#7971](#7971))
- Implement new experimental push rules for some users. ([\#7997](#7997))
- Remove redundant and unreliable signature check for v1 Identity Service lookup responses. ([\#8001](#8001))
- Improve the performance of the register endpoint. ([\#8009](#8009))
- Reduce less useful output in the newsfragment CI step. Add a link to the changelog section of the contributing guide on error. ([\#8024](#8024))
- Rename storage layer objects to be more sensible. ([\#8033](#8033))
- Change the default log config to reduce disk I/O and storage for new servers. ([\#8040](#8040))
- Add an assertion on `prev_events` in `create_new_client_event`. ([\#8041](#8041))
- Add a comment to `ServerContextFactory` about the use of `SSLv23_METHOD`. ([\#8043](#8043))
- Log `OPTIONS` requests at `DEBUG` rather than `INFO` level to reduce amount logged at `INFO`. ([\#8049](#8049))
- Reduce amount of outbound request logging at `INFO` level. ([\#8050](#8050))
- It is no longer necessary to explicitly define `filters` in the logging configuration. (Continuing to do so is redundant but harmless.) ([\#8051](#8051))
- Add and improve type hints. ([\#8058](#8058), [\#8064](#8064), [\#8060](#8060), [\#8067](#8067))
babolivier pushed a commit that referenced this pull request Sep 1, 2021
* commit '66f24449d':
  Improve performance of the register endpoint (#8009)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

polling /register to check if email verification is complete is CPU intensive
2 participants