Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSC2290: Separate Endpoints for Threepid Binding #2290

Merged
merged 30 commits into from
Oct 6, 2019
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
fdea3e3
wip
anoadragon453 Sep 12, 2019
7096092
init
anoadragon453 Sep 12, 2019
f5b10c6
cleanup
anoadragon453 Sep 12, 2019
5193c31
Cleaner API endpoints
anoadragon453 Sep 13, 2019
cb7c072
Two new endpoints instead of one
anoadragon453 Sep 16, 2019
5b259bf
Fix homeserver binding example
anoadragon453 Sep 16, 2019
1fc1e3c
run on sentence
anoadragon453 Sep 16, 2019
196f27e
Update proposals/2290-separate-threepid-bind-hs.md
anoadragon453 Sep 16, 2019
7b656e9
Update proposals/2290-separate-threepid-bind-hs.md
anoadragon453 Sep 16, 2019
f36ed9a
typos
anoadragon453 Sep 16, 2019
f06ba49
Assign meaning to bind and add
anoadragon453 Sep 16, 2019
4bc005a
Remove threepid explanation
anoadragon453 Sep 16, 2019
af24676
parameter, not endpoint
anoadragon453 Sep 16, 2019
2a55310
Clarify why MSC2229 was made obselete
anoadragon453 Sep 17, 2019
c57250b
Apply suggestions from code review
anoadragon453 Sep 17, 2019
0b67f34
Address review comments
anoadragon453 Sep 17, 2019
3eda9f7
Merge branch 'anoa/msc_separate_hs_api' of github.com:matrix-org/matr…
anoadragon453 Sep 17, 2019
1e69a7f
be assertive
anoadragon453 Sep 17, 2019
169174e
Suggest the use of a unstable flag
anoadragon453 Sep 17, 2019
53519f9
Pin a spec version when we link to it
anoadragon453 Sep 17, 2019
e50bb3d
Mention that homeserver's should remember binds done through them
anoadragon453 Sep 17, 2019
87d641c
Describe what the IS and HS are doing in the examples
anoadragon453 Sep 17, 2019
bd64ffc
Homeservers shouldn't proxy to user-provided identity servers anymore
anoadragon453 Sep 18, 2019
40420d9
Update proposals/2290-separate-threepid-bind-hs.md
anoadragon453 Sep 19, 2019
9311e89
Update proposals/2229-rebind-existing-3pid.md
anoadragon453 Sep 24, 2019
219ebff
typo fix
anoadragon453 Sep 26, 2019
1a51a24
UIAA on /account/3pid/add
anoadragon453 Sep 26, 2019
0332d53
Merge branch 'anoa/msc_separate_hs_api' of github.com:matrix-org/matr…
anoadragon453 Sep 26, 2019
ec7e795
reflow
anoadragon453 Sep 26, 2019
46e7137
Don't remove id_server and id_access_token
anoadragon453 Sep 26, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions proposals/2229-rebind-existing-3pid.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# Allowing 3PID Owners to Rebind

## Note: This MSC has been made obselete by MSC2290.
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved

MSC2290 provides two separate API endpoints, one for adding a 3PID to the
homeserver, and another for binding to an identity server. These new
endpoints will allow the homeserver to enforce rules on emails that already
exist on the homeserver, only when modifying homeserver email, while only
needing to forward requests when binding to an identity server. This removes
the problem MSC2229 is trying to solve, and it is thus made obselete.

---

```
3PID
noun
Expand Down
215 changes: 215 additions & 0 deletions proposals/2290-separate-threepid-bind-hs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
# Separate Endpoints for Binding Threepids
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved

*Note: This MSC obseletes
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved
[MSC2229](https://github.com/matrix-org/matrix-doc/pull/2229), which dealt
with changing the rules of the `bind` flag on [POST
/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid).
That endpoint is being deprecated as part of this MSC, thus MSC2229 is no
longer relevant.*

On the Client Server API there is currently a single endpoint for binding a
threepid (an email or a phone number): [POST
/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid).
Depending on whether the `bind` flag is `true` or `false`, the threepid will
be bound to either a user's account on the homeserver, or both the homeserver
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved
and an identity server. Note that we use the term `add` when talking about
adding a threepid to a homeserver, and `bind` when binding a threepid to an
identity server. This terminology will be used throughout the rest of this
proposal.

Typically, when using the `/account/3pid` endpoint, the identity server
handles the verification -- either by sending an email to an email address,
or a SMS message to a phone number. Once completed, the homeserver will check
with the identity server that verification had indeed happened, and if so,
the threepid would be either added to the homeserver, or added to the
homeserver and bound to the identity server simultaneously.

Now, consider the fact that the identity server used in this process is
provided by the user, using the endpoint's `id_server` parameter. If the user were
to supply a malicious identity server that would immediately answer "yes" to
any threepid validation, then the user could add any threepid to their
account on the homeserver (which is likely not something homeserver admins want).

To be clear, this is not a long-standing security issue. It is not a problem
in any released version of Synapse, as Synapse keeps a list of "trusted
identity servers" that acts a whitelist for what identity servers a user can
specify.

The concept of this whitelist is being removed in this MSC however, as part
of lessening the reliance of homeservers on identity servers. This cannot be
done while the homeserver is still trusting an identity server for validation
of threepids. If the endpoints are split, the homeserver will handle the
validation of threepids being added to user accounts, and identity servers
will validate threepids being bound to themselves.

## Proposal

To solve this problem, two new endpoints will be added to the Client Server
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relatedly: my main question is how hitting /bind on the HS differs from hitting the IS directly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HS will just proxy this request. However, it will remember that the bind occurred, and when it comes time to unbind all of a user's threepids (during account deactivation or when they explicitly request it), then the homeserver is aware of what binds occurred where and can carry out the unbinds on the user's behalf.

But other than that it is simply just proxying the request.

API: `POST /account/3pid/bind` and `POST /account/3pid/add`. Both will
require authentication and be rate-limited. The request parameters of `POST
/account/3pid/bind` are the same as [POST
/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid),
minus the `bind` flag, and the contents of `three_pid_creds` have been
brought to the top level of the request body. The request parameters of `POST
/account/3pid/add` will simply consist of a JSON body containing
`client_secret` and `sid`.

The homeserver should prevent a threepid being added to a user's account if
it already part of another user's account. However, the homeserver should not
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved
check for existing threepids when binding to an identity server. Identity
servers do not enforce this requirement and neither should the proxying
homeserver.

An example of binding a threepid to an identity server with this new endpoint
is as follows:

First the client must request the threepid be validated by its chosen identity server.

```
POST https://identity.server/_matrix/identity/v2/validate/email/requestToken

{
"client_secret": "don'tT3ll",
"email": "bob@example.com",
"send_attempt": 1
}
```

The identity server must send an email to the specified address, including a
link to a URL on the identity server which will accept the validation session
ID, the given client_secret, and a randomly-generated token.

Once an email has been sent, the user clicks the link in the email, which
notifies the identity server that the email has been verified.

Next, the client completes the bind by calling the new endpoint on the homeserver:
KitsuneRal marked this conversation as resolved.
Show resolved Hide resolved

```
POST https://home.server/_matrix/client/r0/account/3pid/bind
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved

{
"id_server": "example.org",
"id_access_token": "abc123_OpaqueString",
"sid": "abc123987",
"client_secret": "don'tT3ll"
}
```

The homeserver will then make a bind request to the specified identity server
on behalf of the user. The homeserver will record if the bind was successful
and notify the user. The homeserver will remember this bind and the identity
server it occurred on so that it can perform an unbind later if the user
requests it or their account is deactivated.

The threepid has now been bound on the user's requested identity server
without causing that threepid to be used for password resets or any other
homeserver-related functions.

For completeness, here is an example of adding a threepid to the homeserver
only, using the `/account/3pid/add` endpoint:

The homeserver is validating the threepid in this instance, so the client
must use the `/requestToken` endpoint of the homeserver:

```
POST https://home.server/_matrix/client/r0/account/3pid/email/requestToken

{
"client_secret": "don'tT3ll",
"email": "bob@example.com",
"send_attempt": 1,
}
```

Here the homeserver must send an email to the specified address, including a
link to a URL on the homeserver which will accept the validation session ID,
the given client_secret, and a randomly-generated token.

Once an email has been sent, the user clicks the link in the email, which
notifies the homeserver that the threepid has been verified.

The client then sends a request to the endpoint on the homeserver to add
the threepid to a user's account.

```
POST https://home.server/_matrix/client/r0/account/3pid/add

{
"sid": "abc123987",
"client_secret": "don'tT3ll"
}
```

The homeserver checks the threepid validation session referred to by the
given ID and client_secret was validated, and if so adds the threepid to the
user's account.

To achieve the above flows, some changes need to be made to existing
endpoints. The [POST
/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid)
endpoint is deprecated as the two new endpoints replace its functionality.
The `bind` parameter is to be removed, with the endpoint functioning as if
`bind` was `false`. Allowing an endpoint to add a threepid to both the
identity server and homeserver at the same time requires one to trust the
other, which is the exact behaviour we're trying to eliminate. Doing this
also helps backward compatibility, as explained in [Backwards
compatibility](#backwards-compatibility).

The `id_server` and `id_access_token` parameters are to be removed
Copy link
Member

@turt2live turt2live Sep 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this obsoletes MSC2263 too then? :/

They were made optional in 2263 for the reason of maintaining backwards compatibility with clients. Removing them does not maintain that backwards compatibility. In fact, 2263 even introduces an error code to let the HS make the decision about whether or not it cares to give power to the homeserver without harming the protocol.

This is duplicated effort.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can leave them as optional parameters but homeserver should ignore them else they are opening themselves up to malicious identity servers.

And then at that point there's not much point in keeping them as optional parameters versus removing them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By removing them, it becomes impossible for /bind to work. The client will have no way of contacting an identity server through the homeserver, so the homeserver might end up rejecting /bind. If there's no way to proxy the calls to the IS through the HS, there's no point in /bind and we should just remove the bind flag from /account/3pid instead of splitting the endpoint.

I'd rather we keep the fields as optional and put a red box saying "you should abuse M_SERVER_NOT_TRUSTED at every given opportunity" until the deprecation period has passed and they get removed (as they are deprecated by MSC2263).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, iiuc the problem is:

  • We don't want to support id_server in synapse because we want to ditch the concept of trusted ID servers
  • This isn't compatible with MSC2263 because that requires that the endpoints still work when given id_server

I think, as long as the requests still work in synapse with an id_server (modulo the fact they won't bind, but that's covered by this MSC), that's fine? The HS will send a token (not via the IS specified) but the right thing will still happen.

So I would suggest maybe keeping them optional for now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping them optional but returning M_SERVER_NOT_TRUSTED works I suppose.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2263 was specifically written to allow Synapse to not support id_server ftr: it can just throw M_SERVER_NOT_TRUSTED for everything.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't have homeservers throw M_SERVER_NOT_TRUSTED else old clients trying to send id_server will get confused when they get this error.

But new servers can just ignore the parameter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still expect that servers throw the error instead of ignoring it, as ignoring it is against the spec.

It'd be no different than a 500 error in the eyes of the client (which it should be handling because /requestToken is certainly in the top 10 for most error-prone endpoints).

Copy link
Contributor

@jryans jryans Sep 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a client perspective, we'd like old Riots to still be able add 3PIDs to the HS account.

If requestToken begins throwing an error when id_server is included, those clients will no longer have any path to add a 3PID. The old Riots would have to have known to special case and ignore M_SERVER_NOT_TRUSTED, which did not exist in the past.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jryans sounds like the server shouldn't be throwing the error then and instead handle the id_server as deprecated (as proposed by #2263). If the server doesn't want to handle it, it is no different from it throwing a 500 error for a misconfigured email server.

It would be intentional that adding 3PIDs be blocked - it's an error related to something the server is not comfortable with doing.

This all should have come out on MSC2263, not here, as well. I'd still have the same opinion, but it's off-topic for this MSC (now).

from all of the Client-Server API's `requestToken` endpoints. That is:

* [POST /account/3pid/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-email-requesttoken)
* [POST /account/3pid/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-msisdn-requesttoken)
* [POST /register/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-register-email-requesttoken)
* [POST /register/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-register-msisdn-requesttoken)
* [POST /account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-password-email-requesttoken)
* [POST /account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-password-msisdn-requesttoken)

Either the homeserver itself or a service that the homeserver delegates to
should be handling the sending of validation messages, not a user-provided
server. Any mention of the homeserver being able to proxy to an identity
server in the above endpoint descriptions, as well as the text "It is
imperative that the homeserver keep a list of trusted Identity Servers and
only proxies to those that it trusts." is to be removed from all parts of the
spec, as the homeserver should no longer need to trust any identity servers.
turt2live marked this conversation as resolved.
Show resolved Hide resolved

## Tradeoffs

One may question why clients don't just contact an identity server directly
to bind a threepid, bypassing the implications of binding through a
homeserver. While this will work, binds should still occur through a
homeserver such that the homeserver can keep track of which binds were made,
which is important when a user wishes to deactivate their account (and remove
all of their bindings made on different identity servers).

A verification could occur on an identity server, which could then tell the
homeserver that a validation happened, but then there are security
considerations about how to authenticate an identity server in that instance
(and prevent people pretending to be identity servers and telling homeservers
about hundreds of fake threepid additions to a user's account).

## Backwards compatibility
anoadragon453 marked this conversation as resolved.
Show resolved Hide resolved

A new flag will be added to `/versions`' unstable_features section,
`m.separate_add_and_bind`. If this flag is present and set to `true`, then
clients should use the new API endpoints to carry out threepid adds and
binds. If this flag is not present or set to `false`, clients should use
`/account/3pid`, being aware that they can only bind threepids to the
homeserver, not the identity server.

Old matrix clients will continue to use the `/account/3pid` endpoint. This
MSC removes the `bind` parameter and forces `/account/3pid` calls to act as
if `bind` was set to `false`. Old clients will still be able to add 3pids to
the homeserver, but not bind to the identity server. New homeservers must
ignore any `id_server` information passed to this endpoint.

## Security considerations

Reducing the homeserver's trust in identity servers should be a boost to
security and improve decentralisation in the Matrix ecosystem to boot.

Some endpoints of the Client Server API allow a user to provide an
`id_server` parameter. Caution should be taken for homeserver developers to
stop using these user-provided identity servers for any sensitive tasks where
possible, such as password reset or account registration, if it removes the
concept of a trusted identity server list.