Option to generate URL previews at the receiving client, not the server

[carlos22]

If the URL previews are enabled the riot-web app makes a request to
https://matrix.org/_matrix/media/r0/preview_url (assuming that matrix.org is your home-server). This would not be necessary if the app runs inside of electron. The sesstion.webRequest API could be used instead.
Benefits: For E2E rooms the user could have the URL preview without 'leaking' content of the chat to the home-server. The server doesn't need to proxy that much additional requests (can handle more users - theoretically)

See: electron/electron#3640

Related to element-hq/element-web#2084

[carlos22]

Could we use the Platform abstraction to properly handle this?

[ara4n]

the reason that the server generates previews atm is to stop remote sites tracking IPs of clients. This is particularly relevant for E2E users, I suspect. We don't want to be in a situation where you have 1000 people sitting in an E2E room, and someone turns up and posts a url of https://evil.com or whatever, grabs a list of the IP addresses of all the connected users, and then attacks them. There is currently no way for users to discover other users' IPs (and admins can only see the IPs connecting to their local HS).

[carlos22]

Ok that makes sense. For E2E direct-chats things are different, but matrix/riot does not distinguish between them. So it is all about how much you trust your homeserver...

[ara4n]

This very much feels like it should be a configurable option in the end. "Do you want to preview URLs via your server or via your client? Do you want to preview URLs in E2E chats?"

[carlos22]

@ara4n agreed. As much as I want to keep settings simple, but in this case i think people should decide.

[tessgadwa]

From a UX perspective, I also agree there should be a default but have it configurable.

[ara4n]

Alternatively, we could make it the sending client's problem (as per #1139). It's an interesting question as to whether the preview should be telling the user what they will see if they click on the page (whilst leaking their IP, and hammering the server)... or telling the user what the sender intended the preview to be (as per #1139).

Unsure how to proceed here, really.

[carlos22]

Thats not a bad idea to offload the problem to the sender. Because most likely the sender opened the URL anyway? Some Apps in Andorid 6 do something similar, they take a screenshot and add it to the "share" processs.

[ara4n]

the only reason not to do it is trust, with spammers going and misrepresenting the target's contents in the thumbnail, as @CoffeeMatrix implies. this could be solved at a social layer (eg with reputation rules), but it's a bit of a shame to make for ourselves when the receiving client could just gen the thumbnail themselves (at the expense of revealing their IP and hammering the URL). luckily the code is pretty much the same whether it's receiver or sender genning the preview though.

[carlos22]

But whats the point? The spammers also cloud send URLs/Images with offensive content?
I think the protection of your IP and all the parameters that it sends to a server are more precocious than protection of images that are not correct. And if we are still talking about E2E Chats, than a spammer should not be verified and therefore no thumbnails should be showed?!

[lheckemann]

Generating the preview client-side also won't be possible in the browser because of XHR cross-origin restrictions, as far as I know.

[akontsevich]

Also vote to preview on the client!!!

[kevincox]

I would also like to see previews on the client. For example I am often sharing "secret links" in E2EE chats. I am not too worried about leaking my IP (at least in some rooms) but I would rather not share that URL with the server.

I think the best solution is to make the sender generate a preview. The receiver can then choose to display it, generate its own preview locally (not yet implemented) or ask the homeserver to generate the preview. I would consider sending the preview as a reference to the original message so that it could be added by any user if the sender neglects to do so.

Random thoughts:

1. A user sending "spoofed" previews isn't really an issue, they can send me whatever rich content they want anyways. However users should be aware that the preview isn't "trusted".
2. Allowing IP leaks by default is a terrible idea. But there may be situations where it makes sense to allow a user to select receiver-client previews, generally on a per-room basis.
3. Some domains are trusted. For example maybe I don't mind google.com or twitter.com seeing my IP because I trust them to hide it from any attacker.
4. Sender-sent previews are also nice because they provide a snapshot at the time of sending.
5. CORS makes it difficult to generate previews for web-based clients. Most websites don't allow this unfortunately. However a best-effort option could still be nice.
6. It would be possible to have different clients "attach" previews to messages. So maybe my desktop client can't generate a preview but someone else can and will attach it. This could be useful for web-based or minimal clients. This could even be done by a bot if desired (maybe a public room wants to do this so that URLs are reliably previewed, even if some users are linking non-CORS sites from a web client)

[teohhanhui]

Generating on the sender side is the right thing to do.

Keybase does the same: https://book.keybase.io/docs/chat/link-previews

[kevincox]

That link seems to show that keybase does it client side.

[teohhanhui]

That link seems to show that keybase does it client side.

Yes, server-side is simply not an option when it's end-to-end encrypted.

I'm actually backing up what you've said above:

I think the best solution is to make the sender generate a preview.

[kevincox]

Oh sorry, I misread "sender-side" as "server-side".

[HarHarLinks]

A common talking point and strong counter-argument to client-side previews appears to be

People don't like leaking their IP address

Personally, I think this use case described by kevincox applies more often:

For example I am often sharing "secret links" in E2EE chats. I am not too worried about leaking my IP (at least in some rooms) but I would rather not share that URL with the server.

This applies only limitedly (?? can you say this in english?) when you both are hosted on a trustworthy homeserver, e.g. all conversation parties are self-hosting. For the time being this is the rarer scenario, however you could argue that privacy concerned people would tend to do this anyway if they are able to.

In my opinion the strongest argument is that generating server-side takes any control out of your hands. Even if it is your own server, it will get associated with opening the link. When you are concerned about your IP leaking, it is reasonable to assume that you have taken steps for your whole PC, as you likely use more services than just element. For example you are likely using a VPN you trust. It seems reasonable that you are going to click a link you received eventually. In this case it would be better and safer to generate the preview from your own client.

I think element-hq/element-android#3038 makes an interesting point: "[room] Administrators get to decide if the users of a room can individually enable/disable URL-Previews for their client. (defaulting to: disabled each)". In the context of this issue this could mean that room admins may disable server-side preview generation for all room participants.

[t3chguy]

Client side generation in the web can be problematic due to CORS preventing access to other domains unless they have ACAO: *

[Doomsdayrs]

Signal generates previews client side, and there seems to be no issues with that

[t3chguy]

@Doomsdayrs Signal doesn't look to be a webapp?

[Doomsdayrs]

@Doomsdayrs Signal doesn't look to be a webapp?

It is not, but the android app shows that it is possible to get such a system done. Load the embed on user side and the user sends it out

[t3chguy]

Only Web apps are bound by ACAO headers, they're enforced by web browsers, not by native applications.

[K4LCIFER]

I would also much prefer that URL thumbnails be generated locally instead of on the server. To me, it makes little practical sense to have it generated on the server, not to mention the security implications of it being generated on the server. I presume that if it is being generated on the server, then the server knows that that specific user is requesting a thumbnail for that specific link contained within a specific room; however, if the URLs are generated client side, its just a normal websearch from the user that the server knows nothing about. I don't think that there should be any option to toggle it server-side. I think that all URL thumbnails should either be generated locally, or not at all (by default, I think URL thumbnails should be off).

[maltfield]

+1 for client-side (either sender- or receiver-generated)

I'll just add that many of the more privacy-concerned users are likely already forcing all of their traffic through a shared proxy server (eg VPN or Tor) anyway. So "leaking" their IP isn't really a concern for many users.

I do think there should be some client-side cache, though. Once the URL is fetched once by the user, they shouldn't be sending the query off to regenerate the preview the next time they reboot their device and launch element again (I guess this concern would be elegantly solved by generating the preview sender-side).

[HarHarLinks]

client side generation would be a way to generate previews for URLs that can only really be accessed after having logged in (by reusing the browser session, if it is a web client). a use case would be github issues on secret repos, though today we have bots to work around that.

[kevincox]

by reusing the browser session, if it is a web client

This won't work for the general case because if a site can generate a preview for another site's private data that is a cross-site information leak. Browsers also are starting to block cross-site cookies by default, so even sites with this vulnerability will not work in the near future.

If you did want previews for non-public information the options I can see are:

1. Use a privileged client to pull your authentication info out of your browser (or have the user configure cookies/auth manually)
2. Have a bot with access attach previews to messages.
3. Teach your client to generate this preview with a "special" method such as using a configured GitHub token to fetch preview data from the API.
4. Embed some sort of token in the URL that allows previews.

None of these are general solutions. 1 requires a privileged client or manual configuration, 2 requires a bot with manual configuration, 3 requires manual auth and per-site code and 4 requires website support and user-action to get this custom link.

For private information I think that 2 (bot) will be the easiest option forward. This is basically how it works in Slack. You sign into the "Slack App" and then it can generate and attach previews to messages using your authentication information. However this does mean that we will want the option for other users to attach previews to other users' messages.

Having different users attach previews to your messages is definitely flexible but also has lots of abuse potential. So likely there would need to be permission controls and probably UX.

1. Permission level to control who can attach previews to messages. (Probably with a special case for attaching previews to their own messages).
2. Some sort of client UX to indicate who generated which preview.

I suspect that we want 2 no matter what. But we will also need 1 if people can attach previews to other user's messages. IDK if this should be room permissions, or configured by the client. I can see use cases for both. I would lean towards client-side because then each user is control of which previews they see. However for large rooms it may be best to add at least some default rules to avoid confusing preview spam.

[Mikaela]

Generating previews on client would be scary considering how much Element's user-agent tells the Matrix server already #2309.

I'll just add that many of the more privacy-concerned users are likely already forcing all of their traffic through a shared proxy server (eg VPN or Tor) anyway.

Tor is seldom an option judging by #200 and Matrix isn't that great for privacy.

[drzewiec]

Just to throw my $.02 in... generating previews on the client side may be necessary to get them to work at all in some cases. I've been playing around with running a homeserver in Digital Ocean (my usual provider of choice), and half the URLs I try to test preview with return nothing at all, because the host is blacklisting DO's IP.

I get why they do it, of course. But I also don't really know what recourse I have - from experience with AWS their IPs get blocked too, and I would expect most any hosting provider is going to be the same. So without the ability to generate URL previews from the client end, I'm kind of stuck in terms of being able to take advantage of this feature.

[maltfield]

@drzewiec interesting point, but the opposite could be true too: where users that must use commercial VPNs (because of the risks associated with the country that they live-in) and their shared VPN IP address might be blocked (or proxied by some front like CloudFlare or Akamai that present a CAPTCHA challenge that would break previews).

But this is a good argument to do it client-side twice: [1] first try it at the sender's side and [2] (if it arrives without a preview) later try at the receiver's side.