Dynamic per-request / per-session / per-site-instance proxy configuration

[riastradh-brave]

In order to isolate circuits for different first-party origins, we need to dynamically choose the proxy configuration (using SOCKS5 authentication as in #462) for each request as a function of the first-party origin, the session, and some state.

The state is necessary so that we can implement a 'New Tor circuit for this site' button that, in addition to flushing cookies and other local state for the site, causes all subsequent requests to use a different Tor circuit.

For example, if I'm browsing Wikipedia about Marxist revolutionary movements in Bulgarian, Brave should use the proxy socks5://bg.wikipedia.org:abcd0123@127.0.0.1:9050 for all those requests. If, at the same time, I'm browsing the London School of Economics grad school application site, we should use the proxy socks5://lse.ac.uk:dead4567@127.0.0.1:9050 so that even if there's an image on that site hosted by Wikimedia Commons also used by one of the Bulgarian Marxist Wikipedia pages, the requests for these will be uncorrelated and nobody will connect my ulterior motives in grad school.

Then if I want to read about explosives on Bulgarian Wikipedia, but don't want to reveal it's connected to the Marxist revolutionary movements, the 'New Tor circuit for this site' button should cause all subsequent requests to use socks5://bg.wikipedia.org:8901face@127.0.0.1:9050. Here the username is the first-party origin; the password is a string generated independently at random for each session, each site, and each time the user hits the 'New Tor circuit for this site' button.

(In the long term, the 'New Tor circuit for this site' may or may not be how we want to isolate browsing; see brave/browser-laptop#12922 to track these UX questions.)

This is tricky because the byzantine maze of proxy configuration abstractions in src/net/proxy is not really designed to keep state. The draft in my Tor branch just sets a static proxy configuration for now.

Here are a few approaches that we might try:

1. Write a proxy autoconfig script, either with data: URL or a local file: URL.

   • Problem: Proxy autoconfig scripts aren't passed the first-party origin -- only the request URL. So, unless we change the API of proxy autoconfig scripts, we would create many unnecessary circuits; there's not much anonymity benefit to be had from isolating the network traffic for all requests made from a single page load, and there's a substantial cost to setting up the circuits.
   • Problem: It's not totally clear what state proxy autoconfig scripts can reliably keep. I spent a few hours digging through the code in net/proxy to figure this out and I came out dazed and confused.

2. Write logic in browser-laptop to intercept all requests using session.webRequest.onBeforeRequest and replace the proxy configuration for that one request, with state in the main context. (This is extend session.webRequest.onBeforeRequest to support a proxy URL #463.)

   • Problem: We can't just do session.setProxy in the onBeforeRequest callback, because the requests do not happen in order in lock step.
   • Problem: We may need support in the C++ code in net/url_request to specify a proxy configuration for each request independently. Currently each session object has a single fixed proxy configuration.

3. Write a ProxyResolver in net/proxy in C++ for Tor sessions, possibly modelled after the per-OS system proxy resolvers.

   • Problem: The byzantine maze of proxy {resolver, config, service, server, resolver factory, ...} abstractions is difficult to navigate.
   • Problem: It may be difficult to make this keep state independently for each Brave session at a higher level.

4. Give each first-party origin its own session object (in the sense of electron.session). As a bonus, this obviates the need to implement third-party isolation in various things like cookie stores, local storage, etc., wherever we haven't done so already. Then we can just use a static proxy configuration in each session on its own with no changes to C++ code. (This would make it more of a browser-laptop issue than a muon issue, but I already made the issue here.)

   • Problem: We may have some difficulty with ordinary browser state management. For example, we will need to figure out how to make the back/forward buttons work when we change sessions because we changed first-party origin.

5. Use BrowserContext::CreateRequestContextForStoragePartition to create a proxy per storage partition / Site Instance, and set ShouldUseProcessPerSite to true for the Tor browser context in order to ensure that different origins do not share a Site Instance.

[diracdeltas]

discussed in-person and in slack. we are going to start with per-session proxies instead of per request.

[bridiver]

I'm not sure if this is really that complicated. You can call set_proxy_service on the urlrequestcontext before the request starts. You can't do that with session api, but it could be added inside onBeforeRequest

[riastradh-brave]

@bridiver Will calling set_proxy_service apply only to that one request? I was under the impression that a urlrequestcontext is shared by many requests.

[diracdeltas]

Added option 5 which is what @darkdh is currently working on

[riastradh-brave]

done in #473