Implement support for NetworkService

[magreenblatt]

Original report by Mike Wiedenbauer (Bitbucket: shagkur, GitHub: shagkur).

---

Chromium is moving from an in-process network stack implementation to a separate NetworkService process.

Tracking issue and design docs at https://bugs.chromium.org/p/chromium/issues/detail?id=598073

Related embedder-dev@ thread: https://groups.google.com/a/chromium.org/d/msg/embedder-dev/DYE94KHLMx4/g_vhK2T4CAAJ

Chromium NetworkService status updates are being sent to the services-dev@ mailing list. Here's the most recent update: https://groups.google.com/a/chromium.org/d/msg/services-dev/bhiYEx0d10I/c-VgO4scBgAJ. Based on this update I'm guessing the old code path will be removed after M75 branches (mid-April).

[magreenblatt]

Issue #2497 is also related.

[magreenblatt]

@shagkur Can you share any progress on this issue? I'm planning to pick this up shortly as well, given the approaching old path removal deadline.

[magreenblatt]

Generalizing this issue to cover all NetworkService-related changes.

[magreenblatt]

• changed title from "Change CefURLRequest implementation to use the new NetworkService of chroimium" to "Implement support for NetworkService"

[magreenblatt]

Key methods for NetworkService initialization:

• OnNetworkServiceCreated called when the NetworkService, accessible through content::GetNetworkService(), is created.
• CreateNetworkContext creates a NetworkContext for a BrowserContext's StoragePartition. When the network service is enabled it must return a NetworkContext using the network service.
• GetNetworkContextsParentDirectory returns the parent paths that contain all the network service's BrowserContexts' storage.
• GetSystemNetworkContext returns the system network context.
• GetSystemSharedURLLoaderFactory returns a SharedURLLoaderFactory attached to the system network context.

Request handling/interception with NetworkService:

Custom schemes are handled using URLLoaderFactory (for example, ExtensionURLLoaderFactory). These objects are created via:

• CreateURLLoaderFactoryForNetworkRequests for network requests in specific cases (see ContentBrowserClient docs).
• RegisterNonNetworkSubresourceURLLoaderFactories for general resource requests.
• RegisterNonNetworkNavigationURLLoaderFactories for navigation requests.

Existing URLLoaderFactory interfaces can be intercepted using WillCreateURLLoaderFactory.

Arbitrary requests can be intercepted using URLLoaderRequestInterceptor (for example, OfflinePageURLLoaderRequestInterceptor). These objects are created via WillCreateURLLoaderRequestInterceptors.

[magreenblatt]

CefURLRequest can be implemented using a custom URLLoader (likely starting with a fork of SimpleURLLoader).

[magreenblatt]

Current status:

• NetworkService is now enabled by default.
• The old network implementation has been deleted in master and newer branches. M76 (3809) is the last branch to support --disable-features=NetworkService.
• Related known/open issues can be found here.

Summary of changes (see commit log for a full list of changes):

• Usage has changed as follows:

  • All cache_path values must be relative to a new CefSettings.root_cache_path value.
  • CefCookieManager callbacks are now executed on the UI thread (previously the IO thread).

• The following methods have been removed:

  • CefCookieManager::CreateManager
  • CefCookieManager::GetBlockingManager
  • CefCookieManager::SetStoragePath
  • CefRequestContextHandler::GetCookieManager

• The following methods have been added:

  • CefFrame::CreateURLRequest
  • GetResourceRequestHandler in CefRequestHandler and CefRequestContextHandler

• The following methods have been moved/renamed:

  • CefRequestContext::GetDefaultCookieManager to GetCookieManager.
  • CefRequestHandler::* to CefResourceRequestHandler::*

Other related enhancements (not blocking NetworkService by default):

• Move certificate-related callbacks to CefResourceRequestHandler (OnCertificateError, OnSelectClientCertificate).
• Move CefServer to network process?
• API redundancy: CefRequestCallback has same functionality as CefCallback (issue API redundancy: CefRequestCallback has same functionality as CefCallback #1861)

[magreenblatt]

@shagkur if you would like to work on the CefURLRequest support feel free to use my PR as a starting point -- we can merge all of the commits at some point in the future.

[magreenblatt]

Original comment by Mike Wiedenbauer (Bitbucket: shagkur, GitHub: shagkur).

---

@magreenblatt Sorry for not being much responsive these days, but i'm currently at an on-site visit for my company and can't do much on this topic. Next week looks better and i'll start on it on your branch (already had a quick look at it today)

[magreenblatt]

@shagkur sounds good, thanks! :)

[magreenblatt]

Cookies can be accessed/modified from the browser process using StoragePartition::GetCookieManagerForBrowserProcess, but that doesn't provide the same proxy capabilities that we currently offer via CefRequestContextHandler::GetCookieHandler. We might need to register some handlers in the network service/process directly.

Call stack for creation of the NetworkContext:

>	libcef.dll!ProfileNetworkContextService::CreateNetworkContextParams(bool in_memory, const base::FilePath & relative_partition_path) Line 371	C++
 	libcef.dll!ProfileNetworkContextService::CreateNetworkContext(bool in_memory, const base::FilePath & relative_partition_path) Line 154	C++
 	libcef.dll!Profile::CreateNetworkContext(bool in_memory, const base::FilePath & relative_partition_path) Line 258	C++
 	libcef.dll!CefContentBrowserClient::CreateNetworkContext(content::BrowserContext * context, bool in_memory, const base::FilePath & relative_partition_path) Line 1222	C++
 	content.dll!content::StoragePartitionImpl::InitNetworkContext() Line 1412	C++
 	content.dll!content::StoragePartitionImpl::GetNetworkContext() Line 803	C++
 	content.dll!content::URLLoaderFactoryGetter::HandleNetworkFactoryRequestOnUIThread(mojo::InterfaceRequest<network::mojom::URLLoaderFactory> network_factory_request, bool is_corb_enabled) Line 301	C++
 	content.dll!content::URLLoaderFactoryGetter::HandleFactoryRequests() Line 152	C++
 	content.dll!content::URLLoaderFactoryGetter::Initialize(content::StoragePartitionImpl * partition) Line 143	C++
 	content.dll!content::StoragePartitionImpl::Create(content::BrowserContext * context, bool in_memory, const base::FilePath & relative_partition_path, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & partition_domain) Line 726	C++
 	content.dll!content::StoragePartitionImplMap::Get(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & partition_domain, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & partition_name, bool in_memory, bool can_create) Line 397	C++
 	content.dll!content::`anonymous namespace'::GetStoragePartitionFromConfig(content::BrowserContext * browser_context, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & partition_domain, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & partition_name, bool in_memory, bool can_create) Line 218	C++
 	content.dll!content::BrowserContext::GetStoragePartition(content::BrowserContext * browser_context, content::SiteInstance * site_instance, bool can_create) Line 446	C++
 	content.dll!content::BrowserContext::GetDefaultStoragePartition(content::BrowserContext * browser_context) Line 478	C++
 	libcef.dll!CefBrowserContextImpl::Initialize() Line 309	C++
 	libcef.dll!CefRequestContextImpl::Initialize() Line 613	C++
 	libcef.dll!CefRequestContextImpl::CreateGlobalRequestContext(const CefStructBase<CefRequestContextSettingsTraits> & settings) Line 164	C++
 	libcef.dll!CefBrowserMainParts::PreMainMessageLoopRun() Line 199	C++

CefContentBrowserClient::CreateNetworkContext calls Profile::CreateNetworkContext which results in a call to ProfileNetworkContextService::CreateNetworkContextParams which sets CookieManagerParams values.

The NetworkContext is uniquely identified by these parameters:

    content::BrowserContext* context,
    bool in_memory,
    const base::FilePath& relative_partition_path

The |relative_partition_path| value, if non-empty, will be appended to the BrowserContext's cache path.

If we use BrowserContext::GetStoragePartition(BrowserContext* browser_context, SiteInstance* site_instance) instead of GetDefaultStoragePartition then we can further customize the configuration by implementing ContentBrowserClient::GetStoragePartitionConfigForSite. However, it might not make sense to use SiteInstance given the renderer process implications (see the docs on that class).

[magreenblatt]

It's probably time that we move to the intended Chromium model (see discussion here) and get rid of the special CefRequestContextHandler::GetCookieHandler handling (which currently requires all kinds of hacks in Chromium code to implement SiteInstance proxying). Just create your CefBrowser with a different CefRequestContext and cache_path if you want separate cookie storage. We can then re-implement CefCookieManager to use StoragePartition::GetCookieManagerForBrowserProcess for access to the underlying cookie storage.

[magreenblatt]

Add initial NetworkService support (see issue #2622).

This change adds an --enable-network-service command-line flag to run with
NetworkService enabled.

To test: Run cefclient --enable-network-service --disable-extensions. The
application should start, load a website and exit without crashing.
Network-related handlers (for cookies, schemes, requests, etc) and extensions
are not expected to work yet.

There should be no functional change when running without the NetworkService
enabled.

→ <<cset 6b2c1fe9693f (bb)>>

[magreenblatt]

Remove methods that modify cookie storage at runtime (see issue #2622).

This change removes cookie and request handler functionality that will not
supported by the NetworkService. Specifically, it is no longer possible to
change cookie storage locations at runime by returning a different
CefCookieManager for an already initialized CefRequestContext. After this change
you will need to use a separate CefRequestContext when creating a CefBrowser if
you require separate cookie storage.

The following methods have been removed:

• CefCookieManager::CreateManager
• CefCookieManager::GetBlockingManager
• CefCookieManager::SetStoragePath
• CefRequestContextHandler::GetCookieManager

The following methods have been renamed:

• CefRequestContext::GetDefaultCookieManager to GetCookieManager.

This change substantially simplifies the network implementation in CEF because
it is no longer necessary to proxy objects that are normally owned by Chromium.
Chromium patches that are no longer necessary will be removed as a follow-up
commit.

To test: Verify that ceftests --gtest_filter=-PluginTest.* pass with
NetworkService disabled. Plugin tests will be fixed in a follow-up commit.

→ <<cset a23e8452443d (bb)>>

[magreenblatt]

Move the frame/handler association to CefResourceContext (see issue #2622).

A CefBrowserHostImpl is created using a CefRequestContextImpl that may have a
CefRequestContextHandler. Multiple CefRequestContextImpl may share the same
underlying CefBrowserContext which owns a CefResourceContext. IO-thread
callbacks from Chromium are often associated with a CefResourceContext and the
target frame is identified using render_process_id/render_frame_id routing IDs.

This change forwards frame create/delete notifications from CefBrowserHostImpl
(or CefMimeHandlerViewGuestDelegate) to CefResourceContext so that it can
properly resolve the association from routing ID to Handler when queried from
CefPluginServiceFilter::IsPluginAvailable.

To test: Verify that all ceftests pass with NetworkService disabled.

→ <<cset ea27dff33839 (bb)>>

[magreenblatt]

Remove Chromium patches that are no longer required (see issue #2622).

→ <<cset 9b43d265c38d (bb)>>

[magreenblatt]

Enforce cache_path requirements for NetworkService (see issue #2622).

This change adds a new CefSettings.root_cache_path value that must be either
equal to or a parent directory of all CefSettings.cache_path and
CefRequestContextSettings.cache_path values. The sandbox may block read/write
access from the NetworkService to directories that do not meet this requirement.

To test: Run cefclient with a combination of the following flags:

--cache-path=c:\temp\cache
Cache data should be persisted to the specified directory.

--request-context-per-browser
A separate numbered cache directory should be created underneath the
cache-path directory for each new browser instance.

--enable-network-service --disable-extensions
Same tests, but with NetworkService enabled.

Known issues:

• When NetworkService is enabled a C:\temp\cache\cache\Cache directory is
  created (should be C:\temp\cache\Cache).

→ <<cset b65f336f8126 (bb)>>

[magreenblatt]

Original comment by Mike Wiedenbauer (Bitbucket: shagkur, GitHub: shagkur).

---

@magreenblatt Is it still valid to retrieve the CefRequestContext(Impl) and hence the CefBrowserContext via the "GetRequestContextOnUIThread" function in CefURLRequest? Because URLLoader and SimpleURLLoader need a URLLoaderFactory which, if i understand CefBrowserContext correctly, is provided by that class?

[magreenblatt]

@shagkur The logic in GetRequestContextOnUIThread for retrieving the CefRequestContextImpl and CefBrowserContext looks fine. You should then be able to retrieve the URLLoaderFactory via content::BrowserContext::GetDefaultStoragePartition(browser_context)->GetURLLoaderFactoryForBrowserProcess().

[magreenblatt]

Add CookieManagerImpl for NetworkService (see issue #2622).

To test: Run ceftests --gtest_filter=CookieTest.*:-CookieTest.GetCookieManager* --enable-network-service

There should be no functional change when running without the NetworkService
enabled.

Known issues:

• CefCookieManager::SetSupportedSchemes is not yet implemented.

→ <<cset 85c34c4dcf43 (bb)>>

[magreenblatt]

Move CookieManager callbacks to the UI thread (see issue #2622).

The Chromium content layer (which also exposes the NetworkService interface)
generally runs on the UI thread. Previous use of the IO thread for CookieManager
callbacks is an implementation detail of the old network stack that shouldn't be
exposed to clients.

To test: Run ceftests. They should pass as expected.

→ <<cset b949d86c40f8 (bb)>>

[magreenblatt]

• edited description

[magreenblatt]

The NetworkService process is created with a command line like:

You can debug the NetworkService by running a CEF-based application with --wait-for-debugger-children=utility and attaching to the appropriate child process. A good place to set a breakpoint is the posting site for UtilityServiceFactory::RunNetworkServiceOnIOThread.

A ContentUtilityClient::RegisterNetworkBinders method will also be called on startup.

There does not appear to be a capability for configuring the NetworkService directly from inside that process. Instead,
the NetworkService is configured from the browser process using NetworkServiceClient. This object is created via CefContentBrowserClient::CreateNetworkContext and followed by a call to CefContentBrowserClient::OnNetworkServiceCreated, where it can be retrieved using network_service->client(). This interface cannot overridden directly but instead delegates handling to objects returned via callbacks like ContentBrowserClient::CreateLoginDelegate.

[magreenblatt]

We need to call CookieMonster::SetCookieableSchemes in the NetworkProcess after the CookieMonster is created [1] and before CookieMonster::MarkCookieStoreAsInitialized is called for the first time [2]. Simply adding a CookieManager::SetCookieableSchemes method (in cookie_manager.h and cookie_manager.mojom) is not sufficient because URLLoaderFactory::CreateLoaderAndStart will always be called first.

CookieManagerParams (discussed above) has the following members:

  // Schemes that unconditionally allow cookies from secure origins.
  array<string> secure_origin_cookies_allowed_schemes;

  // Schemes that unconditionally allow cookies from the same scheme.
  array<string> matching_scheme_cookies_allowed_schemes;

  // Schemes that unconditionally allow third party cookies.
  array<string> third_party_cookies_allowed_schemes;

These are checked via CookieSettings::GetCookieSetting but not currently considered by NetworkContext::ApplyContextParamsToBuilder when creating the CookieMonster (where they could be accessed via params_->cookie_manager_params).

The best option is probably to add a new value in CookieManagerParams which would be set in ProfileNetworkContextService::CreateNetworkContextParams. On the NetworkService side this value would be passed to CookieMonster::SetCookieableSchemes immediately after the CookieMonster is created in NetworkContext::ApplyContextParamsToBuilder.

[1] Creation call stack:

>	net.dll!net::CookieMonster::CookieMonster(scoped_refptr<net::CookieMonster::PersistentCookieStore> store, net::ChannelIDService * channel_id_service, base::TimeDelta last_access_threshold, net::NetLog * net_log) Line 369	C++
 	net.dll!net::CookieMonster::CookieMonster(scoped_refptr<net::CookieMonster::PersistentCookieStore> store, net::ChannelIDService * channel_id_service, net::NetLog * net_log) Line 346	C++
 	net.dll!net::URLRequestContextBuilder::Build() Line 443	C++
 	network_service.dll!network::NetworkContext::ApplyContextParamsToBuilder(network::URLRequestContextBuilderMojo * builder) Line 1973	C++
 	network_service.dll!network::NetworkContext::MakeURLRequestContext() Line 2193	C++
 	network_service.dll!network::NetworkContext::NetworkContext(network::NetworkService * network_service, mojo::InterfaceRequest<network::mojom::NetworkContext> request, mojo::StructPtr<network::mojom::NetworkContextParams> params, base::OnceCallback<void (network::NetworkContext *)> on_connection_close_callback) Line 535	C++
 	[External Code]	
 	network_service.dll!network::NetworkService::CreateNetworkContext(mojo::InterfaceRequest<network::mojom::NetworkContext> request, mojo::StructPtr<network::mojom::NetworkContextParams> params) Line 446	C++
 	network_service.dll!network::mojom::NetworkServiceStubDispatch::Accept(network::mojom::NetworkService * impl, mojo::Message * message) Line 3872	C++

[2] MarkCookieStoreAsInitialized call stack:

>	net.dll!net::CookieMonster::MarkCookieStoreAsInitialized() Line 784	C++
 	net.dll!net::CookieMonster::DoCookieCallbackForHostOrDomain(base::OnceCallback<void ()> callback, base::BasicStringPiece<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > host_or_domain) Line 1825	C++
 	net.dll!net::CookieMonster::DoCookieCallbackForURL(base::OnceCallback<void ()> callback, const GURL & url) Line 1819	C++
 	net.dll!net::CookieMonster::GetCookieListWithOptionsAsync(const GURL & url, const net::CookieOptions & options, base::OnceCallback<void (const std::vector<net::CanonicalCookie,std::allocator<net::CanonicalCookie> > &, const std::vector<net::CookieWithStatus,std::allocator<net::CookieWithStatus> > &)> callback) Line 457	C++
 	net.dll!net::URLRequestHttpJob::AddCookieHeaderAndStart() Line 705	C++
 	net.dll!net::URLRequestHttpJob::Start() Line 442	C++
 	net.dll!net::URLRequest::StartJob(net::URLRequestJob * job) Line 698	C++
 	net.dll!net::URLRequest::BeforeRequestComplete(int error) Line 638	C++
 	net.dll!net::URLRequest::Start() Line 557	C++
 	network_service.dll!network::URLLoader::ScheduleStart() Line 531	C++
 	network_service.dll!network::URLLoader::URLLoader(net::URLRequestContext * url_request_context, network::mojom::NetworkServiceClient * network_service_client, base::OnceCallback<void (network::mojom::URLLoader *)> delete_callback, mojo::InterfaceRequest<network::mojom::URLLoader> url_loader_request, int options, const network::ResourceRequest & request, mojo::InterfacePtr<network::mojom::URLLoaderClient> url_loader_client, const net::NetworkTrafficAnnotationTag & traffic_annotation, const network::mojom::URLLoaderFactoryParams * factory_params, unsigned int request_id, scoped_refptr<network::ResourceSchedulerClient> resource_scheduler_client, base::WeakPtr<network::KeepaliveStatisticsRecorder> keepalive_statistics_recorder, base::WeakPtr<network::NetworkUsageAccumulator> network_usage_accumulator, network::mojom::TrustedURLLoaderHeaderClient * url_loader_header_client) Line 447	C++
 	[External Code]	
 	network_service.dll!network::URLLoaderFactory::CreateLoaderAndStart(mojo::InterfaceRequest<network::mojom::URLLoader> request, int routing_id, int request_id, unsigned int options, const network::ResourceRequest & url_request, mojo::InterfacePtr<network::mojom::URLLoaderClient> client, const net::MutableNetworkTrafficAnnotationTag & traffic_annotation) Line 135	C++
 	network_service.dll!network::cors::CorsURLLoaderFactory::CreateLoaderAndStart(mojo::InterfaceRequest<network::mojom::URLLoader> request, int routing_id, int request_id, unsigned int options, const network::ResourceRequest & resource_request, mojo::InterfacePtr<network::mojom::URLLoaderClient> client, const net::MutableNetworkTrafficAnnotationTag & traffic_annotation) Line 121	C++
 	network_service.dll!network::mojom::URLLoaderFactoryStubDispatch::Accept(network::mojom::URLLoaderFactory * impl, mojo::Message * message) Line 210	C++

[magreenblatt]

Original comment by Mike Wiedenbauer (Bitbucket: shagkur, GitHub: shagkur).

---

@magreenblatt I guess i then just can call CefBrowserContext::GetURLLoaderFactory(), as it seems it's exactly doing what you described i should do?:) I'll create a new class "CefURLLoader" which will be, implementation wise, based on SimpleURLLoader.

[magreenblatt]

@shagkur Sounds good, I didn't realize we had that method :D

[magreenblatt]

Fix crash on shutdown with extensions enabled (see issue #2622).

We don't support the WebRequestAPI yet, so we shouldn't be creating any objects
for it.

To test: Run cefclient --enable-network-service and then close the app.

→ <<cset 9ce29e8ec5cc (bb)>>

[magreenblatt]

Fix CHECK on startup when running with --trace-startup=mojom (see issue #2622).

→ <<cset 941d53ebfd57 (bb)>>

[magreenblatt]

Mojo messages related to the NetworkService can be debugged as follows:

1. Add enable_mojo_tracing=true to GN_DEFINES and rebuild.
2. Run with the --enable-network-service --trace-startup=mojom command-line flags.
3. Navigate to chrome://tracing in Google Chrome.
4. Click the "Load" button and select the chrometrace.log file created by step 2 (this file will be in your current working directory, or out\Debug_GN_x86\obj\cef when running with Visual Studio).

You'll see a view like this (go here for general information on using the tracing tool):

[magreenblatt]

Original comment by Mike Wiedenbauer (Bitbucket: shagkur, GitHub: shagkur).

---

@magreenblatt I've created the follwing PR: https://bitbucket.org/chromiumembedded/cef/pull-requests/227
It's still work in progress, but i'd appreciate any comments on this, please (so i can implement it in the correct way). It's based on SimpleURLLoader tho.
I had to decline (delete) the PR because i rebased my working branch on this to your "cef_spotify/net_service" fork/branch.
Now i'd like to create a PR from mine into your fork/branch rather than against the chromimumembedded master. How can i achive this?
The reason behind is that if i create against the chromimumembedded master i'd also create a PR for all your changes too, which would be wrong in that case.