On demand loading of ScopedRouteConfiguration (#12640)

Add a field to the current protobuf of ScopedRouteConfiguration to enable on demand scoped route table loading. The on demand scope route tables will be loaded lazily. The lazy loading feature of route table associated with scope is achieved by extending the current vhds on_demand filter to support lazy loading of RouteConfigurationscoped route discovery service.If a scoped route configuration is set to be loaded lazily, upon a http request using SRDS, when the corresponding route table of a scope is not found, post a callback to control plane, request the route table from the management server, after the route table has been initialized, continue the filter chain.

https://docs.google.com/document/d/15GX30U5CH2bsWUyQRkiiQ_nbMCoklvgP_ObrDaSlkuc/edit?usp=sharing

Risk Level: Low
Testing: add unit tests and integration test to verifiy behavior changes

Fixes #10641

Signed-off-by: chaoqinli <chaoqinli@google.com>

api/envoy/config/route/v3/scoped_route.proto

@@ -104,6 +104,9 @@ message ScopedRouteConfiguration {
     repeated Fragment fragments = 1 [(validate.rules).repeated = {min_items: 1}];
   }
 
+  // Whether the RouteConfiguration should be loaded on demand.
+  bool on_demand = 4;
+
   // The name assigned to the routing scope.
   string name = 1 [(validate.rules).string = {min_bytes: 1}];
 

docs/root/configuration/http/http_filters/on_demand_updates_filter.rst

@@ -1,15 +1,21 @@
 .. _config_http_filters_on_demand:
 
-On-demand VHDS Updates
-======================
+On-demand VHDS and S/RDS Updates
+================================
 
-The on-demand VHDS filter is used to request a :ref:`virtual host <envoy_v3_api_msg_config.route.v3.VirtualHost>`
+The on demand filter can be used to support either on demand VHDS or S/RDS update if configured in the filter chain.
+
+The on-demand update filter can be used to request a :ref:`virtual host <envoy_v3_api_msg_config.route.v3.VirtualHost>`
 data if it's not already present in the :ref:`Route Configuration <envoy_v3_api_msg_config.route.v3.RouteConfiguration>`. The
 contents of the *Host* or *:authority* header is used to create the on-demand request. For an on-demand
 request to be created, :ref:`VHDS <envoy_v3_api_field_config.route.v3.RouteConfiguration.vhds>` must be enabled and either *Host*
 or *:authority* header be present.
 
-On-demand VHDS cannot be used with SRDS at this point.
+The on-demand update filter can also be used to request a *Route Configuration* data if RouteConfiguration is specified to be 
+loaded on demand in the :ref:`Scoped RouteConfiguration <envoy_v3_api_msg_config.route.v3.ScopedRouteConfiguration>`. 
+The contents of the HTTP header is used to find the scope and create the on-demand request. 
+
+On-demand VHDS and on-demand S/RDS can not be used at the same time at this point.
 
 Configuration
 -------------

docs/root/intro/arch_overview/http/http_routing.rst

@@ -59,10 +59,11 @@ Route Scope
 Scoped routing enables Envoy to put constraints on search space of domains and route rules.
 A :ref:`Route Scope<envoy_api_msg_ScopedRouteConfiguration>` associates a key with a :ref:`route table <arch_overview_http_routing_route_table>`.
 For each request, a scope key is computed dynamically by the HTTP connection manager to pick the :ref:`route table<envoy_api_msg_RouteConfiguration>`.
+RouteConfiguration associated with scope can be loaded on demand with :ref:`v3 API reference <envoy_v3_api_msg_extensions.filters.http.on_demand.v3.OnDemand>` configured and on demand filed in protobuf set to true.
 
 The Scoped RDS (SRDS) API contains a set of :ref:`Scopes <envoy_v3_api_msg_config.route.v3.ScopedRouteConfiguration>` resources, each defining independent routing configuration,
 along with a :ref:`ScopeKeyBuilder <envoy_v3_api_msg_extensions.filters.network.http_connection_manager.v3.ScopedRoutes.ScopeKeyBuilder>`
-defining the key construction algorithm used by Envoy to look up the scope corresponding to each request.
+defining the key construction algorithm used by Envoy to look up the scope corresponding to each request. 
 
 For example, for the following scoped route configuration, Envoy will look into the "addr" header value, split the header value by ";" first, and use the first value for key 'x-foo-key' as the scope key.
 If the "addr" header value is "foo=1;x-foo-key=127.0.0.1;x-bar-key=1.1.1.1", then "127.0.0.1" will be computed as the scope key to look up for corresponding route configuration.

include/envoy/http/filter.h

@@ -523,15 +523,6 @@ class StreamDecoderFilterCallbacks : public virtual StreamFilterCallbacks {
    */
   virtual void
   requestRouteConfigUpdate(RouteConfigUpdatedCallbackSharedPtr route_config_updated_cb) PURE;
-
-  /**
-   *
-   * @return absl::optional<Router::ConfigConstSharedPtr>. Contains a value if a non-scoped RDS
-   * route config provider is used. Scoped RDS provides are not supported at the moment, as
-   * retrieval of a route configuration in their case requires passing of http request headers
-   * as a parameter.
-   */
-  virtual absl::optional<Router::ConfigConstSharedPtr> routeConfig() PURE;
 };
 
 /**

include/envoy/router/scopes.h

@@ -8,6 +8,77 @@
 namespace Envoy {
 namespace Router {
 
+/**
+ * Scope key fragment base class.
+ */
+class ScopeKeyFragmentBase {
+public:
+  bool operator!=(const ScopeKeyFragmentBase& other) const { return !(*this == other); }
+
+  bool operator==(const ScopeKeyFragmentBase& other) const {
+    if (typeid(*this) == typeid(other)) {
+      return hash() == other.hash();
+    }
+    return false;
+  }
+  virtual ~ScopeKeyFragmentBase() = default;
+
+  // Hash of the fragment.
+  virtual uint64_t hash() const PURE;
+};
+
+/**
+ *  Scope Key is composed of non-null fragments.
+ **/
+class ScopeKey {
+public:
+  ScopeKey() = default;
+  ScopeKey(ScopeKey&& other) = default;
+
+  // Scopekey is not copy-assignable and copy-constructible as it contains unique_ptr inside itself.
+  ScopeKey(const ScopeKey&) = delete;
+  ScopeKey operator=(const ScopeKey&) = delete;
+
+  // Caller should guarantee the fragment is not nullptr.
+  void addFragment(std::unique_ptr<ScopeKeyFragmentBase>&& fragment) {
+    ASSERT(fragment != nullptr, "null fragment not allowed in ScopeKey.");
+    updateHash(*fragment);
+    fragments_.emplace_back(std::move(fragment));
+  }
+
+  uint64_t hash() const { return hash_; }
+  bool operator!=(const ScopeKey& other) const;
+  bool operator==(const ScopeKey& other) const;
+
+private:
+  // Update the key's hash with the new fragment hash.
+  void updateHash(const ScopeKeyFragmentBase& fragment) {
+    std::stringbuf buffer;
+    buffer.sputn(reinterpret_cast<const char*>(&hash_), sizeof(hash_));
+    const auto& fragment_hash = fragment.hash();
+    buffer.sputn(reinterpret_cast<const char*>(&fragment_hash), sizeof(fragment_hash));
+    hash_ = HashUtil::xxHash64(buffer.str());
+  }
+
+  uint64_t hash_{0};
+  std::vector<std::unique_ptr<ScopeKeyFragmentBase>> fragments_;
+};
+
+using ScopeKeyPtr = std::unique_ptr<ScopeKey>;
+
+// String fragment.
+class StringKeyFragment : public ScopeKeyFragmentBase {
+public:
+  explicit StringKeyFragment(absl::string_view value)
+      : value_(value), hash_(HashUtil::xxHash64(value_)) {}
+
+  uint64_t hash() const override { return hash_; }
+
+private:
+  const std::string value_;
+  const uint64_t hash_;
+};
+
 /**
  * The scoped routing configuration.
  */
@@ -22,6 +93,13 @@ class ScopedConfig : public Envoy::Config::ConfigProvider::Config {
    * @return ConfigConstSharedPtr the router's Config matching the request headers.
    */
   virtual ConfigConstSharedPtr getRouteConfig(const Http::HeaderMap& headers) const PURE;
+
+  /**
+   * Based on the incoming HTTP request headers, returns the hash value of its scope key.
+   * @param headers the request headers to match the scoped routing configuration against.
+   * @return unique_ptr of the scope key computed from header.
+   */
+  virtual ScopeKeyPtr computeScopeKey(const Http::HeaderMap&) const { return {}; }
 };
 
 using ScopedConfigConstSharedPtr = std::shared_ptr<const ScopedConfig>;

source/common/http/BUILD

@@ -240,6 +240,7 @@ envoy_cc_library(
         "//source/common/http/http3:well_known_names",
         "//source/common/network:utility_lib",
         "//source/common/router:config_lib",
+        "//source/common/router:scoped_rds_lib",
         "//source/common/stats:timespan_lib",
         "//source/common/stream_info:stream_info_lib",
         "//source/common/tracing:http_tracer_lib",

source/common/http/async_client_impl.h

@@ -86,7 +86,6 @@ class AsyncStreamImpl : public AsyncClient::Stream,
   void requestRouteConfigUpdate(Http::RouteConfigUpdatedCallbackSharedPtr) override {
     NOT_IMPLEMENTED_GCOVR_EXCL_LINE;
   }
-  absl::optional<Router::ConfigConstSharedPtr> routeConfig() override { return {}; }
 
   // Http::AsyncClient::Stream
   void sendHeaders(RequestHeaderMap& headers, bool end_stream) override;

source/common/http/conn_manager_impl.cc

@@ -480,13 +480,61 @@ void ConnectionManagerImpl::chargeTracingStats(const Tracing::Reason& tracing_re
   }
 }
 
+// TODO(chaoqin-li1123): Make on demand vhds and on demand srds works at the same time.
 void ConnectionManagerImpl::RdsRouteConfigUpdateRequester::requestRouteConfigUpdate(
-    const std::string host_header, Event::Dispatcher& thread_local_dispatcher,
+    Http::RouteConfigUpdatedCallbackSharedPtr route_config_updated_cb) {
+  absl::optional<Router::ConfigConstSharedPtr> route_config = parent_.routeConfig();
+  Event::Dispatcher& thread_local_dispatcher =
+      parent_.connection_manager_.read_callbacks_->connection().dispatcher();
+  if (route_config.has_value() && route_config.value()->usesVhds()) {
+    ASSERT(!parent_.request_headers_->Host()->value().empty());
+    const auto& host_header = absl::AsciiStrToLower(parent_.request_headers_->getHostValue());
+    requestVhdsUpdate(host_header, thread_local_dispatcher, std::move(route_config_updated_cb));
+    return;
+  } else if (parent_.snapped_scoped_routes_config_ != nullptr) {
+    Router::ScopeKeyPtr scope_key =
+        parent_.snapped_scoped_routes_config_->computeScopeKey(*parent_.request_headers_);
+    // If scope_key is not null, the scope exists but RouteConfiguration is not initialized.
+    if (scope_key != nullptr) {
+      requestSrdsUpdate(std::move(scope_key), thread_local_dispatcher,
+                        std::move(route_config_updated_cb));
+      return;
+    }
+  }
+  // Continue the filter chain if no on demand update is requested.
+  (*route_config_updated_cb)(false);
+}
+
+void ConnectionManagerImpl::RdsRouteConfigUpdateRequester::requestVhdsUpdate(
+    const std::string& host_header, Event::Dispatcher& thread_local_dispatcher,
     Http::RouteConfigUpdatedCallbackSharedPtr route_config_updated_cb) {
   route_config_provider_->requestVirtualHostsUpdate(host_header, thread_local_dispatcher,
                                                     std::move(route_config_updated_cb));
 }
 
+void ConnectionManagerImpl::RdsRouteConfigUpdateRequester::requestSrdsUpdate(
+    Router::ScopeKeyPtr scope_key, Event::Dispatcher& thread_local_dispatcher,
+    Http::RouteConfigUpdatedCallbackSharedPtr route_config_updated_cb) {
+  // Since inline scope_route_config_provider is not fully implemented and never used,
+  // dynamic cast in constructor always succeed and the pointer should not be null here.
+  ASSERT(scoped_route_config_provider_ != nullptr);
+  Http::RouteConfigUpdatedCallback scoped_route_config_updated_cb =
+      Http::RouteConfigUpdatedCallback(
+          [this, weak_route_config_updated_cb = std::weak_ptr<Http::RouteConfigUpdatedCallback>(
+                     route_config_updated_cb)](bool scope_exist) {
+            // If the callback can be locked, this ActiveStream is still alive.
+            if (auto cb = weak_route_config_updated_cb.lock()) {
+              // Refresh the route before continue the filter chain.
+              if (scope_exist) {
+                parent_.refreshCachedRoute();
+              }
+              (*cb)(scope_exist && parent_.hasCachedRoute());
+            }
+          });
+  scoped_route_config_provider_->onDemandRdsUpdate(std::move(scope_key), thread_local_dispatcher,
+                                                   std::move(scoped_route_config_updated_cb));
+}
+
 ConnectionManagerImpl::ActiveStream::ActiveStream(ConnectionManagerImpl& connection_manager,
                                                   uint32_t buffer_limit)
     : connection_manager_(connection_manager),
@@ -520,11 +568,12 @@ ConnectionManagerImpl::ActiveStream::ActiveStream(ConnectionManagerImpl& connect
       connection_manager.config_.routeConfigProvider() != nullptr) {
     route_config_update_requester_ =
         std::make_unique<ConnectionManagerImpl::RdsRouteConfigUpdateRequester>(
-            connection_manager.config_.routeConfigProvider());
+            connection_manager.config_.routeConfigProvider(), *this);
   } else if (connection_manager_.config_.isRoutable() &&
              connection_manager.config_.scopedRouteConfigProvider() != nullptr) {
     route_config_update_requester_ =
-        std::make_unique<ConnectionManagerImpl::NullRouteConfigUpdateRequester>();
+        std::make_unique<ConnectionManagerImpl::RdsRouteConfigUpdateRequester>(
+            connection_manager.config_.scopedRouteConfigProvider(), *this);
   }
   ScopeTrackerScopeState scope(this,
                                connection_manager_.read_callbacks_->connection().dispatcher());
@@ -1138,21 +1187,18 @@ void ConnectionManagerImpl::ActiveStream::refreshCachedTracingCustomTags() {
   }
 }
 
+// TODO(chaoqin-li1123): Make on demand vhds and on demand srds works at the same time.
 void ConnectionManagerImpl::ActiveStream::requestRouteConfigUpdate(
-    Event::Dispatcher& thread_local_dispatcher,
     Http::RouteConfigUpdatedCallbackSharedPtr route_config_updated_cb) {
-  ASSERT(!request_headers_->Host()->value().empty());
-  const auto& host_header = absl::AsciiStrToLower(request_headers_->getHostValue());
-  route_config_update_requester_->requestRouteConfigUpdate(host_header, thread_local_dispatcher,
-                                                           std::move(route_config_updated_cb));
+  route_config_update_requester_->requestRouteConfigUpdate(route_config_updated_cb);
 }
 
 absl::optional<Router::ConfigConstSharedPtr> ConnectionManagerImpl::ActiveStream::routeConfig() {
-  if (connection_manager_.config_.routeConfigProvider() == nullptr) {
-    return {};
+  if (connection_manager_.config_.routeConfigProvider() != nullptr) {
+    return absl::optional<Router::ConfigConstSharedPtr>(
+        connection_manager_.config_.routeConfigProvider()->config());
   }
-  return absl::optional<Router::ConfigConstSharedPtr>(
-      connection_manager_.config_.routeConfigProvider()->config());
+  return {};
 }
 
 void ConnectionManagerImpl::ActiveStream::onLocalReply(Code code) {
@@ -1196,7 +1242,8 @@ void ConnectionManagerImpl::ActiveStream::encodeHeaders(ResponseHeaderMap& heade
     connection_manager_.config_.dateProvider().setDateHeader(headers);
   }
 
-  // Following setReference() is safe because serverName() is constant for the life of the listener.
+  // Following setReference() is safe because serverName() is constant for the life of the
+  // listener.
   const auto transformation = connection_manager_.config_.serverHeaderTransformation();
   if (transformation == ConnectionManagerConfig::HttpConnectionManagerProto::OVERWRITE ||
       (transformation == ConnectionManagerConfig::HttpConnectionManagerProto::APPEND_IF_ABSENT &&