Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable check cache and refactory mixer config loading #197

Merged
merged 5 commits into from
Mar 23, 2017
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/envoy/mixer/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ cc_proto_library(
cc_library(
name = "filter_lib",
srcs = [
"config.cc",
"config.h",
"http_control.cc",
"http_control.h",
"http_filter.cc",
Expand Down
20 changes: 13 additions & 7 deletions src/envoy/mixer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,7 @@ This Proxy will use Envoy and talk to Mixer server.
curl http://localhost:7070/echo -d "hello world"
```

## How to configurate HTTP filters

### *mixer* filter:
## How to configurate HTTP Mixer filters

This filter will intercept all HTTP requests and call Mixer. Here is its config:

Expand All @@ -72,17 +70,25 @@ This filter will intercept all HTTP requests and call Mixer. Here is its config:
"forward_attributes" : {
"attribute_name1": "attribute_value1",
"attribute_name2": "attribute_value2"
}
},
"quota_name": "RequestCount",
"quota_amount": "1",
"check_cache_keys": [
"request.host",
"request.path",
"origin.user"
]
}
```

Notes:
* mixer_server is required
* mixer_attributes: these attributes will be send to the mixer
* forward_attributes: these attributes will be forwarded to the upstream istio/proxy.
* "quota.name" and "quota.amount" are used for quota call. "quota.amount" is default to 1 if missing.
* quota_name, quota_amount are used for making quota call. quota_amount is default to 1 if missing.
* check_cache_keys is to cache check calls. If missing or empty, check calls are not cached.

By default, mixer filter forwards attributes and does not invoke mixer server. You can customize this behavior per HTTP route by supplying an opaque config:
By default, mixer filter forwards attributes and does not invoke mixer server. You can customize this behavior per HTTP route by supplying an opaque config in the route config:

```
"opaque_config": {
Expand All @@ -91,4 +97,4 @@ By default, mixer filter forwards attributes and does not invoke mixer server. Y
}
```

This config reverts the behavior by sending requests to mixer server but not forwarding any attributes.
This route opaque config reverts the behavior by sending requests to mixer server but not forwarding any attributes.
96 changes: 96 additions & 0 deletions src/envoy/mixer/config.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
/* Copyright 2017 Istio Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#include "src/envoy/mixer/config.h"

using ::istio::mixer_client::Attributes;

namespace Http {
namespace Mixer {
namespace {

// The Json object name for mixer-server.
const std::string kJsonNameMixerServer("mixer_server");

// The Json object name for static attributes.
const std::string kJsonNameMixerAttributes("mixer_attributes");

// The Json object name to specify attributes which will be forwarded
// to the upstream istio proxy.
const std::string kJsonNameForwardAttributes("forward_attributes");

// The Json object name for quota name and amount.
const std::string kJsonNameQuotaName("quota_name");
const std::string kJsonNameQuotaAmount("quota_amount");

// The Json object name for check cache keys.
const std::string kJsonNameCheckCacheKeys("check_cache_keys");

void ReadString(const Json::Object& json, const std::string& name,
std::string* value) {
if (json.hasObject(name)) {
*value = json.getString(name);
}
}

void ReadStringMap(const Json::Object& json, const std::string& name,
std::map<std::string, std::string>* map) {
if (json.hasObject(name)) {
json.getObject(name)->iterate(
[map](const std::string& key, const Json::Object& obj) -> bool {
(*map)[key] = obj.asString();
return true;
});
}
}

void ReadStringVector(const Json::Object& json, const std::string& name,
std::vector<std::string>* value) {
if (json.hasObject(name)) {
auto v = json.getStringArray(name);
value->swap(v);
}
}

} // namespace

void MixerConfig::Load(const Json::Object& json) {
ReadString(json, kJsonNameMixerServer, &mixer_server);

ReadStringMap(json, kJsonNameMixerAttributes, &mixer_attributes);
ReadStringMap(json, kJsonNameForwardAttributes, &forward_attributes);

ReadString(json, kJsonNameQuotaName, &quota_name);
ReadString(json, kJsonNameQuotaAmount, &quota_amount);

ReadStringVector(json, kJsonNameCheckCacheKeys, &check_cache_keys);
}

void MixerConfig::ExtractQuotaAttributes(Attributes* attr) const {
if (!quota_name.empty()) {
attr->attributes[ ::istio::mixer_client::kQuotaName] =
Attributes::StringValue(quota_name);

int64_t amount = 1; // default amount to 1.
if (!quota_amount.empty()) {
amount = std::stoi(quota_amount);
}
attr->attributes[ ::istio::mixer_client::kQuotaAmount] =
Attributes::Int64Value(amount);
}
}

} // namespace Mixer
} // namespace Http
53 changes: 53 additions & 0 deletions src/envoy/mixer/config.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/* Copyright 2017 Istio Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#pragma once

#include "precompiled/precompiled.h"

#include "envoy/json/json_object.h"
#include "include/attribute.h"

namespace Http {
namespace Mixer {

// A config for mixer filter
struct MixerConfig {
// the mixer server address
std::string mixer_server;

// These static attributes will be send to mixer in both
// Check and Report.
std::map<std::string, std::string> mixer_attributes;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may lose some context here. Just wondering what is the relation between mixer_attributes and forward_attributes? And why we need a standalone quota attributes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mixer_attributes will be send to mixer server
forward_attributes will be forwarded to upstream so that next hop proxy can send them to mixer server.
Above attributes are for Check and Report.
Quota is a separate call. Here we use some standalone attributes to trigger the calls.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will mixer_attributes and forward_attributes have overlap/different entries?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they have overlapped entries, mixer_attributes win.


// These attributes will be forwarded to upstream.
std::map<std::string, std::string> forward_attributes;

// Quota attributes.
std::string quota_name;
std::string quota_amount;

// The attribute names for check cache.
std::vector<std::string> check_cache_keys;

// Load the config from envoy config.
void Load(const Json::Object& json);

// Extract quota attributes.
void ExtractQuotaAttributes(::istio::mixer_client::Attributes* attr) const;
};

} // namespace Mixer
} // namespace Http
12 changes: 9 additions & 3 deletions src/envoy/mixer/envoy.conf.template
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,15 @@
"mixer_server": "${MIXER_SERVER}",
"mixer_attributes": {
"target.uid": "POD222",
"target.namespace": "XYZ222",
"quota.name": "RequestCount"
}
"target.namespace": "XYZ222"
},
"quota_name": "RequestCount",
"quota_amount": "1",
"check_cache_keys": [
"request.host",
"request.path",
"origin.user"
]
}
},
{
Expand Down
29 changes: 8 additions & 21 deletions src/envoy/mixer/http_control.cc
Original file line number Diff line number Diff line change
Expand Up @@ -108,29 +108,16 @@ void FillRequestInfoAttributes(const AccessLog::RequestInfo& info,

} // namespace

HttpControl::HttpControl(const std::string& mixer_server,
std::map<std::string, std::string>&& attributes)
: config_attributes_(std::move(attributes)) {
HttpControl::HttpControl(const MixerConfig& mixer_config)
: mixer_config_(mixer_config) {
::istio::mixer_client::MixerClientOptions options;
options.mixer_server = mixer_server;
options.mixer_server = mixer_config_.mixer_server;
options.check_options.cache_keys.insert(
mixer_config_.check_cache_keys.begin(),
mixer_config_.check_cache_keys.end());
mixer_client_ = ::istio::mixer_client::CreateMixerClient(options);

// Extract quota attributes
auto it = config_attributes_.find(::istio::mixer_client::kQuotaName);
if (it != config_attributes_.end()) {
quota_attributes_.attributes[ ::istio::mixer_client::kQuotaName] =
Attributes::StringValue(it->second);
config_attributes_.erase(it);

int64_t amount = 1; // default amount to 1.
it = config_attributes_.find(::istio::mixer_client::kQuotaAmount);
if (it != config_attributes_.end()) {
amount = std::stoi(it->second);
config_attributes_.erase(it);
}
quota_attributes_.attributes[ ::istio::mixer_client::kQuotaAmount] =
Attributes::Int64Value(amount);
}
mixer_config_.ExtractQuotaAttributes(&quota_attributes_);
}

void HttpControl::FillCheckAttributes(HeaderMap& header_map, Attributes* attr) {
Expand All @@ -148,7 +135,7 @@ void HttpControl::FillCheckAttributes(HeaderMap& header_map, Attributes* attr) {

FillRequestHeaderAttributes(header_map, attr);

for (const auto& attribute : config_attributes_) {
for (const auto& attribute : mixer_config_.mixer_attributes) {
SetStringAttribute(attribute.first, attribute.second, attr);
}
}
Expand Down
8 changes: 4 additions & 4 deletions src/envoy/mixer/http_control.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
#include "common/http/headers.h"
#include "envoy/http/access_log.h"
#include "include/client.h"
#include "src/envoy/mixer/config.h"

namespace Http {
namespace Mixer {
Expand All @@ -37,8 +38,7 @@ typedef std::shared_ptr<HttpRequestData> HttpRequestDataPtr;
class HttpControl final : public Logger::Loggable<Logger::Id::http> {
public:
// The constructor.
HttpControl(const std::string& mixer_server,
std::map<std::string, std::string>&& attributes);
HttpControl(const MixerConfig& mixer_config);

// Make mixer check call.
void Check(HttpRequestDataPtr request_data, HeaderMap& headers,
Expand All @@ -56,8 +56,8 @@ class HttpControl final : public Logger::Loggable<Logger::Id::http> {

// The mixer client
std::unique_ptr<::istio::mixer_client::MixerClient> mixer_client_;
// The attributes read from the config file.
std::map<std::string, std::string> config_attributes_;
// The mixer config
const MixerConfig& mixer_config_;
// Quota attributes; extracted from envoy filter config.
::istio::mixer_client::Attributes quota_attributes_;
};
Expand Down
36 changes: 11 additions & 25 deletions src/envoy/mixer/http_filter.cc
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#include "envoy/server/instance.h"
#include "envoy/ssl/connection.h"
#include "server/config/network/http_connection_manager.h"
#include "src/envoy/mixer/config.h"
#include "src/envoy/mixer/http_control.h"
#include "src/envoy/mixer/utils.h"

Expand All @@ -33,16 +34,6 @@ namespace Http {
namespace Mixer {
namespace {

// The Json object name for mixer-server.
const std::string kJsonNameMixerServer("mixer_server");

// The Json object name for static attributes.
const std::string kJsonNameMixerAttributes("mixer_attributes");

// The Json object name to specify attributes which will be forwarded
// to the upstream istio proxy.
const std::string kJsonNameForwardAttributes("forward_attributes");

// Switch to turn off attribute forwarding
const std::string kJsonNameForwardSwitch("mixer_forward");

Expand Down Expand Up @@ -100,35 +91,30 @@ class Config : public Logger::Loggable<Logger::Id::http> {
std::shared_ptr<HttpControl> http_control_;
Upstream::ClusterManager& cm_;
std::string forward_attributes_;
MixerConfig mixer_config_;

public:
Config(const Json::Object& config, Server::Instance& server)
: cm_(server.clusterManager()) {
std::string mixer_server;
if (config.hasObject(kJsonNameMixerServer)) {
mixer_server = config.getString(kJsonNameMixerServer);
} else {
mixer_config_.Load(config);
if (mixer_config_.mixer_server.empty()) {
log().error(
"mixer_server is required but not specified in the config: {}",
__func__);
} else {
log().debug("Called Mixer::Config constructor with mixer_server: ",
mixer_config_.mixer_server);
}

Utils::StringMap attributes =
Utils::ExtractStringMap(config, kJsonNameForwardAttributes);
if (!attributes.empty()) {
std::string serialized_str = Utils::SerializeStringMap(attributes);
if (!mixer_config_.forward_attributes.empty()) {
std::string serialized_str =
Utils::SerializeStringMap(mixer_config_.forward_attributes);
forward_attributes_ =
Base64::encode(serialized_str.c_str(), serialized_str.size());
log().debug("Mixer forward attributes set: ", serialized_str);
}

std::map<std::string, std::string> mixer_attributes =
Utils::ExtractStringMap(config, kJsonNameMixerAttributes);

http_control_ = std::make_shared<HttpControl>(mixer_server,
std::move(mixer_attributes));
log().debug("Called Mixer::Config constructor with mixer_server: ",
mixer_server);
http_control_ = std::make_shared<HttpControl>(mixer_config_);
}

std::shared_ptr<HttpControl>& http_control() { return http_control_; }
Expand Down
12 changes: 8 additions & 4 deletions src/envoy/mixer/integration_test/envoy.conf
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,14 @@
"mixer_server": "localhost:29091",
"mixer_attributes": {
"target.uid": "POD222",
"target.namespace": "XYZ222",
"quota.name": "RequestCount",
"quota.amount": "5"
}
"target.namespace": "XYZ222"
},
"quota_name": "RequestCount",
"quota_amount": "5",
"check_cache_keys": [
"request.host",
"request.path"
]
}
},
{
Expand Down
Loading