Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: dependency injection of message validation. #7072

Merged
merged 10 commits into from
May 31, 2019

Conversation

htuch
Copy link
Member

@htuch htuch commented May 24, 2019

This is the plumbing to support #6651 and #6818, where we remove the global control over message
validation type and put it under the control of a new injectable interface. It's pretty horrible
IMHO that there is this much churn; the closest major context object that we could associate with is
Api.

The main from the new dependency is that we can:

  1. Inject different validators, e.g. one for static config when ingesting bootstrap resources, another for xDS.
  2. Validation state, so that we can reach back to the server stats for the purpose of tracking unknown fields.

This PR introduces a visitor pattern for the validator and starts to do the threading needed for dependency injection here.

It's possible that we could extend this visitor for deprecated fields in the future to provide finer grained control over these alongside unknown, if there is a use case.

No new behavior should be added in this PR.

Risk level: Low.
Testing: bazel test //test/...

Signed-off-by: Harvey Tuch htuch@google.com

@htuch htuch requested review from mattklein123 and alyssawilk May 24, 2019 18:46
@mattklein123 mattklein123 self-assigned this May 24, 2019
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skimmed and I can't think of a better way of doing this if we want to get rid of the global, which I agree we should do anyway. Also, per your comment, I think the visitor pattern can hopefully also allow us to report/log/etc. unknown fields when the proto folks add that support so this seems good regardless. +1 to continue.

/wait

@htuch
Copy link
Member Author

htuch commented May 28, 2019

@mattklein123 OK, I'm continuing down this path. So far I've converted the //source/... but not tests. It's pretty tedious, so I'm hoping someone speaks up if they do object to this approach before I dive into the test code.

This is the plumbing to support envoyproxy#6651 and envoyproxy#6818, where we remove the global control over message
validation type and put it under the control over a new injectable interface. It's pretty horrible
IMHO that there is this much churn; the closest major context object that we could associate with is
Api, but it's not part of the filter API as such and we will want to later inject different
validators at different points, depending on whether we are reading the static bootstrap or on xDS.

No new behavior should be added in this PR.

Risk level: Low.
Testing: bazel test //test/...

Signed-off-by: Harvey Tuch <htuch@google.com>
@htuch htuch force-pushed the unknown-fields branch from cc23262 to 2d1cd91 Compare May 30, 2019 03:52
@htuch htuch changed the title RFC WiP: injectable message validator for unknown fields. config: dependency injection of message validation. May 30, 2019
@htuch htuch marked this pull request as ready for review May 30, 2019 03:57
@htuch htuch requested review from lizan, snowp and zuercher as code owners May 30, 2019 03:57
htuch added 3 commits May 30, 2019 13:21
Signed-off-by: Harvey Tuch <htuch@google.com>
Signed-off-by: Harvey Tuch <htuch@google.com>
@htuch
Copy link
Member Author

htuch commented May 30, 2019

@mattklein123 this is ready for review. I suggest doing a pass as is, without CI passing if there are still failures, as any failures probably are interesting to independently examine as they get fixed.

Signed-off-by: Harvey Tuch <htuch@google.com>
@@ -108,6 +111,8 @@ void RdsRouteConfigSubscription::onConfigUpdate(
if (config_update_info_->routeConfiguration().has_vhds()) {
ENVOY_LOG(debug, "rds: vhds configuration present, starting vhds: config_name={} hash={}",
route_config_name_, config_update_info_->configHash());
// TODO(dmitri-d): It's unsafe to depend directly on factory context here,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmitri-d I tagged you here. The use of factory context outside of the constructor is probably unsafe, since the subscription might outlive the listener (at least that's what ASAN tells me).

Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this super tedious change, definitely for the better. LGTM w/ 2 small questions.

/wait-any

// TODO(htuch): clean this up when protobuf supports JSON/YAML unknown field
// detection directly.
options.ignore_unknown_fields = true;
const auto relaxed_status = Protobuf::util::JsonStringToMessage(json, &message, options);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think there are any perf concerns for doing this parse twice? I think not, but wanted to double check. If we care about perf you could probably short circuit by asking the visitor whether it wants to accept unknown fields or not which I think would be a very simple interface change. Not sure if that is worth it or not.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this won't have any performance implications on the happy path, since we only do a single JSON -> protobuf conversion there. It's only when we fail that we double work, in which case we are rejecting config. Once we have a clean solution to protocolbuffers/protobuf#5967, even the unhappy path can be fast.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

source/common/router/rds_impl.cc Outdated Show resolved Hide resolved
Signed-off-by: Harvey Tuch <htuch@google.com>
@htuch htuch merged commit 8d1ad35 into envoyproxy:master May 31, 2019
@htuch htuch deleted the unknown-fields branch May 31, 2019 22:41
htuch pushed a commit that referenced this pull request Jun 3, 2019
Fixes a typo from #7072.

Risk Level: Low

Signed-off-by: James Buckland <jbuckland@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants