[1.12] [http1] Preserve LWS from the middle of HTTP1 header values that requ… #12319

lambdai · 2020-07-28T06:13:03Z

…ire multiple dispatch calls to process (#10886)

Commit Message:

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Signed-off-by: Antonio Vicente avd@google.com
Signed-off-by: Yuchen Dai silentdai@gmail.com

Additional Description:
Allow compiling on clang-10
Risk Level:
Testing:
Docs Changes:
Release Notes:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]

…ire multiple dispatch calls to process (envoyproxy#10886) Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice. Signed-off-by: Antonio Vicente <avd@google.com> Signed-off-by: Yuchen Dai <silentdai@gmail.com>

lambdai · 2020-07-28T06:29:41Z

source/common/http/header_map_impl.cc

@@ -150,6 +150,46 @@ void HeaderString::append(const char* data, uint32_t size) {
  string_length_ += size;
 }

+void HeaderString::rtrim() {


I am not sure why and how rtrim() is invoked only with Type::Inline implementation.
It might be true in #10886 but not here.
Any advices?

There are 2 operations that are done on HeaderString: Construction and append. Construction of an empty HeaderString results in an Inline headerstring. append converts to inline if not already inline.

Can you provide info on where rtrim is called on non Inline headerstrings?

lambdai · 2020-07-28T08:54:23Z

/azp run

azure-pipelines · 2020-07-28T08:54:32Z

Azure Pipelines successfully started running 1 pipeline(s).

source/common/http/header_map_impl.cc

test/common/http/header_map_impl_test.cc

lambdai · 2020-07-28T23:47:02Z

I will fix the DCO after

lambdai · 2020-07-28T23:48:30Z

I may accidentally introduce the cx_integration_test...
Sorry I need to force push

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

antoniovicente

There seem to be some spurious additional changes, but the functional parts of the PR look correct.

* doc: version 1.12.7 Signed-off-by: Yuchen Dai <silentdai@gmail.com> * Preserve LWS from the middle of HTTP1 header values that requ… (envoyproxy#12319) * [http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process (envoyproxy#10886) Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice. Also various fix to allow build on clang-10 Signed-off-by: Antonio Vicente <avd@google.com> Signed-off-by: Yuchen Dai <silentdai@gmail.com> * http: header map security fixes for duplicate headers (#197) (#204) Previously header matching did not match on all headers for non-inline headers. This patch changes the default behavior to always logically match on all headers. Multiple individual headers will be logically concatenated with ',' similar to what is done with inline headers. This makes the behavior effectively consistent. This behavior can be temporary reverted by setting the runtime value "envoy.reloadable_features.header_match_on_all_headers" to "false". Targeted fixes have been additionally performed on the following extensions which make them consider all duplicate headers by default as a comma concatenated list: 1) Any extension using CEL matching on headers. 2) The header to metadata filter. 3) The JWT filter. 4) The Lua filter. Like primary header matching used in routing, RBAC, etc. this behavior can be disabled by setting the runtime value "envoy.reloadable_features.header_match_on_all_headers" to false. Finally, the setCopy() header map API previously only set the first header in the case of duplicate non-inline headers. setCopy() now behaves similiarly to the other set*() APIs and replaces all found headers with a single value. This may have had security implications in the extauth filter which uses this API. This behavior can be disabled by setting the runtime value "envoy.reloadable_features.http_set_copy_replace_all_headers" to false. Fixes https://github.com/envoyproxy/envoy-setec/issues/188 Signed-off-by: Matt Klein <mklein@lyft.com> Signed-off-by: Yuchen Dai <silentdai@gmail.com> * fix wasm compile Signed-off-by: Yuchen Dai <silentdai@gmail.com> * fix typo Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* doc: version 1.12.7 Signed-off-by: Yuchen Dai <silentdai@gmail.com> * Preserve LWS from the middle of HTTP1 header values that requ… (envoyproxy#12319) * [http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process (envoyproxy#10886) Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice. Also various fix to allow build on clang-10 Signed-off-by: Antonio Vicente <avd@google.com> Signed-off-by: Yuchen Dai <silentdai@gmail.com> * http: header map security fixes for duplicate headers (#197) (#204) Previously header matching did not match on all headers for non-inline headers. This patch changes the default behavior to always logically match on all headers. Multiple individual headers will be logically concatenated with ',' similar to what is done with inline headers. This makes the behavior effectively consistent. This behavior can be temporary reverted by setting the runtime value "envoy.reloadable_features.header_match_on_all_headers" to "false". Targeted fixes have been additionally performed on the following extensions which make them consider all duplicate headers by default as a comma concatenated list: 1) Any extension using CEL matching on headers. 2) The header to metadata filter. 3) The JWT filter. 4) The Lua filter. Like primary header matching used in routing, RBAC, etc. this behavior can be disabled by setting the runtime value "envoy.reloadable_features.header_match_on_all_headers" to false. Finally, the setCopy() header map API previously only set the first header in the case of duplicate non-inline headers. setCopy() now behaves similiarly to the other set*() APIs and replaces all found headers with a single value. This may have had security implications in the extauth filter which uses this API. This behavior can be disabled by setting the runtime value "envoy.reloadable_features.http_set_copy_replace_all_headers" to false. Fixes https://github.com/envoyproxy/envoy-setec/issues/188 Signed-off-by: Matt Klein <mklein@lyft.com> Signed-off-by: Yuchen Dai <silentdai@gmail.com> * fix wasm compile Signed-off-by: Yuchen Dai <silentdai@gmail.com> * fix typo Signed-off-by: Yuchen Dai <silentdai@gmail.com> Co-authored-by: Yuchen Dai <silentdai@gmail.com>

lambdai requested a review from yanavlasov July 28, 2020 06:13

lambdai requested review from jmarantz and lizan as code owners July 28, 2020 06:13

lambdai commented Jul 28, 2020

View reviewed changes

lambdai changed the title ~~[http1] Preserve LWS from the middle of HTTP1 header values that requ…~~ [1.12] [http1] Preserve LWS from the middle of HTTP1 header values that requ… Jul 28, 2020

alyssawilk assigned antoniovicente Jul 28, 2020

antoniovicente reviewed Jul 28, 2020

View reviewed changes

source/common/http/header_map_impl.cc Outdated Show resolved Hide resolved

source/common/http/header_map_impl.cc Show resolved Hide resolved

test/common/http/header_map_impl_test.cc Show resolved Hide resolved

lambdai added 2 commits July 28, 2020 16:49

rtrim made accurate

dd4c6bb

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

More rtrim fix

b0132a1

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

lambdai force-pushed the 12.lws branch from 5d6d888 to b0132a1 Compare July 28, 2020 23:49

antoniovicente approved these changes Jul 29, 2020

View reviewed changes

lambdai merged commit a624dee into envoyproxy:release/v1.12 Jul 31, 2020

lambdai deleted the 12.lws branch July 31, 2020 21:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[1.12] [http1] Preserve LWS from the middle of HTTP1 header values that requ… #12319

[1.12] [http1] Preserve LWS from the middle of HTTP1 header values that requ… #12319

lambdai commented Jul 28, 2020

lambdai Jul 28, 2020

antoniovicente Jul 28, 2020

lambdai commented Jul 28, 2020

azure-pipelines bot commented Jul 28, 2020

lambdai commented Jul 28, 2020

lambdai commented Jul 28, 2020

antoniovicente left a comment

[1.12] [http1] Preserve LWS from the middle of HTTP1 header values that requ… #12319

[1.12] [http1] Preserve LWS from the middle of HTTP1 header values that requ… #12319

Conversation

lambdai commented Jul 28, 2020

lambdai Jul 28, 2020

Choose a reason for hiding this comment

antoniovicente Jul 28, 2020

Choose a reason for hiding this comment

lambdai commented Jul 28, 2020

azure-pipelines bot commented Jul 28, 2020

lambdai commented Jul 28, 2020

lambdai commented Jul 28, 2020

antoniovicente left a comment

Choose a reason for hiding this comment