-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow customization of netty channel handles before and during decompression #10261
Allow customization of netty channel handles before and during decompression #10261
Conversation
…tep in the pipeline to analyze request headers Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change ddaca29 Incompatible componentsSkipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git] |
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
@@ Coverage Diff @@
## main #10261 +/- ##
============================================
+ Coverage 71.16% 71.21% +0.04%
- Complexity 58297 58397 +100
============================================
Files 4830 4843 +13
Lines 274903 275193 +290
Branches 40059 40068 +9
============================================
+ Hits 195637 195980 +343
+ Misses 62882 62814 -68
- Partials 16384 16399 +15
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for getting this out @cwperks
I'm seeing a lot of changes - whereas I'd prefer we slim this down to only the createHeaderVerifier
. It looks like there were many unrelated changes that had to be propped up to support generating requests objects in a way they are usable by the existing RestHandler wrapper which reuse many aspects of the SecurityPlugins current logic.
I'm going to spend some cycles wrapping my head around this change and come up with some prototypes for other suggestions. This is definitely on the right path - maybe we can figure out how to adjust the scope of changes.
...les/transport-netty4/src/main/java/org/opensearch/http/netty4/Netty4HttpServerTransport.java
Show resolved
Hide resolved
...les/transport-netty4/src/main/java/org/opensearch/http/netty4/Netty4HttpServerTransport.java
Outdated
Show resolved
Hide resolved
plugins/transport-nio/src/main/java/org/opensearch/http/nio/HttpReadWriteHandler.java
Outdated
Show resolved
Hide resolved
...les/transport-netty4/src/main/java/org/opensearch/http/netty4/Netty4HttpServerTransport.java
Show resolved
Hide resolved
modules/transport-netty4/src/main/java/org/opensearch/http/netty4/Netty4HttpRequestHandler.java
Outdated
Show resolved
Hide resolved
modules/transport-netty4/src/main/java/org/opensearch/http/netty4/Netty4HttpRequestHandler.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
393fdb5
to
fbacd86
Compare
Signed-off-by: Peter Nied <petern@amazon.com>
fbacd86
to
aec43e9
Compare
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nicely done, thanks for the test coverage and the enhancement to the request pipeline.
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-10261-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 dad525aefaab01b8452f9db8d7fba70a4d3b5cc8
# Push it to GitHub
git push --set-upstream origin backport/backport-10261-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x Then, create a pull request where the |
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.11 2.11
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.11
# Create a new branch
git switch --create backport/backport-10261-to-2.11
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 dad525aefaab01b8452f9db8d7fba70a4d3b5cc8
# Push it to GitHub
git push --set-upstream origin backport/backport-10261-to-2.11
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.11 Then, create a pull request where the |
…nd during decompression (opensearch-project#10261) Backport of opensearch-project#10261 from dad525a Signed-off-by: Peter Nied <petern@amazon.com>
…d during decompression (opensearch-project#10261) Backport of opensearch-project#10261 from dad525a Signed-off-by: Peter Nied <petern@amazon.com>
Backport of #10261 to 2.x. Signed-off-by: Craig Perkins <cwperx@amazon.com>
…d during decompression (opensearch-project#10261) Signed-off-by: Peter Nied <petern@amazon.com>
…d during decompression (#10261) (#11086) * [Backport 1.3] Allow customization of netty channel handles before and during decompression (#10261) Signed-off-by: Peter Nied <petern@amazon.com> * Fix test cases issues by switching to marked instead of blocked workflow Signed-off-by: Peter Nied <petern@amazon.com> * Fix spotless issues Signed-off-by: Peter Nied <petern@amazon.com> --------- Signed-off-by: Peter Nied <petern@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com>
…ression (opensearch-project#10261) Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Description
Companion PR: opensearch-project/security#3418
In some cases, a request is considered invalid from its headers and its unnecessary to inspect the body of the request. For compressed requests, decompressed bodies can sometimes be quite large so its a benefit to skip decompression if the body is only being discarded.
This PR allows a plugin to extend the pipeline and determine whether decompression can be skipped in the pipeline if the step is unnecessary.
Related Issues
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.