[FEATURE] Broaden SecureSettingsFactory to include http transports #12907

reta · 2024-03-25T19:11:28Z

Description

The further integration with transport plugins and security plugin revealed a couple of areas of improvements:

split SecureTransportSettingsProvider into SecureTransportSettingsProvider (native transport) and SecureHttpTransportSettingsProvider (HTTP transport): some transport plugins only provide one kind of transport, having fine grained control of the security settings only for particular transport would simplify the configuration
support TransportAdapterProvider<?> abstraction to alter the transport specific processing pipelines: the use cases here are primarily driven by security plugin which adds header verifier and decompressor, since every transport has pretty much independent implementation, it becomes the responsibility of the SecureTransportSettingsProvider and SecureHttpTransportSettingsProvider providers to supply the right one for a particular transport

Related Issues

Resolves #12903

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2024-03-25T19:34:41Z

Compatibility status:

Checks if related components are compatible with change fdba4b7

Incompatible components

Incompatible components: [https://github.com/opensearch-project/security.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/sql.git]

github-actions · 2024-03-25T19:56:04Z

✅ Gradle check result for efdc334: SUCCESS

codecov · 2024-03-25T19:57:42Z

Codecov Report

Attention: Patch coverage is 90.00000% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 71.41%. Comparing base (b15cb0c) to head (fdba4b7).
Report is 104 commits behind head on main.

Files	Patch %	Lines
...tp/netty4/ssl/SecureNetty4HttpServerTransport.java	88.88%	2 Missing and 1 partial ⚠️
...a/org/opensearch/common/network/NetworkModule.java	94.11%	0 Missing and 1 partial ⚠️
...earch/plugins/SecureTransportSettingsProvider.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12907      +/-   ##
============================================
- Coverage     71.42%   71.41%   -0.01%     
- Complexity    59978    60239     +261     
============================================
  Files          4985     5017      +32     
  Lines        282275   283902    +1627     
  Branches      40946    41149     +203     
============================================
+ Hits         201603   202737    +1134     
- Misses        63999    64292     +293     
- Partials      16673    16873     +200

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-03-26T21:30:17Z

✅ Gradle check result for cac3d99: SUCCESS

...ort-netty4/src/main/java/org/opensearch/http/netty4/ssl/SecureNetty4HttpServerTransport.java

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

github-actions · 2024-03-27T13:54:18Z

❕ Gradle check result for 6eb644e: UNSTABLE

TEST FAILURES:

      1 org.opensearch.cluster.allocation.ClusterRerouteIT.testDelayWithALargeAmountOfShards
      1 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

github-actions · 2024-03-27T17:36:27Z

✅ Gradle check result for 3632744: SUCCESS

reta · 2024-03-27T18:04:47Z

@peternied @cwperks would appreciate if you folks have time to look, pretty small change in scope, the accompanying OpenSearch security change is here: opensearch-project/security#4179

dblock

Looks pretty straightforward.

...ort-netty4/src/main/java/org/opensearch/http/netty4/ssl/SecureNetty4HttpServerTransport.java

stephen-crawford

Two small notes but looks good.

...ort-netty4/src/main/java/org/opensearch/http/netty4/ssl/SecureNetty4HttpServerTransport.java

server/src/main/java/org/opensearch/common/network/NetworkModule.java

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

reta · 2024-03-28T13:01:06Z

Two small notes but looks good.

Thanks @scrawfor99 , added more tests to cover configuration time exceptions

github-actions · 2024-03-28T13:46:11Z

✅ Gradle check result for fdba4b7: SUCCESS

opensearch-trigger-bot · 2024-03-28T14:34:39Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-12907-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 a103b84e157a12f8217e34fa41260de7ce13969f
# Push it to GitHub
git push --set-upstream origin backport/backport-12907-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-12907-to-2.x.

peternied · 2024-03-28T14:38:05Z

@reta Could you manually perform the backport?

…pensearch-project#12907) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> (cherry picked from commit a103b84)

…12907) (#12965) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> (cherry picked from commit a103b84)

…pensearch-project#12907) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

…pensearch-project#12907) Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

github-actions bot added enhancement Enhancement or improvement to existing feature or request Other v2.14.0 v3.0.0 Issues and PRs related to version 3.0.0 labels Mar 25, 2024

reta mentioned this pull request Mar 25, 2024

Switch to built-in security transports from core opensearch-project/security#4119

Merged

3 tasks

reta force-pushed the issue-12903 branch from efdc334 to cac3d99 Compare March 26, 2024 20:42

cwperks reviewed Mar 27, 2024

View reviewed changes

...ort-netty4/src/main/java/org/opensearch/http/netty4/ssl/SecureNetty4HttpServerTransport.java Outdated Show resolved Hide resolved

reta added 2 commits March 27, 2024 09:06

[FEATURE] Improve built-in secure transports support

baa8493

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Address code review comments

6eb644e

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

reta force-pushed the issue-12903 branch from cac3d99 to 6eb644e Compare March 27, 2024 13:06

Minor changes in APIs

3632744

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

github-actions bot added the security Anything security related label Mar 27, 2024

reta marked this pull request as ready for review March 27, 2024 17:59

reta requested review from peternied, abbashus, adnapibar, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, dbwiddis, dreamer-89 and gbbafna as code owners March 27, 2024 17:59

reta requested review from sachinpkale, saratvemulapalli, shwetathareja, sohami, tlfeng and VachaShah as code owners March 27, 2024 17:59

reta mentioned this pull request Mar 27, 2024

[FEATURE] Improve built-in secure transports support opensearch-project/security#4179

Merged

3 tasks

dblock approved these changes Mar 27, 2024

View reviewed changes

cwperks reviewed Mar 27, 2024

View reviewed changes

...ort-netty4/src/main/java/org/opensearch/http/netty4/ssl/SecureNetty4HttpServerTransport.java Show resolved Hide resolved

cwperks approved these changes Mar 27, 2024

View reviewed changes

stephen-crawford approved these changes Mar 27, 2024

View reviewed changes

...ort-netty4/src/main/java/org/opensearch/http/netty4/ssl/SecureNetty4HttpServerTransport.java Show resolved Hide resolved

server/src/main/java/org/opensearch/common/network/NetworkModule.java Show resolved Hide resolved

Address code review comments, added more tests

fdba4b7

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

peternied approved these changes Mar 28, 2024

View reviewed changes

peternied merged commit a103b84 into opensearch-project:main Mar 28, 2024
30 checks passed

peternied changed the title ~~[FEATURE] Improve built-in secure transports support~~ [FEATURE] Broaden SecureSettingsFactory to include http transports Mar 28, 2024

peternied added the backport 2.x Backport to 2.x branch label Mar 28, 2024

opensearch-trigger-bot bot added the backport-failed label Mar 28, 2024

reta added a commit to reta/OpenSearch that referenced this pull request Mar 28, 2024

[FEATURE] Broaden SecureSettingsFactory to include http transports (o…

25f064c

…pensearch-project#12907) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> (cherry picked from commit a103b84)

peternied pushed a commit that referenced this pull request Mar 28, 2024

[FEATURE] Broaden SecureSettingsFactory to include http transports (#…

48881de

…12907) (#12965) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> (cherry picked from commit a103b84)

reta mentioned this pull request Mar 28, 2024

[Backport 2.x] [FEATURE] Broaden SecureSettingsFactory to include http transports (#12907) #12965

Merged

bbarani added this to the Identity features for extensions milestone Apr 15, 2024

harshavamsi pushed a commit to harshavamsi/OpenSearch that referenced this pull request Apr 29, 2024

[FEATURE] Broaden SecureSettingsFactory to include http transports (o…

ad0068c

…pensearch-project#12907) Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

BrewTestBot mentioned this pull request May 14, 2024

opensearch 2.14.0 Homebrew/homebrew-core#171722

Merged

machenity mentioned this pull request Sep 2, 2024

[BUG] Continuous SSL exceptions post upgrade from 2.11 to 2.15 opensearch-project/security#4718

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] Broaden SecureSettingsFactory to include http transports #12907

[FEATURE] Broaden SecureSettingsFactory to include http transports #12907

reta commented Mar 25, 2024 •

edited

Loading

github-actions bot commented Mar 25, 2024 •

edited

Loading

github-actions bot commented Mar 25, 2024

codecov bot commented Mar 25, 2024 •

edited

Loading

github-actions bot commented Mar 26, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

reta commented Mar 27, 2024

dblock left a comment

stephen-crawford left a comment

reta commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

opensearch-trigger-bot bot commented Mar 28, 2024

peternied commented Mar 28, 2024

[FEATURE] Broaden SecureSettingsFactory to include http transports #12907

[FEATURE] Broaden SecureSettingsFactory to include http transports #12907

Conversation

reta commented Mar 25, 2024 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Mar 25, 2024 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Mar 25, 2024

codecov bot commented Mar 25, 2024 • edited Loading

Codecov Report

github-actions bot commented Mar 26, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

reta commented Mar 27, 2024

dblock left a comment

Choose a reason for hiding this comment

stephen-crawford left a comment

Choose a reason for hiding this comment

reta commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

opensearch-trigger-bot bot commented Mar 28, 2024

peternied commented Mar 28, 2024

reta commented Mar 25, 2024 •

edited

Loading

github-actions bot commented Mar 25, 2024 •

edited

Loading

codecov bot commented Mar 25, 2024 •

edited

Loading