Add a new node role 'search' which is dedicated to provide search capability #4689

tlfeng · 2022-10-06T00:23:01Z

Description

Add a new node role which is dedicated to provide search capability.
The role name is search.

Issues Resolved

Resolve #4652

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tianli Feng <ftianli@amazon.com>

github-actions · 2022-10-06T00:44:22Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3880/
CommitID: e51b0c9

github-actions · 2022-10-06T00:53:00Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3881/
CommitID: 4c65296

…archer Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # CHANGELOG.md

github-actions · 2022-10-06T05:09:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3890/
CommitID: d079bd3

github-actions · 2022-10-06T05:15:20Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3891/
CommitID: 017b202

Signed-off-by: Tianli Feng <ftianli@amazon.com>

github-actions · 2022-10-06T05:59:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3899/
CommitID: d02e5fb

Signed-off-by: Tianli Feng <ftianli@amazon.com>

…archer Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # CHANGELOG.md

Signed-off-by: Tianli Feng <ftianli@amazon.com>

kotwanikunal · 2022-10-10T16:46:32Z

server/src/main/java/org/opensearch/cluster/node/DiscoveryNodeRole.java

+            if (nodeVersion.onOrAfter(Version.V_2_4_0)) {
+                return this;
+            } else {
+                return DiscoveryNodeRole.DATA_ROLE;


Do we necessarily need a fallback?

Let me check what will be like when there is no fallback. So far I think without the fallback, the new role will be taken as "UnknownRole" when communicating with node in older versions, but I need to figure out the impact.

OpenSearch/server/src/main/java/org/opensearch/cluster/node/DiscoveryNodeRole.java

Line 314 in 00fb2b7

static class UnknownRole extends DiscoveryNodeRole {

I removed the fallback in the latest commit 9a05a90.
Looks like nothing wrong without the fallback for the new role.

I started 2 nodes to form a cluster.
In 3.0.0 node:

$ curl -XGET "http://localhost:9201/_cat/nodes?v&pretty" ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles cluster_manager name 127.0.0.1 19 17 0 0.09 0.15 0.09 s search - node-3.0 127.0.0.1 61 17 0 0.09 0.15 0.09 dm data,master * node-2.4

In 2.4.0 node:

$ curl -XGET "http://localhost:9200/_cat/nodes?v&pretty" ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles cluster_manager name 127.0.0.1 18 17 0 0.04 0.15 0.08 - search - node-3.0 127.0.0.1 60 17 0 0.04 0.15 0.08 dm data,master * node-2.4

github-actions · 2022-10-10T17:10:55Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/4281/
CommitID: 6f99a3c

github-actions · 2022-10-10T17:12:36Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/4282/
CommitID: f985034

…archer Signed-off-by: Tianli Feng <ftianli@amazon.com>

Signed-off-by: Tianli Feng <ftianli@amazon.com>

github-actions · 2022-10-11T05:33:33Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/4353/
CommitID: 9a05a90

codecov-commenter · 2022-10-11T05:35:46Z

Codecov Report

Merging #4689 (9a05a90) into main (9f415eb) will increase coverage by 0.21%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #4689      +/-   ##
============================================
+ Coverage     70.59%   70.81%   +0.21%     
- Complexity    57514    57582      +68     
============================================
  Files          4660     4660              
  Lines        276466   276470       +4     
  Branches      40285    40285              
============================================
+ Hits         195170   195780     +610     
+ Misses        65060    64421     -639     
- Partials      16236    16269      +33

Impacted Files	Coverage Δ
...rest/src/main/java/org/opensearch/client/Node.java	`92.72% <100.00%> (+1.98%)`	⬆️
...ava/org/opensearch/cluster/node/DiscoveryNode.java	`86.15% <100.00%> (+1.10%)`	⬆️
...org/opensearch/cluster/node/DiscoveryNodeRole.java	`88.05% <100.00%> (+1.90%)`	⬆️
...pensearch/client/cluster/RemoteConnectionInfo.java	`0.00% <0.00%> (-68.30%)`	⬇️
...a/org/opensearch/client/cluster/ProxyModeInfo.java	`0.00% <0.00%> (-60.00%)`	⬇️
...cluster/coordination/PendingClusterStateStats.java	`20.00% <0.00%> (-48.00%)`	⬇️
...r/src/main/java/org/opensearch/http/HttpStats.java	`21.05% <0.00%> (-47.37%)`	⬇️
...search/search/aggregations/pipeline/EwmaModel.java	`24.44% <0.00%> (-44.45%)`	⬇️
.../opensearch/client/cluster/RemoteInfoResponse.java	`61.53% <0.00%> (-38.47%)`	⬇️
...pensearch/action/ingest/DeletePipelineRequest.java	`31.25% <0.00%> (-37.50%)`	⬇️
... and 473 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

kotwanikunal

LGTM.

opensearch-trigger-bot · 2022-10-11T17:21:27Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-4689-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c1272c181e6bd96d1db725dfbbcd5ac8e6058502
# Push it to GitHub
git push --set-upstream origin backport/backport-4689-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4689-to-2.x.

…ability (opensearch-project#4689) Signed-off-by: Tianli Feng <ftianli@amazon.com>

…ability (#4689) (#4739) Signed-off-by: Tianli Feng <ftianli@amazon.com> Signed-off-by: Tianli Feng <ftianli@amazon.com>

…ability (opensearch-project#4689) Signed-off-by: Tianli Feng <ftianli@amazon.com>

Add a new node role - remote searcher

e51b0c9

Signed-off-by: Tianli Feng <ftianli@amazon.com>

tlfeng changed the title ~~Add a new node role - remote searcher~~ Add a new node role - remote_searcher Oct 6, 2022

tlfeng changed the title ~~Add a new node role - remote_searcher~~ Add a new node role which can provide search capability for remote shard Oct 6, 2022

Add changelog in 'Added' section

4c65296

Signed-off-by: Tianli Feng <ftianli@amazon.com>

tlfeng added enhancement Enhancement or improvement to existing feature or request feature New feature or request Indexing & Search v3.0.0 Issues and PRs related to version 3.0.0 and removed feature New feature or request labels Oct 6, 2022

Merge remote-tracking branch 'upstream/main' into node-role-remote-se…

017b202

…archer Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # CHANGELOG.md

tlfeng force-pushed the node-role-remote-searcher branch from d079bd3 to 017b202 Compare October 6, 2022 04:44

fix testNodeCountsWithDeprecatedMasterRole()

d02e5fb

Signed-off-by: Tianli Feng <ftianli@amazon.com>

Tianli Feng added 2 commits October 10, 2022 09:41

Change to role name from remote_searcher to search

0e50ba0

Signed-off-by: Tianli Feng <ftianli@amazon.com>

Merge remote-tracking branch 'upstream/main' into node-role-remote-se…

6f99a3c

…archer Signed-off-by: Tianli Feng <ftianli@amazon.com> # Conflicts: # CHANGELOG.md

tlfeng changed the title ~~Add a new node role which can provide search capability for remote shard~~ Add a new node role which is dedicated to provide search capability Oct 10, 2022

tlfeng marked this pull request as ready for review October 10, 2022 16:43

tlfeng requested review from a team and reta as code owners October 10, 2022 16:43

tlfeng changed the title ~~Add a new node role which is dedicated to provide search capability~~ Add a new node role 'search' which is dedicated to provide search capability Oct 10, 2022

Update the changelog to include the role name

f985034

Signed-off-by: Tianli Feng <ftianli@amazon.com>

tlfeng added backport 2.x Backport to 2.x branch v2.4.0 'Issues and PRs related to version v2.4.0' labels Oct 10, 2022

kotwanikunal reviewed Oct 10, 2022

View reviewed changes

Tianli Feng added 2 commits October 10, 2022 21:31

Merge remote-tracking branch 'upstream/main' into node-role-remote-se…

8899724

…archer Signed-off-by: Tianli Feng <ftianli@amazon.com>

Remove fallback of the new role for older versions

9a05a90

Signed-off-by: Tianli Feng <ftianli@amazon.com>

kotwanikunal approved these changes Oct 11, 2022

View reviewed changes

andrross approved these changes Oct 11, 2022

View reviewed changes

tlfeng merged commit c1272c1 into opensearch-project:main Oct 11, 2022

tlfeng deleted the node-role-remote-searcher branch October 11, 2022 17:21

tlfeng pushed a commit to tlfeng/OpenSearch that referenced this pull request Oct 11, 2022

Add a new node role 'search' which is dedicated to provide search cap…

39054c1

…ability (opensearch-project#4689) Signed-off-by: Tianli Feng <ftianli@amazon.com>

This was referenced Oct 11, 2022

[2.x] Add a new node role 'search' which is dedicated to provide search capability (#4689) #4739

Merged

[Searchable Snapshots] Add a new node role for remote search capabilities #4652

Closed

dblock pushed a commit that referenced this pull request Oct 11, 2022

Add a new node role 'search' which is dedicated to provide search cap…

747aa97

…ability (#4689) (#4739) Signed-off-by: Tianli Feng <ftianli@amazon.com> Signed-off-by: Tianli Feng <ftianli@amazon.com>

ashking94 pushed a commit to ashking94/OpenSearch that referenced this pull request Nov 7, 2022

Add a new node role 'search' which is dedicated to provide search cap…

14366a7

…ability (opensearch-project#4689) Signed-off-by: Tianli Feng <ftianli@amazon.com>

shwetathareja mentioned this pull request Apr 20, 2023

[RFC] Reader and Writer Separation in OpenSearch #7258

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a new node role 'search' which is dedicated to provide search capability #4689

Add a new node role 'search' which is dedicated to provide search capability #4689

tlfeng commented Oct 6, 2022 •

edited

Loading

github-actions bot commented Oct 6, 2022

github-actions bot commented Oct 6, 2022

github-actions bot commented Oct 6, 2022

github-actions bot commented Oct 6, 2022

github-actions bot commented Oct 6, 2022

kotwanikunal Oct 10, 2022

tlfeng Oct 10, 2022

tlfeng Oct 11, 2022 •

edited

Loading

github-actions bot commented Oct 10, 2022

github-actions bot commented Oct 10, 2022

github-actions bot commented Oct 11, 2022

codecov-commenter commented Oct 11, 2022

kotwanikunal left a comment

opensearch-trigger-bot bot commented Oct 11, 2022

Add a new node role 'search' which is dedicated to provide search capability #4689

Add a new node role 'search' which is dedicated to provide search capability #4689

Conversation

tlfeng commented Oct 6, 2022 • edited Loading

Description

Issues Resolved

Check List

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 6, 2022

Gradle Check (Jenkins) Run Completed with:

kotwanikunal Oct 10, 2022

Choose a reason for hiding this comment

tlfeng Oct 10, 2022

Choose a reason for hiding this comment

tlfeng Oct 11, 2022 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Oct 10, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 10, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 11, 2022

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Oct 11, 2022

Codecov Report

kotwanikunal left a comment

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Oct 11, 2022

tlfeng commented Oct 6, 2022 •

edited

Loading

tlfeng Oct 11, 2022 •

edited

Loading