Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add APIs (GET/PUT) to decommission awareness attribute #4261

Merged
merged 35 commits into from
Oct 2, 2022
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
e3dcf4a
Add APIs (GET/PUT) to decommission awareness attribute
imRishN Aug 18, 2022
1dfc3cd
Add Get Decommission API
imRishN Aug 18, 2022
f5f676a
Adding UT
imRishN Aug 22, 2022
4c8b65b
Add UTs for Rest Layer
imRishN Aug 22, 2022
8c2ddbc
Fix
imRishN Aug 22, 2022
c7e4bcc
Applying spotless check
imRishN Aug 22, 2022
e2b38cb
Add temp URL
imRishN Aug 22, 2022
1c9fe9f
Bug fix
imRishN Aug 22, 2022
17f2548
Fix
imRishN Aug 23, 2022
f9bfff9
Fixes
imRishN Aug 25, 2022
6ba62c2
Fix spotless and precommit checks
imRishN Aug 25, 2022
6f2f3d3
Add java docs and fix test
imRishN Sep 1, 2022
81748c1
Empty-Commit
imRishN Sep 1, 2022
f4ec908
Rename PutDecommissionAction to DecommissionAction
imRishN Sep 2, 2022
0c08d98
Refactor GetDecommissionAction to GetDecommissionStateAction
imRishN Sep 2, 2022
a1289a2
Fix spotless check
imRishN Sep 2, 2022
93ae7c2
fix empty response
imRishN Sep 8, 2022
c9b0b48
minor fix
imRishN Sep 22, 2022
00270f0
Update changelog
imRishN Sep 25, 2022
7152932
Fix API PR after service merge
imRishN Sep 25, 2022
9a9f5af
Fix spotless check
imRishN Sep 25, 2022
1c845db
Fix parsing issue
imRishN Sep 25, 2022
40ffde7
Handle master abdication more gracefully
imRishN Sep 26, 2022
da2ea43
Remove timeout and refactoring
imRishN Sep 26, 2022
a206a08
Update join executor tests
imRishN Sep 26, 2022
1873519
Fix spotless check
imRishN Sep 26, 2022
c0d5de7
Empty-Commit
imRishN Sep 26, 2022
fbe0e7d
Fix create decommission request
imRishN Sep 27, 2022
ef33115
Merge remote-tracking branch 'upstream/main' into decommission-api/pr
imRishN Sep 27, 2022
908fe08
Merge remote-tracking branch 'upstream/main' into decommission-api/pr
imRishN Sep 28, 2022
fc6dd88
Fix Response & Fix spotless check
imRishN Sep 29, 2022
14f4ce0
Use String utility
imRishN Sep 29, 2022
4aa3790
Empty-Commit
imRishN Sep 30, 2022
534a3de
Merge branch 'main' into decommission-api/pr
imRishN Sep 30, 2022
80449a8
Merge branch 'main' into decommission-api/pr
Bukhtawar Oct 2, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [Remote Store] Change behaviour in replica recovery for remote translog enabled indices ([#4318](https://github.com/opensearch-project/OpenSearch/pull/4318))
- Unmute test RelocationIT.testRelocationWhileIndexingRandom ([#4580](https://github.com/opensearch-project/OpenSearch/pull/4580))
- Add DecommissionService and helper to execute awareness attribute decommissioning ([#4084](https://github.com/opensearch-project/OpenSearch/pull/4084))

- Add APIs (GET/PUT) to decommission awareness attribute ([#4261](https://github.com/opensearch-project/OpenSearch/pull/4261))

### Deprecated

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -887,7 +887,9 @@ public void testApiNamingConventions() throws Exception {
"nodes.usage",
"nodes.reload_secure_settings",
"search_shards",
"remote_store.restore", };
"remote_store.restore",
"cluster.put_decommission_awareness",
"cluster.get_decommission_awareness", };
List<String> booleanReturnMethods = Arrays.asList("security.enable_user", "security.disable_user", "security.change_password");
Set<String> deprecatedMethods = new HashSet<>();
deprecatedMethods.add("indices.force_merge");
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"cluster.get_decommission_awareness": {
"documentation": {
"url": "https://opensearch.org/docs/latest/opensearch/rest-api/decommission/",
"description": "Get details and status of decommissioned attribute"
},
"stability": "experimental",
imRishN marked this conversation as resolved.
Show resolved Hide resolved
imRishN marked this conversation as resolved.
Show resolved Hide resolved
"url": {
"paths": [
{
"path": "/_cluster/decommission/awareness/_status",
"methods": [
"GET"
]
}
]
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"cluster.put_decommission_awareness": {
"documentation": {
"url": "https://opensearch.org/docs/latest/opensearch/rest-api/decommission/",
"description": "Decommissions an awareness attribute"
},
"stability": "experimental",
"url": {
"paths": [
{
"path": "/_cluster/decommission/awareness/{awareness_attribute_name}/{awareness_attribute_value}",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decommissioning is only supported on awareness attributes, what about decommissioning by nodeId, nodeName etc.?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, we are supporting decommissioning an awareness attribute. Incrementally, we can have this feature for decommissioning a node

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we just keep the API generic and pass awareness as query param instead?

Copy link
Member Author

@imRishN imRishN Sep 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we make awareness query param then putting attribute type and attribute value real confusing.
As part of discussion here in the RFC - #3639 we discussed to have this as /_cluster/decommission/awareness/{awareness_attribute_name}/{awareness_attribute_value} which is also in consistence with weights API

Copy link
Collaborator

@Bukhtawar Bukhtawar Sep 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do two GET APIs

  1. GET /_cluster/decommission/awareness/{awareness_attribute_name}/_status
  2. GET /_cluster/decommission/awareness/_status

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened issue to add another route - #4640

"methods": [
"PUT"
],
"parts": {
"awareness_attribute_name": {
"type": "string",
"description": "Awareness attribute name"
},
"awareness_attribute_value": {
"type": "string",
"description": "Awareness attribute value"
}
}
}
]
}
}
}
12 changes: 12 additions & 0 deletions server/src/main/java/org/opensearch/action/ActionModule.java
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@
import org.opensearch.action.admin.cluster.configuration.ClearVotingConfigExclusionsAction;
import org.opensearch.action.admin.cluster.configuration.TransportAddVotingConfigExclusionsAction;
import org.opensearch.action.admin.cluster.configuration.TransportClearVotingConfigExclusionsAction;
import org.opensearch.action.admin.cluster.decommission.awareness.get.GetDecommissionStateAction;
import org.opensearch.action.admin.cluster.decommission.awareness.get.TransportGetDecommissionStateAction;
import org.opensearch.action.admin.cluster.decommission.awareness.put.DecommissionAction;
import org.opensearch.action.admin.cluster.decommission.awareness.put.TransportDecommissionAction;
import org.opensearch.action.admin.cluster.health.ClusterHealthAction;
import org.opensearch.action.admin.cluster.health.TransportClusterHealthAction;
import org.opensearch.action.admin.cluster.node.hotthreads.NodesHotThreadsAction;
Expand Down Expand Up @@ -305,6 +309,7 @@
import org.opensearch.rest.action.admin.cluster.RestDeleteRepositoryAction;
import org.opensearch.rest.action.admin.cluster.RestDeleteSnapshotAction;
import org.opensearch.rest.action.admin.cluster.RestDeleteStoredScriptAction;
import org.opensearch.rest.action.admin.cluster.RestGetDecommissionStateAction;
import org.opensearch.rest.action.admin.cluster.RestGetRepositoriesAction;
import org.opensearch.rest.action.admin.cluster.RestGetScriptContextAction;
import org.opensearch.rest.action.admin.cluster.RestGetScriptLanguageAction;
Expand All @@ -317,6 +322,7 @@
import org.opensearch.rest.action.admin.cluster.RestNodesStatsAction;
import org.opensearch.rest.action.admin.cluster.RestNodesUsageAction;
import org.opensearch.rest.action.admin.cluster.RestPendingClusterTasksAction;
import org.opensearch.rest.action.admin.cluster.RestDecommissionAction;
import org.opensearch.rest.action.admin.cluster.RestPutRepositoryAction;
import org.opensearch.rest.action.admin.cluster.RestPutStoredScriptAction;
import org.opensearch.rest.action.admin.cluster.RestReloadSecureSettingsAction;
Expand Down Expand Up @@ -683,6 +689,10 @@ public <Request extends ActionRequest, Response extends ActionResponse> void reg
// Remote Store
actions.register(RestoreRemoteStoreAction.INSTANCE, TransportRestoreRemoteStoreAction.class);

// Decommission actions
actions.register(DecommissionAction.INSTANCE, TransportDecommissionAction.class);
actions.register(GetDecommissionStateAction.INSTANCE, TransportGetDecommissionStateAction.class);

return unmodifiableMap(actions.getRegistry());
}

Expand Down Expand Up @@ -873,6 +883,8 @@ public void initRestHandlers(Supplier<DiscoveryNodes> nodesInCluster) {
}
}
registerHandler.accept(new RestCatAction(catActions));
registerHandler.accept(new RestDecommissionAction());
registerHandler.accept(new RestGetDecommissionStateAction());

// Remote Store APIs
if (FeatureFlags.isEnabled(FeatureFlags.REMOTE_STORE)) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.action.admin.cluster.decommission.awareness.get;

import org.opensearch.action.ActionType;

/**
* Get decommission action
*
* @opensearch.internal
*/
public class GetDecommissionStateAction extends ActionType<GetDecommissionStateResponse> {

public static final GetDecommissionStateAction INSTANCE = new GetDecommissionStateAction();
public static final String NAME = "cluster:admin/decommission/awareness/get";

private GetDecommissionStateAction() {
super(NAME, GetDecommissionStateResponse::new);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.action.admin.cluster.decommission.awareness.get;

import org.opensearch.action.ActionRequestValidationException;
import org.opensearch.action.support.clustermanager.ClusterManagerNodeReadRequest;
import org.opensearch.common.io.stream.StreamInput;
import org.opensearch.common.io.stream.StreamOutput;

import java.io.IOException;

/**
* Get Decommissioned attribute request
*
* @opensearch.internal
*/
public class GetDecommissionStateRequest extends ClusterManagerNodeReadRequest<GetDecommissionStateRequest> {

public GetDecommissionStateRequest() {}

public GetDecommissionStateRequest(StreamInput in) throws IOException {
super(in);
}

@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
}

@Override
public ActionRequestValidationException validate() {
return null;
}
}
imRishN marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.action.admin.cluster.decommission.awareness.get;

import org.opensearch.action.support.clustermanager.ClusterManagerNodeReadOperationRequestBuilder;
import org.opensearch.client.OpenSearchClient;

/**
* Get decommission request builder
*
* @opensearch.internal
*/
public class GetDecommissionStateRequestBuilder extends ClusterManagerNodeReadOperationRequestBuilder<
GetDecommissionStateRequest,
GetDecommissionStateResponse,
GetDecommissionStateRequestBuilder> {

/**
* Creates new get decommissioned attributes request builder
*/
public GetDecommissionStateRequestBuilder(OpenSearchClient client, GetDecommissionStateAction action) {
super(client, action, new GetDecommissionStateRequest());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.action.admin.cluster.decommission.awareness.get;

import org.opensearch.OpenSearchParseException;
import org.opensearch.action.ActionResponse;
import org.opensearch.cluster.decommission.DecommissionAttribute;
import org.opensearch.cluster.decommission.DecommissionStatus;
import org.opensearch.common.io.stream.StreamInput;
import org.opensearch.common.io.stream.StreamOutput;
import org.opensearch.common.xcontent.ToXContentObject;
import org.opensearch.common.xcontent.XContentBuilder;
import org.opensearch.common.xcontent.XContentParser;

import java.io.IOException;
import java.util.Locale;
import java.util.Objects;

import static org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken;

/**
* Response for decommission status
*
* @opensearch.internal
*/
public class GetDecommissionStateResponse extends ActionResponse implements ToXContentObject {

private final DecommissionAttribute decommissionedAttribute;
private final DecommissionStatus status;

GetDecommissionStateResponse() {
this(null, null);
}

GetDecommissionStateResponse(DecommissionAttribute decommissionedAttribute, DecommissionStatus status) {
this.decommissionedAttribute = decommissionedAttribute;
this.status = status;
}

GetDecommissionStateResponse(StreamInput in) throws IOException {
if (in.readBoolean()) {
this.decommissionedAttribute = new DecommissionAttribute(in);
} else {
this.decommissionedAttribute = null;
}
imRishN marked this conversation as resolved.
Show resolved Hide resolved
if (in.readBoolean()) {
this.status = DecommissionStatus.fromString(in.readString());
} else {
this.status = null;
}
}

@Override
public void writeTo(StreamOutput out) throws IOException {
boolean isNotNullDecommissionAttribute = this.decommissionedAttribute != null;
boolean isNotNullStatus = this.status != null;
imRishN marked this conversation as resolved.
Show resolved Hide resolved
out.writeBoolean(isNotNullDecommissionAttribute);
if (isNotNullDecommissionAttribute) {
decommissionedAttribute.writeTo(out);
}
out.writeBoolean(isNotNullStatus);
if (isNotNullStatus) {
out.writeString(status.status());
}
}
imRishN marked this conversation as resolved.
Show resolved Hide resolved

public DecommissionAttribute getDecommissionedAttribute() {
return decommissionedAttribute;
}

public DecommissionStatus getDecommissionStatus() {
return status;
}

@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
builder.startObject("awareness");
if (decommissionedAttribute != null) {
builder.field(decommissionedAttribute.attributeName(), decommissionedAttribute.attributeValue());
}
builder.endObject();
if (status != null) {
builder.field("status", status);
}
builder.endObject();
return builder;
}

public static GetDecommissionStateResponse fromXContent(XContentParser parser) throws IOException {
ensureExpectedToken(XContentParser.Token.START_OBJECT, parser.nextToken(), parser);
String attributeType = "awareness";
XContentParser.Token token;
DecommissionAttribute decommissionAttribute = null;
DecommissionStatus status = null;
while ((token = parser.nextToken()) != XContentParser.Token.END_OBJECT) {
if (token == XContentParser.Token.FIELD_NAME) {
String currentFieldName = parser.currentName();
if (attributeType.equals(currentFieldName)) {
if (parser.nextToken() != XContentParser.Token.START_OBJECT) {
throw new OpenSearchParseException(
"failed to parse decommission attribute type [{}], expected object",
attributeType
);
}
token = parser.nextToken();
if (token != XContentParser.Token.END_OBJECT) {
if (token == XContentParser.Token.FIELD_NAME) {
String fieldName = parser.currentName();
String value;
token = parser.nextToken();
if (token == XContentParser.Token.VALUE_STRING) {
value = parser.text();
} else {
throw new OpenSearchParseException(
"failed to parse attribute [{}], expected string for attribute value",
fieldName
);
}
decommissionAttribute = new DecommissionAttribute(fieldName, value);
parser.nextToken();
} else {
throw new OpenSearchParseException("failed to parse attribute type [{}], unexpected type", attributeType);
}
} else {
throw new OpenSearchParseException("failed to parse attribute type [{}]", attributeType);
}
} else if ("status".equals(currentFieldName)) {
if (parser.nextToken() != XContentParser.Token.VALUE_STRING) {
throw new OpenSearchParseException(
"failed to parse status of decommissioning, expected string but found unknown type"
);
}
status = DecommissionStatus.fromString(parser.text().toLowerCase(Locale.ROOT));
} else {
throw new OpenSearchParseException(
"unknown field found [{}], failed to parse the decommission attribute",
currentFieldName
);
}
}
}
return new GetDecommissionStateResponse(decommissionAttribute, status);
}

@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
GetDecommissionStateResponse that = (GetDecommissionStateResponse) o;
return decommissionedAttribute.equals(that.decommissionedAttribute) && status == that.status;
}

@Override
public int hashCode() {
return Objects.hash(decommissionedAttribute, status);
}
}
Loading