Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Snapshots Status API to High Level Rest Client #31515

Merged
merged 17 commits into from
Jul 11, 2018
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
import org.elasticsearch.action.admin.cluster.storedscripts.DeleteStoredScriptRequest;
import org.elasticsearch.action.admin.cluster.storedscripts.GetStoredScriptRequest;
import org.elasticsearch.action.admin.cluster.snapshots.delete.DeleteSnapshotRequest;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotsStatusRequest;
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest;
import org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheRequest;
Expand Down Expand Up @@ -845,6 +846,20 @@ static Request verifyRepository(VerifyRepositoryRequest verifyRepositoryRequest)
return request;
}

static Request snapshotsStatus(SnapshotsStatusRequest snapshotsStatusRequest) {
String endpoint = new EndpointBuilder().addPathPartAsIs("_snapshot")
.addPathPart(snapshotsStatusRequest.repository())
.addCommaSeparatedPathParts(snapshotsStatusRequest.snapshots())
.addPathPartAsIs("_status")
.build();
Request request = new Request(HttpGet.METHOD_NAME, endpoint);

Params parameters = new Params(request);
parameters.withMasterTimeout(snapshotsStatusRequest.masterNodeTimeout());
parameters.putParam("ignore_unavailable", Boolean.toString(snapshotsStatusRequest.ignoreUnavailable()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might make sense to build out ignore_unavailable into a method now that its used in 2x places.

return request;
}

static Request deleteSnapshot(DeleteSnapshotRequest deleteSnapshotRequest) {
String endpoint = new EndpointBuilder().addPathPartAsIs("_snapshot")
.addPathPart(deleteSnapshotRequest.repository())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@
import org.elasticsearch.action.admin.cluster.repositories.put.PutRepositoryResponse;
import org.elasticsearch.action.admin.cluster.repositories.verify.VerifyRepositoryRequest;
import org.elasticsearch.action.admin.cluster.repositories.verify.VerifyRepositoryResponse;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotsStatusRequest;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotsStatusResponse;
import org.elasticsearch.action.admin.cluster.snapshots.delete.DeleteSnapshotRequest;
import org.elasticsearch.action.admin.cluster.snapshots.delete.DeleteSnapshotResponse;

Expand Down Expand Up @@ -164,6 +166,37 @@ public void verifyRepositoryAsync(VerifyRepositoryRequest verifyRepositoryReques
VerifyRepositoryResponse::fromXContent, listener, emptySet());
}

/**
* Gets the status of any snapshots currently in progress. If snapshot names are provided, this will return detailed status information
* for them even if they are not currently running.
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html"> Snapshot and Restore
* API on elastic.co</a>
* @param snapshotsStatusRequest the request
* @param options the request options (e.g. headers), use {@link RequestOptions#DEFAULT} if nothing needs to be customized
* @return the response
* @throws IOException in case there is a problem sending the request or parsing back the response
*/
public SnapshotsStatusResponse snapshotsStatus(SnapshotsStatusRequest snapshotsStatusRequest, RequestOptions options)
throws IOException {
return restHighLevelClient.performRequestAndParseEntity(snapshotsStatusRequest, RequestConverters::snapshotsStatus, options,
SnapshotsStatusResponse::fromXContent, emptySet());
}

/**
* Asynchronously gets the status of any snapshots currently in progress. If snapshot names are provided, this will return detailed
* status information for them even if they are not currently running.
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html"> Snapshot and Restore
* API on elastic.co</a>
* @param snapshotsStatusRequest the request
* @param options the request options (e.g. headers), use {@link RequestOptions#DEFAULT} if nothing needs to be customized
* @param listener the listener to be notified upon request completion
*/
public void snapshotsStatusAsync(SnapshotsStatusRequest snapshotsStatusRequest, RequestOptions options,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you rename the methods to status ? That's how it's defined in our REST spec.

ActionListener<SnapshotsStatusResponse> listener) {
restHighLevelClient.performRequestAsyncAndParseEntity(snapshotsStatusRequest, RequestConverters::snapshotsStatus, options,
SnapshotsStatusResponse::fromXContent, listener, emptySet());
}

/**
* Deletes a snapshot.
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html"> Snapshot and Restore
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
import org.elasticsearch.action.admin.cluster.snapshots.delete.DeleteSnapshotRequest;
import org.elasticsearch.action.admin.cluster.storedscripts.DeleteStoredScriptRequest;
import org.elasticsearch.action.admin.cluster.storedscripts.GetStoredScriptRequest;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotsStatusRequest;
import org.elasticsearch.action.admin.indices.alias.Alias;
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest;
import org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest.AliasActions;
Expand Down Expand Up @@ -1858,6 +1859,28 @@ public void testVerifyRepository() {
assertThat(expectedParams, equalTo(request.getParameters()));
}

public void testSnapshotsStatus() {
Map<String, String> expectedParams = new HashMap<>();
String repository = randomIndicesNames(1, 1)[0];
String[] snapshots = randomIndicesNames(1, 5);
StringBuilder snapshotNames = new StringBuilder(snapshots[0]);
for (int idx = 1; idx < snapshots.length; idx++) {
snapshotNames.append(",").append(snapshots[idx]);
}
boolean ignoreUnavailable = randomBoolean();
String endpoint = "/_snapshot/" + repository + "/" + snapshotNames.toString() + "/_status";

SnapshotsStatusRequest snapshotsStatusRequest = new SnapshotsStatusRequest(repository, snapshots);
setRandomMasterTimeout(snapshotsStatusRequest, expectedParams);
snapshotsStatusRequest.ignoreUnavailable(ignoreUnavailable);
expectedParams.put("ignore_unavailable", Boolean.toString(ignoreUnavailable));

Request request = RequestConverters.snapshotsStatus(snapshotsStatusRequest);
assertThat(request.getEndpoint(), equalTo(endpoint));
assertThat(request.getMethod(), equalTo(HttpGet.METHOD_NAME));
assertThat(request.getParameters(), equalTo(expectedParams));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test that the body is null?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 1a85d5b

}

public void testDeleteSnapshot() {
Map<String, String> expectedParams = new HashMap<>();
String repository = randomIndicesNames(1, 1)[0];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@

package org.elasticsearch.client;

import org.apache.http.entity.ContentType;
import org.apache.http.nio.entity.NStringEntity;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.action.admin.cluster.repositories.delete.DeleteRepositoryRequest;
import org.elasticsearch.action.admin.cluster.repositories.delete.DeleteRepositoryResponse;
Expand All @@ -28,6 +30,9 @@
import org.elasticsearch.action.admin.cluster.repositories.put.PutRepositoryResponse;
import org.elasticsearch.action.admin.cluster.repositories.verify.VerifyRepositoryRequest;
import org.elasticsearch.action.admin.cluster.repositories.verify.VerifyRepositoryResponse;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotsStatusRequest;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotsStatusResponse;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.action.admin.cluster.snapshots.delete.DeleteSnapshotRequest;
import org.elasticsearch.action.admin.cluster.snapshots.delete.DeleteSnapshotResponse;
import org.elasticsearch.common.xcontent.XContentType;
Expand All @@ -38,6 +43,7 @@
import java.util.Locale;

import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.is;

public class SnapshotIT extends ESRestHighLevelClientTestCase {

Expand All @@ -55,6 +61,12 @@ private Response createTestSnapshot(String repository, String snapshot) throws I
return highLevelClient().getLowLevelClient().performRequest(createSnapshot);
}

private Response createTestSnapshot(String repository, String snapshot, String index) throws IOException {
Request createSnapshot = new Request("put", String.format(Locale.ROOT, "_snapshot/%s/%s", repository, snapshot));
createSnapshot.addParameter("wait_for_completion", "true");
createSnapshot.setEntity(new NStringEntity("{\"indices\":\""+index+"\"}", ContentType.APPLICATION_JSON));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also use setJsonEntity if you wish, that's handy

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, neato

return highLevelClient().getLowLevelClient().performRequest(createSnapshot);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should rebase on 90d62e6. It included a createTestSnapshot method here as well for his tests.


public void testCreateRepository() throws IOException {
PutRepositoryResponse response = createTestRepository("test", FsRepository.TYPE, "{\"location\": \".\"}");
Expand Down Expand Up @@ -119,6 +131,26 @@ public void testVerifyRepository() throws IOException {
assertThat(response.getNodes().size(), equalTo(1));
}

public void testSnapshotsStatus() throws IOException {
PutRepositoryResponse putRepositoryResponse = createTestRepository("test", FsRepository.TYPE, "{\"location\": \".\"}");
assertTrue(putRepositoryResponse.isAcknowledged());

createIndex("test_index", Settings.EMPTY);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe variablize test_index so its reused in the next few places


Response snapshotResponse = createTestSnapshot("test", "snapshot", "test_index");
assertEquals(2, snapshotResponse.getHttpResponse().getStatusLine().getStatusCode() / 100);

SnapshotsStatusRequest request = new SnapshotsStatusRequest();
request.repository("test");
request.snapshots(new String[]{"snapshot"});
SnapshotsStatusResponse response = execute(request, highLevelClient().snapshot()::snapshotsStatus,
highLevelClient().snapshot()::snapshotsStatusAsync);
assertThat(response.getSnapshots().size(), equalTo(1));
assertThat(response.getSnapshots().get(0).getSnapshot().getRepository(), equalTo("test"));
assertThat(response.getSnapshots().get(0).getSnapshot().getSnapshotId().getName(), equalTo("snapshot"));
assertThat(response.getSnapshots().get(0).getIndices().containsKey("test_index"), is(true));
}

public void testDeleteSnapshot() throws IOException {
String repository = "test_repository";
String snapshot = "test_snapshot";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@
import org.elasticsearch.action.admin.cluster.repositories.verify.VerifyRepositoryResponse;
import org.elasticsearch.action.admin.cluster.snapshots.delete.DeleteSnapshotRequest;
import org.elasticsearch.action.admin.cluster.snapshots.delete.DeleteSnapshotResponse;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotStatus;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotsStatusRequest;
import org.elasticsearch.action.admin.cluster.snapshots.status.SnapshotsStatusResponse;
import org.elasticsearch.client.ESRestHighLevelClientTestCase;
import org.elasticsearch.client.Request;
import org.elasticsearch.client.RequestOptions;
Expand Down Expand Up @@ -73,7 +76,6 @@
public class SnapshotClientDocumentationIT extends ESRestHighLevelClientTestCase {

private static final String repositoryName = "test_repository";

private static final String snapshotName = "test_snapshot";

public void testSnapshotCreateRepository() throws IOException {
Expand Down Expand Up @@ -367,6 +369,74 @@ public void onFailure(Exception e) {
}
}

public void testSnapshotSnapshotsStatus() throws IOException {
RestHighLevelClient client = highLevelClient();
createTestRepositories();
createTestSnapshots();

// tag::snapshots-status-request
SnapshotsStatusRequest request = new SnapshotsStatusRequest();
// end::snapshots-status-request

// tag::snapshots-status-request-repository
request.repository(repositoryName); // <1>
// end::snapshots-status-request-repository
// tag::snapshots-status-request-snapshots
String [] snapshots = new String[] {snapshotName};
request.snapshots(snapshots); // <1>
// end::snapshots-status-request-snapshots
// tag::snapshots-status-request-ignoreUnavailable
request.ignoreUnavailable(true); // <1>
// end::snapshots-status-request-ignoreUnavailable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not actually rendered since the doc markup does not have them included in it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOL disregard clicked the wrong link!

// tag::snapshots-status-request-masterTimeout
request.masterNodeTimeout(TimeValue.timeValueMinutes(1)); // <1>
request.masterNodeTimeout("1m"); // <2>
// end::snapshots-status-request-masterTimeout

// tag::snapshots-status-execute
SnapshotsStatusResponse response = client.snapshot().snapshotsStatus(request, RequestOptions.DEFAULT);
// end::snapshots-status-execute

// tag::snapshots-status-response
List<SnapshotStatus> snapshotStatusesResponse = response.getSnapshots();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you go a bit deeper here and show users what they can retrieve from the response? We don't need to show everything but I would add what we'd expect users to generally do.

// end::snapshots-status-response
assertThat(snapshotStatusesResponse.size(), equalTo(1));
assertThat(snapshotStatusesResponse.get(0).getSnapshot().getRepository(), equalTo(repositoryName));
assertThat(snapshotStatusesResponse.get(0).getSnapshot().getSnapshotId().getName(), equalTo(snapshotName));
}

public void testSnapshotSnapshotsStatusAsync() throws InterruptedException {
RestHighLevelClient client = highLevelClient();
{
SnapshotsStatusRequest request = new SnapshotsStatusRequest();

// tag::snapshots-status-execute-listener
ActionListener<SnapshotsStatusResponse> listener =
new ActionListener<SnapshotsStatusResponse>() {
@Override
public void onResponse(SnapshotsStatusResponse snapshotsStatusResponse) {
// <1>
}

@Override
public void onFailure(Exception e) {
// <2>
}
};
// end::snapshots-status-execute-listener

// Replace the empty listener with a blocking listener in test
final CountDownLatch latch = new CountDownLatch(1);
listener = new LatchedActionListener<>(listener, latch);

// tag::snapshots-status-execute-async
client.snapshot().snapshotsStatusAsync(request, RequestOptions.DEFAULT, listener); // <1>
// end::snapshots-status-execute-async

assertTrue(latch.await(30L, TimeUnit.SECONDS));
}
}

public void testSnapshotDeleteSnapshot() throws IOException {
RestHighLevelClient client = highLevelClient();

Expand Down
92 changes: 92 additions & 0 deletions docs/java-rest/high-level/snapshot/snapshots_status.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
[[java-rest-high-snapshot-snapshots-status]]
=== Snapshots Status API

The Snapshots Status API allows to retrieve detailed information about snapshots in progress.

[[java-rest-high-snapshot-snapshots-status-request]]
==== Snapshots Status Request

A `SnapshotsStatusRequest`:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-request]
--------------------------------------------------

==== Optional Arguments
The following arguments can optionally be provided:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-request-repository]
--------------------------------------------------
<1> Sets the repository to check for snapshot statuses currently in progress

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-request-snapshots]
--------------------------------------------------
<1> The list of snapshot names to check the status of. If this is set, the status for the snapshots
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

repository and snapshots are not optional arguments, we will throw exception if they are not provided (see validate method)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this doesn't seem to match up with the current API documentation for this end point. If they are left off the API reports on any currently running snapshots. Not sure if that's a problem or a conscious decision to not support that aspect of the request.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems to be a conscious decision given that we expose the /_snapshot/_status endpoint alone, which is also documented in our REST spec.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed d9d35a6 for the required/optional arguments.

is returned, even if they are not currently in progress

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-request-ignoreUnavailable]
--------------------------------------------------
<1> The command will fail if some of the snapshots are unavailable. The `ignore_unavailable` flag
set to true will return all snapshots that are currently available.

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-request-masterTimeout]
--------------------------------------------------
<1> Timeout to connect to the master node as a `TimeValue`
<2> Timeout to connect to the master node as a `String`

[[java-rest-high-snapshot-snapshots-status-sync]]
==== Synchronous Execution

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-execute]
--------------------------------------------------

[[java-rest-high-snapshot-snapshots-status-async]]
==== Asynchronous Execution

The asynchronous execution of retrieving snapshot statuses requires both the
`SnapshotsStatusRequest` instance and an `ActionListener` instance to be
passed to the asynchronous method:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-execute-async]
--------------------------------------------------
<1> The `SnapshotsStatusRequest` to execute and the `ActionListener`
to use when the execution completes

The asynchronous method does not block and returns immediately. Once it is
completed the `ActionListener` is called back using the `onResponse` method
if the execution successfully completed or using the `onFailure` method if
it failed.

A typical listener for `SnapshotsStatusResponse` looks like:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-execute-listener]
--------------------------------------------------
<1> Called when the execution is successfully completed. The response is
provided as an argument
<2> Called in case of a failure. The raised exception is provided as an argument

[[java-rest-high-snapshot-snapshots-status-response]]
==== Snapshots Status Response

The returned `SnapshotsStatusResponse` allows to retrieve information about the
executed operation as follows:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SnapshotClientDocumentationIT.java[snapshots-status-response]
--------------------------------------------------
2 changes: 2 additions & 0 deletions docs/java-rest/high-level/supported-apis.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -136,12 +136,14 @@ The Java High Level REST Client supports the following Snapshot APIs:
* <<java-rest-high-snapshot-create-repository>>
* <<java-rest-high-snapshot-delete-repository>>
* <<java-rest-high-snapshot-verify-repository>>
* <<java-rest-high-snapshot-snapshots-status>>
* <<java-rest-high-snapshot-delete-snapshot>>

include::snapshot/get_repository.asciidoc[]
include::snapshot/create_repository.asciidoc[]
include::snapshot/delete_repository.asciidoc[]
include::snapshot/verify_repository.asciidoc[]
include::snapshot/snapshots_status.asciidoc[]
include::snapshot/delete_snapshot.asciidoc[]

== Tasks APIs
Expand Down
Loading