Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add samples for PostgresSQL #1781

Merged
merged 16 commits into from
Apr 29, 2022
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion samples/snippets/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@
<version>3.0.0-M5</version>
<configuration>
<systemPropertyVariables>
<spanner.test.instance>spanner-testing-east1</spanner.test.instance>
<spanner.test.instance>pg-enabled-spanner</spanner.test.instance>
<spanner.test.instance.mr>spanner-mr-testing</spanner.test.instance.mr>
<spanner.test.instance.config>nam6</spanner.test.instance.config>
<spanner.test.key.location>us-east1</spanner.test.key.location>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

/**
* Example code for using Async query on Cloud Spanner and convert it to list.
*/
class AsyncQueryToListAsyncExample {
static class Album {
final long singerId;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ static void executeSqlWithCustomTimeoutAndRetrySettings(
.readWriteTransaction()
.run(transaction -> {
String sql =
"INSERT Singers (SingerId, FirstName, LastName)\n"
"INSERT INTO Singers (SingerId, FirstName, LastName)\n"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks fixing this syntax issue in old code.

+ "VALUES (20, 'George', 'Washington')";
long rowCount = transaction.executeUpdate(Statement.of(sql));
System.out.printf("%d record inserted.%n", rowCount);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
/*
* Copyright 2020 Google Inc.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 2022

(Please check everywhere else as well)

*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.spanner;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make more sense to place all the PG samples in a separate sub-package. So something like com.example.spanner.postgresql? (I'm open to both solutions, both the one that is chosen now by prefixing the samples with Pg, as well as moving them to a separate package, but I would like us to take an informed decision on it)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, will decide in today call with @jin-jj and decide the approach.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@olavloite : I think this was templated on https://github.com/googleapis/java-spanner/pull/1700/files.

IIRC, If we move to a sub-package - the class path would end up becoming something like samples/snippets/src/main/java/com/example/spanner/postgres/AsyncQueryToListAsyncExample.java, where the only gain probably will be readability of the class name, and some segregation from the regular GSQL samples.

The seggregation still happens in the current scheme of things, and looking for Spangres samples with an explicit pg in them sounds all right to me.

So we can keep things as is.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @olavloite and @ansh0l. So @Sivakumar-Searce we will keep the class name as-is.


// [START spanner_pg_async_query_to_list]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only this tag has pg_ in between?
Our all tag naming may require update. I will update it here later what is our decision about the tag name for pg code samples.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For package name, we will use spanner_postgresql_*.

Pls fix this one and also apply the same pattern to all other tags.

import com.google.api.core.ApiFuture;
import com.google.cloud.spanner.AsyncResultSet;
import com.google.cloud.spanner.DatabaseClient;
import com.google.cloud.spanner.DatabaseId;
import com.google.cloud.spanner.Spanner;
import com.google.cloud.spanner.SpannerOptions;
import com.google.cloud.spanner.Statement;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class PgAsyncQueryToListAsyncExample {
static class Album {
final long singerId;
final long albumId;
final String albumTitle;

Album(long singerId, long albumId, String albumTitle) {
this.singerId = singerId;
this.albumId = albumId;
this.albumTitle = albumTitle;
}
}

static void asyncQueryToList() throws InterruptedException, ExecutionException, TimeoutException {
// TODO(developer): Replace these variables before running the sample.
String projectId = "my-project";
String instanceId = "my-instance";
String databaseId = "my-database";

try (Spanner spanner =
SpannerOptions.newBuilder().setProjectId(projectId).build().getService()) {
DatabaseClient client =
spanner.getDatabaseClient(DatabaseId.of(projectId, instanceId, databaseId));
asyncQueryToList(client);
}
}

// Execute a query asynchronously and transform the result to a list.
static void asyncQueryToList(DatabaseClient client)
throws InterruptedException, ExecutionException, TimeoutException {
ExecutorService executor = Executors.newSingleThreadExecutor();
ApiFuture<? extends List<AsyncQueryToListAsyncExample.Album>> albums;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ApiFuture<? extends List<AsyncQueryToListAsyncExample.Album>> albums;
ApiFuture<? extends List<Album>> albums;

try (AsyncResultSet resultSet =
client
.singleUse()
.executeQueryAsync(Statement.of("SELECT singerid as \"SingerId\", "
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same as the comments, I am thinking renaming all column name in DDL from SingerId to singer_id, which is more pg friendly. I will circle back later to update what should we do.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will keep the code as-is. No change required.

+ "albumid as \"AlbumId\", albumtitle as \"AlbumTitle\" "
+ "FROM Albums"))) {
// Convert the result set to a list of Albums asynchronously.
albums =
resultSet.toListAsync(
reader -> {
return new AsyncQueryToListAsyncExample.Album(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return new AsyncQueryToListAsyncExample.Album(
return new Album(

reader.getLong("SingerId"),
reader.getLong("AlbumId"),
reader.getString("AlbumTitle"));
},
executor);
}

for (AsyncQueryToListAsyncExample.Album album : albums.get(30L, TimeUnit.SECONDS)) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for (AsyncQueryToListAsyncExample.Album album : albums.get(30L, TimeUnit.SECONDS)) {
for (Album album : albums.get(30L, TimeUnit.SECONDS)) {

System.out.printf("%d %d %s%n", album.singerId, album.albumId, album.albumTitle);
}
executor.shutdown();
}
}
//[END spanner_pg_async_query_to_list]
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
/*
* Copyright 2020 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.spanner;

//[START spanner_async_read_write_transaction]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tag is the same as the Spanner equivalent sample. Please make sure that this (and other samples) have a unique tag.

import com.google.api.core.ApiFunction;
import com.google.api.core.ApiFuture;
import com.google.api.core.ApiFutures;
import com.google.cloud.spanner.AsyncRunner;
import com.google.cloud.spanner.AsyncRunner.AsyncWork;
import com.google.cloud.spanner.DatabaseClient;
import com.google.cloud.spanner.DatabaseId;
import com.google.cloud.spanner.Key;
import com.google.cloud.spanner.Spanner;
import com.google.cloud.spanner.SpannerExceptionFactory;
import com.google.cloud.spanner.SpannerOptions;
import com.google.cloud.spanner.Statement;
import com.google.cloud.spanner.Struct;
import com.google.cloud.spanner.TransactionContext;
import com.google.common.collect.ImmutableList;
import com.google.common.util.concurrent.MoreExecutors;
import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
import java.util.Arrays;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

class PgAsyncRunnerExample {

static void asyncRunner() throws InterruptedException, ExecutionException, TimeoutException {
// TODO(developer): Replace these variables before running the sample.
String projectId = "my-project";
String instanceId = "my-instance";
String databaseId = "my-database";

try (Spanner spanner =
SpannerOptions.newBuilder().setProjectId(projectId).build().getService()) {
DatabaseClient client =
spanner.getDatabaseClient(DatabaseId.of(projectId, instanceId, databaseId));
asyncRunner(client);
}
}

// Execute a read/write transaction asynchronously.
static void asyncRunner(DatabaseClient client)
throws InterruptedException, ExecutionException, TimeoutException {
ExecutorService executor = Executors.newSingleThreadExecutor();

// Create an async transaction runner.
AsyncRunner runner = client.runAsync();
// The transaction returns the total number of rows that were updated as a future array of
// longs.
ApiFuture<long[]> rowCounts =
runner.runAsync(
txn -> {
// Transfer marketing budget from one album to another. We do it in a
// transaction to ensure that the transfer is atomic.
ApiFuture<Struct> album1BudgetFut =
txn.readRowAsync("Albums", Key.of(1, 1), ImmutableList.of("MarketingBudget"));
ApiFuture<Struct> album2BudgetFut =
txn.readRowAsync("Albums", Key.of(2, 2), ImmutableList.of("MarketingBudget"));

try {
// Transaction will only be committed if this condition still holds at the
// time of commit. Otherwise it will be aborted and the AsyncWork will be
// rerun by the client library.
long transfer = 200_000;
if (album2BudgetFut.get().getLong(0) >= transfer) {
long album1Budget = album1BudgetFut.get().getLong(0);
long album2Budget = album2BudgetFut.get().getLong(0);

album1Budget += transfer;
album2Budget -= transfer;
Statement updateStatement1 =
Statement.newBuilder(
"UPDATE Albums "
+ "SET MarketingBudget = $1 "
+ "WHERE SingerId = 1 and AlbumId = 1")
.bind("p1")
.to(album1Budget)
.build();
Statement updateStatement2 =
Statement.newBuilder(
"UPDATE Albums "
+ "SET MarketingBudget = $1 "
+ "WHERE SingerId = 2 and AlbumId = 2")
.bind("p1")
.to(album2Budget)
.build();
return txn.batchUpdateAsync(
ImmutableList.of(updateStatement1, updateStatement2));
} else {
return ApiFutures.immediateFuture(new long[] {0L, 0L});
}
} catch (ExecutionException e) {
throw SpannerExceptionFactory.newSpannerException(e.getCause());
} catch (InterruptedException e) {
throw SpannerExceptionFactory.propagateInterrupt(e);
}
},
executor);

ApiFuture<Long> totalUpdateCount =
ApiFutures.transform(
rowCounts,
new ApiFunction<long[], Long>() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this expression can be simplified to a lambda expression now that we require Java 8 as a minimum.

@SuppressFBWarnings("UVA_USE_VAR_ARGS")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this suppress warnings annotation still needed? (I understand that it is probably copy-pasted from the original, but I'm not sure we still need it)

@Override
public Long apply(long[] input) {
return Arrays.stream(input).sum();
}
},
MoreExecutors.directExecutor());
System.out.printf("%d records updated.%n", totalUpdateCount.get(30L, TimeUnit.SECONDS));
executor.shutdown();
}
}
//[END spanner_async_read_write_transaction]
Loading