Add new Promisify Anything (AWS Athena edition) example #57
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pcholakov
force-pushed
the
promisify-anything
branch
from
d933e5d
to
062ac42
Compare
- bump AWS SDK version - update instructions with dedicated AWS profile - don't retry 404 from client - indicates deployment problem
pcholakov
commented
Feb 15, 2024
Comment on lines
13
to
29
| const executionId = await ctx.sideEffect(async () => { | ||
| try { | ||
| const startQueryResult = await client.send( | ||
| new athena.StartQueryExecutionCommand({ | ||
| QueryString: query ?? QUERY_SQL, | ||
| WorkGroup: "demo-workgroup", | ||
| ClientRequestToken: requestId, | ||
| }), | ||
| ); | ||
| return startQueryResult.QueryExecutionId as string; | ||
| } catch (error) { | ||
| if (error instanceof athena.AthenaServiceException) | ||
| if (error.$fault === "client") throw new restate.TerminalError(error.message); | ||
|
|
||
| throw error; // rethrow other exceptions -- side effect retry policy will handle them | ||
| } | ||
| }); |
There was a problem hiding this comment.
@tillrohrmann WDYT about this alternative:
const startQueryOutput = await ctx.sideEffect(
stopOnError(
(e) => (e as { $fault?: string }).$fault === "client",
async () =>
await client.send(
new athena.StartQueryExecutionCommand({
QueryString: query ?? QUERY_SQL,
WorkGroup: "demo-workgroup",
ClientRequestToken: requestId,
}),
),
),
);
There was a problem hiding this comment.
I think this is a good improvement which makes things easier to understand :-)
pcholakov
force-pushed
the
promisify-anything
branch
from
9539e62
to
31f1fd9
Compare
pcholakov
commented
Feb 16, 2024
Comment on lines
+28
to
+30
| const state = await ctx.sideEffect(async () => | ||
| assertFinalState((await client.send(new athena.GetQueryExecutionCommand(queryId))).QueryExecution?.Status?.State), | ||
| ); |
There was a problem hiding this comment.
@tillrohrmann what do you think about this - as an alternative to a for-loop or throwing inside the sideEffect body?
There was a problem hiding this comment.
Generally looks good. What might not be so easy to understand is that we are using Restate's retry functionality to wait for the query to complete. Maybe adding something like ". Retrying." to the message of the Error in assertFinalState can make this clearer?
|
Thanks for the suggestion – I made a dedicated "RetryableError" class, I
like that!
…On Fri, Feb 16, 2024 at 4:11 PM Till Rohrmann ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In typescript/promisify-anything/src/query-athena.ts
<#57 (comment)>:
> + const state = await ctx.sideEffect(async () =>
+ assertFinalState((await client.send(new athena.GetQueryExecutionCommand(queryId))).QueryExecution?.Status?.State),
+ );
Generally looks good. What might not be so easy to understand is that we
are using Restate's retry functionality to wait for the query to complete.
Maybe adding something like ". Retrying." to the message of the Error in
assertFinalState can make this clearer?
—
Reply to this email directly, view it on GitHub
<#57 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAA4IW66D7L7REUR3ID64R3YT5SITAVCNFSM6AAAAABAWT5HH2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQOBVGI2DSNRYGQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This example shows how to use Restate to promisify a complex dependency, in this case executing queries on AWS Athena.
A long-polling HTTP client is set up to simulate long-polling against the Restate ingress endpoint that wraps a query's
result in a very simple promise. Using a consistent idempotency token turns this into a true durable promise.