Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update name of sdk-v2-to-v3-adapter and improve documentation #29847

Merged
merged 10 commits into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions aws-cdk.code-workspace
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
"rootPath": "packages/@aws-cdk/integ-tests-alpha"
},
{
"name": "sdk-v2-to-v3-adapter",
"rootPath": "packages/@aws-cdk/sdk-v2-to-v3-adapter"
"name": "aws-custom-resource-sdk-adapter",
"rootPath": "packages/@aws-cdk/aws-custom-resource-sdk-adapter"
}
]
},
Expand Down
94 changes: 94 additions & 0 deletions packages/@aws-cdk/aws-custom-resource-sdk-adapter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# AWS Custom Resource SDK Adapter

The AWS custom resource SDK adapter is an internal collection of tools built to maintain compatibility with the contracts defined for `AwsCustomResource` and the current AWS SDK version. `AwsCustomResource` can be used where a single API call can fill a gap in CloudFormation coverage by allowing a user to specify a single API call specific to create, update, and delete stack events. Users specify a `service`, an `action`, and `parameters` as part of the `AwsSdkCall` interface which are used to dynamically invoke the API. Since `AwsCustomResource` was created while SDKv2 was active, it implicitly inherited the SDKv2 contract. As a result, migrating to newer SDK versions will result in breaking changes in the `AwsCustomResource` construct. In its current state, the AWS custom resource SDK adapter contains tooling that allows `AwsCustomResource` to make API calls using SDKv3 and return the associated response. These tools are related to:

* Input type coercion
* Response coercion
* Naming normalizaion

## Input Type Coercion

In general, input types expected by SDKv3 are more restrictive than what was expected by SDKv2. One such example is the regression to number types with respect to AWS SDKv2. More specifically, in SDKv2, number types that are accidentally passed as strings will be silently converted to the right type. SDKv3, however, will not do the conversion which causes the server call to fail because of mismatched types. For example,

**SDKv2**

```ts
const codedeploy = new AWS.CodeDeploy({ region: 'eu-west-1' });

const input: AWS.CodeDeploy.CreateDeploymentConfigInput = {
deploymentConfigName: 'testtest',
computePlatform: 'Lambda',
trafficRoutingConfig: {
type: "TimeBasedLinear",
timeBasedLinear: {
linearInterval: "1" as any, // The type says 'number' but we're forcing strings here
linearPercentage:"5" as any,
},
},
};

// Following call happily succeeds
console.log(await codedeploy.createDeploymentConfig(input).promise());
```

**SDKv3**

```ts
const codedeploy = new CodeDeploy();

const input: CreateDeploymentConfigCommandInput = {
deploymentConfigName: 'testtest',
computePlatform: 'Lambda',
trafficRoutingConfig: {
type: "TimeBasedLinear",
timeBasedLinear: {
linearInterval: "1" as any, // The type says 'number' but we're forcing strings here
linearPercentage:"5" as any,
},
},
};

await codedeploy.createDeploymentConfig(input);

// The above call fails with the following message:
'SerializationException: STRING_VALUE can not be converted to an Integer'
```

Another example is the regression to blob types with respect to SDKv2. More specifically, in SDKv2, input fields marked as blob types used to permissively accept strings, buffers, and uint8arrays. In SDKv3, these same fields now only accept uint8arrays.

In response, the [`Coercer`](./lib/coerce-api-parameters.ts) class was created to coerce input parameters to the type expected by SDKv3. At a high-level, this class coerces parameter types using a state-machine generated using smithy models. The state-machine is gzipped to save bytes and the gzipped representation can be seen [here](./lib/parameter-types.ts).

## Response Coercion

In some cases, API call responses for SDKv3 differ from API call responses for SDKv2. One example is streaming vs. buffered responses. More specifically, SDKv3 prefers not to buffer potentially large responses. For node.js, you must consume the stream or garbage collect the client or its request handler to keep the connection open to new traffic by freeing sockets. For example,

**SDKv2**

```ts
// this buffers (consumes) the stream already
const get = await s3.getObject({ ... }).promise();
```

**SDKv3**

```ts
// consume the stream to free the socket
const get = await s3.getObject({ ... }); // object .Body has unconsumed stream
const str = await get.Body.transformToString(); // consumes the stream
```

Users are able to retrieve values returned as a result of API calls they’ve defined while using the `AwsCustomResource` construct. To be retrievable, the response values must be strings. With SDKv3, the stream must be fully consumed to a string before the custom resource exits. Thus, response coercion was built as another tool in the AWS custom resource SDK adapter. Return type coercion exists in the [api-call](./lib/api-call.ts) file and the logic is implemented in the `coerceSdkv3Response` function.

## Naming Normalization

Originally, the `AwsCustomResource` construct advertised that the `service` argument defined in the `AwsSdkCall` interface could be in any one of the following formats:

* The SDK v2 constructor name: APIGateway
* The SDK v2 constructor name in all lower case: apigateway (primary format)

Similarly, the `action` argument defined in the `AwsSdkCall` interface was advertised as being permitted in any of the following formats:

* The API call name: GetRestApi
* The API call name with a lower case starting letter: getRestApi (primary format)

Migrating `AwsCustomResource` to SDKv3 meant that the permitted `service` and `action` values for SDKv2 needed to be accepted as well as new formats with respect to SDKv3. As a result, naming normalization was established as a way to convert the `service` and `action` arguments into the format expected for SDKv3 API calls. The normalization functions for `service` and `action` can be seen in the [sdk-info](./lib/sdk-info.ts) file and are named `normalizeServiceName` and `normalizeActionName`, respectively. To enable normalization of the `service` argument, the client package names map file from the `aws-sdk-js-codemod` repository is used and can be found [here](./lib/sdk-v2-to-v3.json). More specifically, this file is used to map SDKv2 `service` names to the equivalent SDKv3 `service` name. To enable normalization of the `action` argument a [sdk-v3-metadata](./lib/sdk-v3-metadata.json) file was extracted which contains a list of APIs that end in the word `Command` which allows us to disambiguate around these when generating the SDKv3 `action` name. Note that the [sdk-v3-metadata](./lib/sdk-v3-metadata.json) file also contains a mapping of `service` names into an IAM name allowing us to correctly identify the IAM prefix for each `service`.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@aws-cdk/sdk-v2-to-v3-adapter",
"description": "Adapter to convert AWS SDK v2 to AWS CDK v3 calls",
"name": "@aws-cdk/aws-custom-resource-sdk-adapter",
"description": "Adapter to convert AWS SDK v2 to AWS CDK v3 calls for AwsCustomResource",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we update this description to no longer refer to this as converting v2 to v3 calls, but maybe something like Adapter to convert AWS CDK to AWS SDK compatible calls (or something more generic)?

"private": true,
"version": "0.0.0",
"main": "lib/index.js",
Expand Down Expand Up @@ -36,7 +36,7 @@
"repository": {
"url": "https://github.com/aws/aws-cdk.git",
"type": "git",
"directory": "packages/@aws-cdk/sdk-v2-to-v3-adapter"
"directory": "packages/@aws-cdk/aws-custom-resource-sdk-adapter"
},
"keywords": [
"aws",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"noFallthroughCasesInSwitch": true,
"resolveJsonModule": true,
"composite": true,
"incremental": true
"incremental": true,
},
"include": ["**/*.ts", "**/*.d.ts"]
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/* eslint-disable no-console */
// eslint-disable-next-line import/no-extraneous-dependencies
import { ApiCall } from '@aws-cdk/sdk-v2-to-v3-adapter';
import { ApiCall } from '@aws-cdk/aws-custom-resource-sdk-adapter';

interface AwsApiInput {
readonly service: string;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import * as AWSLambda from 'aws-lambda';
import { AwsSdkCall } from './construct-types';
import { decodeCall, decodeSpecialValues, filterKeys, respond, startsWithOneOf } from './shared';
// eslint-disable-next-line import/no-extraneous-dependencies
import { flatten } from '@aws-cdk/sdk-v2-to-v3-adapter';
import { flatten } from '@aws-cdk/aws-custom-resource-sdk-adapter';

let latestSdkInstalled = false;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
/* eslint-disable no-console */
import { execSync } from 'child_process';
// eslint-disable-next-line import/no-extraneous-dependencies
import { ApiCall } from '@aws-cdk/sdk-v2-to-v3-adapter';
import { ApiCall } from '@aws-cdk/aws-custom-resource-sdk-adapter';
// import the AWSLambda package explicitly,
// which is globally available in the Lambda runtime,
// as otherwise linking this repository with link-all.sh
Expand Down
2 changes: 1 addition & 1 deletion packages/@aws-cdk/custom-resource-handlers/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"devDependencies": {
"@aws-cdk/cdk-build-tools": "0.0.0",
"@aws-cdk/pkglint": "0.0.0",
"@aws-cdk/sdk-v2-to-v3-adapter": "0.0.0",
"@aws-cdk/aws-custom-resource-sdk-adapter": "0.0.0",
"@aws-sdk/client-ecs": "3.451.0",
"@aws-sdk/client-ssm": "3.453.0",
"@aws-sdk/client-kinesis": "3.451.0",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/* eslint-disable no-console */
import { CustomResourceHandler } from './base';
import { AwsApiCallRequest, AwsApiCallResult } from './types';
import { ApiCall, flatten } from '@aws-cdk/sdk-v2-to-v3-adapter';
import { ApiCall, flatten } from '@aws-cdk/aws-custom-resource-sdk-adapter';
import { decodeParameters, deepParseJson } from './utils';

export class AwsApiCallHandler extends CustomResourceHandler<AwsApiCallRequest, AwsApiCallResult | { [key: string]: unknown }> {
Expand Down
2 changes: 1 addition & 1 deletion packages/@aws-cdk/integ-tests-alpha/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
"@aws-cdk/cdk-build-tools": "0.0.0",
"@aws-cdk/integ-runner": "0.0.0",
"@aws-cdk/pkglint": "0.0.0",
"@aws-cdk/sdk-v2-to-v3-adapter": "0.0.0",
"@aws-cdk/aws-custom-resource-sdk-adapter": "0.0.0",
"@aws-sdk/client-ec2": "3.421.0",
"@aws-sdk/client-s3": "3.421.0",
"@aws-sdk/client-sfn": "3.421.0",
Expand Down
3 changes: 0 additions & 3 deletions packages/@aws-cdk/sdk-v2-to-v3-adapter/README.md

This file was deleted.

2 changes: 1 addition & 1 deletion packages/aws-cdk-lib/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@
"@aws-cdk/cdk-build-tools": "0.0.0",
"@aws-cdk/custom-resource-handlers": "0.0.0",
"@aws-cdk/pkglint": "0.0.0",
"@aws-cdk/sdk-v2-to-v3-adapter": "0.0.0",
"@aws-cdk/aws-custom-resource-sdk-adapter": "0.0.0",
"@aws-cdk/spec2cdk": "0.0.0",
"@aws-sdk/client-acm": "3.421.0",
"@aws-sdk/client-account": "3.421.0",
Expand Down
6 changes: 3 additions & 3 deletions scripts/update-sdkv3-parameters-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,15 +87,15 @@ async function main(argv: string[]) {
}

const root = path.resolve(__dirname, '..');
await renderStateMachineToTypeScript(sortedStateMachine, path.join(root, 'packages/@aws-cdk/sdk-v2-to-v3-adapter/lib/parameter-types.ts'));
await renderStateMachineToTypeScript(sortedStateMachine, path.join(root, 'packages/@aws-cdk/aws-custom-resource-sdk-adapter/lib/parameter-types.ts'));

await writeAllServiceToModelFile(allServices, [
path.join(root, 'packages/aws-cdk-lib/custom-resources/lib/helpers-internal/sdk-v3-metadata.json'),
path.join(root, 'packages/@aws-cdk/sdk-v2-to-v3-adapter/lib/sdk-v3-metadata.json'),
path.join(root, 'packages/@aws-cdk/aws-custom-resource-sdk-adapter/lib/sdk-v3-metadata.json'),
]);
await writeV2ToV3Mapping([
path.join(root, 'packages/aws-cdk-lib/custom-resources/lib/helpers-internal/sdk-v2-to-v3.json'),
path.join(root, 'packages/@aws-cdk/sdk-v2-to-v3-adapter/lib/sdk-v2-to-v3.json'),
path.join(root, 'packages/@aws-cdk/aws-custom-resource-sdk-adapter/lib/sdk-v2-to-v3.json'),
]);
}

Expand Down
Loading