Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update INTEGRATION_TESTS.md #27632

Merged
merged 3 commits into from
Oct 20, 2023
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 26 additions & 9 deletions INTEGRATION_TESTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ All Construct libraries in the CDK code base have integration tests that serve t
This is done by running `yarn integ` which will run `cdk deploy` across all of the integration tests in that package.
If you are developing a new integration test or for some other reason want to work on a single integration test
over and over again without running through all the integration tests you can do so using
`yarn integ integ.test-name.js` .Remember to set up AWS credentials before doing this.
`yarn integ integ.test-name.js`. Remember to set up AWS credentials before doing this.
3. (Optionally) Acts as a way to validate that constructs set up the CloudFormation resources as expected.
A successful CloudFormation deployment does not mean that the resources are set up correctly.

Expand Down Expand Up @@ -75,8 +75,7 @@ you have good test coverage.

### Creating a Test

An integration tests is any file located in the `test/` directory that has a name that starts with `integ.`
(e.g. `integ.*.ts`).
Integration tests for stable modules live in `@aws-cdk-testing/framework-integ/test/MODULE_NAME/test/`. Alpha module integ tests still live in their `test/` directories. Names of integration tests start with integ (e.g. `integ.*.ts`).

To create a new integration test, first create a new file, for example `integ.my-new-construct.ts`.
The contents of this file should be a CDK app. For example, a very simple integration test for a
Expand All @@ -86,7 +85,7 @@ _integ.lambda.ts_
```ts
import * as iam from 'aws-cdk-lib/aws-iam';
import * as cdk from 'aws-cdk-lib/core';
import * as lambda from '../lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as integ from '@aws-cdk/integ-tests-alpha';

const app = new cdk.App();
Expand All @@ -102,8 +101,6 @@ const fn = new lambda.Function(stack, 'MyLambda', {
new integ.IntegTest(app, 'LambdaTest', {
testCases: [stack],
});

app.synth();
```

To run the test you would run:
Expand Down Expand Up @@ -247,7 +244,7 @@ declare const app: App;
declare const sm: sfn.StateMachine;
declare const stack: Stack;

const testCase = new IntegTest(app, 'PutEvents', {
const testCase = new integ.IntegTest(app, 'PutEvents', {
testCases: [stack],
});

Expand All @@ -262,12 +259,32 @@ const describe = testCase.assertions.awsApiCall('StepFunctions', 'describeExecut
});

// assert the results
describe.expect(ExpectedResult.objectLike({
describe.expect(integ.ExpectedResult.objectLike({
status: 'SUCCEEDED',
}));
```

Not every test requires an assertion. We typically do not need to assert CloudFormation behavior. For example, if we create an S3 Bucket
If we want to pick out certain values from the api call response, we can use the `assertAtPath()` method, as in the [integ.pipeline-with-additional-inputs.ts](https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk-testing/framework-integ/test/pipelines/test/integ.pipeline-with-additional-inputs.ts) integ test. Note that using the `outputPaths` optional parameter on the `awsApiCall()` function often interacts poorly with the `expect()` function.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that using the outputPaths optional parameter on the awsApiCall() function often interacts poorly with the expect() function.

Why is this the case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea, but it did not work for me, and I have heard from others that it has not worked smoothly for them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, it may be better to deprecate this if it is inconsistent. But not blocking this PR :)


```ts
declare const pipelineName: string;
declare const expectedString: string;

const testCase = new integ.IntegTest(app, 'PipelineAdditionalInputsTest', {
testCases: [stack],
scanlonp marked this conversation as resolved.
Show resolved Hide resolved
});

const source = testCase.assertions.awsApiCall('CodePipeline', 'GetPipeline', {
name: pipelineName,
});

// assert the value at the given path matches the expected string
// the numbers index arryas in the json response object
source.assertAtPath('pipeline.stages.0.actions.0.name', integ.ExpectedResult.stringLikeRegexp(expectedString));
```
A helpful trick is to deploy the integ test with `--no-clean` and then make the api call locally. We can then trace the path to specific values easily.
scanlonp marked this conversation as resolved.
Show resolved Hide resolved

Adding assertions is prefered on all new integ tests; however, it is not strictly required. We typically do not need to assert CloudFormation behavior. For example, if we create an S3 Bucket
with Encryption, we do not need to assert that Encryption is set on the bucket. We can trust that the CloudFormation behavior works.
Some things you should look for in deciding if the test needs an assertion:
scanlonp marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
Loading