From 8f4600c24fedc6319da8858439b0c35a47460179 Mon Sep 17 00:00:00 2001 From: Parker Scanlon <69879391+scanlonp@users.noreply.github.com> Date: Fri, 20 Oct 2023 15:59:04 -0700 Subject: [PATCH] docs: update INTEGRATION_TESTS.md (#27632) This updates our docs surrounding integ tests: - gives the correct location of stable module integ tests - standardizes code examples - adds example for using `assertAtPath()` - warns against using `outputPaths` in `awsApiCall()` - other small updates ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* --- INTEGRATION_TESTS.md | 47 +++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 11 deletions(-) diff --git a/INTEGRATION_TESTS.md b/INTEGRATION_TESTS.md index 269271eb754b5..f8c7fe2b98e0b 100644 --- a/INTEGRATION_TESTS.md +++ b/INTEGRATION_TESTS.md @@ -24,7 +24,7 @@ All Construct libraries in the CDK code base have integration tests that serve t This is done by running `yarn integ` which will run `cdk deploy` across all of the integration tests in that package. If you are developing a new integration test or for some other reason want to work on a single integration test over and over again without running through all the integration tests you can do so using - `yarn integ integ.test-name.js` .Remember to set up AWS credentials before doing this. + `yarn integ integ.test-name.js`. Remember to set up AWS credentials before doing this. 3. (Optionally) Acts as a way to validate that constructs set up the CloudFormation resources as expected. A successful CloudFormation deployment does not mean that the resources are set up correctly. @@ -75,8 +75,7 @@ you have good test coverage. ### Creating a Test -An integration tests is any file located in the `test/` directory that has a name that starts with `integ.` -(e.g. `integ.*.ts`). +Integration tests for stable modules live in `@aws-cdk-testing/framework-integ/test/MODULE_NAME/test/`. Alpha module integ tests still live in their `test/` directories. Names of integration tests start with integ (e.g. `integ.*.ts`). To create a new integration test, first create a new file, for example `integ.my-new-construct.ts`. The contents of this file should be a CDK app. For example, a very simple integration test for a @@ -86,7 +85,7 @@ _integ.lambda.ts_ ```ts import * as iam from 'aws-cdk-lib/aws-iam'; import * as cdk from 'aws-cdk-lib/core'; -import * as lambda from '../lib'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; import * as integ from '@aws-cdk/integ-tests-alpha'; const app = new cdk.App(); @@ -102,8 +101,6 @@ const fn = new lambda.Function(stack, 'MyLambda', { new integ.IntegTest(app, 'LambdaTest', { testCases: [stack], }); - -app.synth(); ``` To run the test you would run: @@ -243,11 +240,13 @@ want to validate that the _integration_ connecting `StepFunctions` to the `Event way to do that is to actually trigger the `StateMachine` and validate that it was successful. ```ts +import * as integ from '@aws-cdk/integ-tests-alpha'; + declare const app: App; declare const sm: sfn.StateMachine; declare const stack: Stack; -const testCase = new IntegTest(app, 'PutEvents', { +const testCase = new integ.IntegTest(app, 'PutEvents', { testCases: [stack], }); @@ -262,21 +261,47 @@ const describe = testCase.assertions.awsApiCall('StepFunctions', 'describeExecut }); // assert the results -describe.expect(ExpectedResult.objectLike({ +describe.expect(integ.ExpectedResult.objectLike({ status: 'SUCCEEDED', })); ``` -Not every test requires an assertion. We typically do not need to assert CloudFormation behavior. For example, if we create an S3 Bucket +If we want to pick out certain values from the api call response, we can use the `assertAtPath()` method, as in the [integ.pipeline-with-additional-inputs.ts](https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk-testing/framework-integ/test/pipelines/test/integ.pipeline-with-additional-inputs.ts) integ test. Note that using the `outputPaths` optional parameter on the `awsApiCall()` function often interacts poorly with the `expect()` function. + +```ts +import * as integ from '@aws-cdk/integ-tests-alpha'; + +declare const app: App; +declare const stack: Stack; +declare const pipelineName: string; +declare const expectedString: string; + +const testCase = new integ.IntegTest(app, 'PipelineAdditionalInputsTest', { + testCases: [stack], +}); + +const source = testCase.assertions.awsApiCall('CodePipeline', 'GetPipeline', { + name: pipelineName, +}); + +// assert the value at the given path matches the expected string +// the numbers index arryas in the json response object +source.assertAtPath('pipeline.stages.0.actions.0.name', integ.ExpectedResult.stringLikeRegexp(expectedString)); +``` +A helpful trick is to deploy the integ test with `--no-clean` and then make the api call locally. We can then trace the path to specific values easily. For example, `> aws codepipeline get-pipeline --name MyFirstPipeline`. + +Adding assertions is prefered on all new integ tests; however, it is not strictly required. We typically do not need to assert CloudFormation behavior. For example, if we create an S3 Bucket with Encryption, we do not need to assert that Encryption is set on the bucket. We can trust that the CloudFormation behavior works. Some things you should look for in deciding if the test needs an assertion: -- Integrations between services (i.e. integration libraries like `aws-cdk-lib/aws-lambda-destinations`, `aws-cdk-lib/aws-stepfunctions-tasks`, etc) -- Anything that bundles or deploys custom code (i.e. does a Lambda function bundled with `aws-cdk-lib/aws-lambda-nodejs` still invoke or did we break bundling behavior) +- Integrations between services (i.e. integration libraries like `aws-cdk-lib/aws-lambda-destinations`, `aws-cdk-lib/aws-stepfunctions-tasks`, etc). +- All custom resources. Must assert the expected behavior of the lambda is correct. +- Anything that bundles or deploys custom code (i.e. does a Lambda function bundled with `aws-cdk-lib/aws-lambda-nodejs` still invoke or did we break bundling behavior). - IAM/Networking connections. - This one is a bit of a judgement call. Most things do not need assertions, but sometimes we handle complicated configurations involving IAM permissions or Networking access. + ## Running Integration Tests Most of the time you will only need to run integration tests for an individual module (i.e. `aws-lambda`). Other times you may need to run tests across multiple modules.