-
Notifications
You must be signed in to change notification settings - Fork 9.2k
/
running-and-writing-acceptance-tests.md
1718 lines (1349 loc) · 79.2 KB
/
running-and-writing-acceptance-tests.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# Running and Writing Acceptance Tests
Terraform includes an acceptance test harness that does most of the repetitive
work involved in testing a resource. For additional information about testing
Terraform Providers, see the [SDKv2 documentation](https://www.terraform.io/plugin/sdkv2/testing).
## Acceptance Tests Often Cost Money to Run
Our acceptance test suite creates real resources, and as a result they cost real money to run.
Because the resources only exist for a short period of time, the total amount
of money required is usually a relatively small amount. That said there are particular services
which are very expensive to run and its important to be prepared for those costs.
Some services which can be cost prohibitive include (among others):
- WorkSpaces
- Glue
- OpenSearch
- RDS
- ACM (Amazon Certificate Manager)
- FSx
- Kinesis Analytics
- EC2
- ElastiCache
- Storage Gateway
We don't want financial limitations to be a barrier to contribution, so if you are unable to
pay to run acceptance tests for your contribution, mention this in your
pull request. We will happily accept "best effort" implementations of
acceptance tests and run them for you on our side. This might mean that your PR
takes a bit longer to merge, but it most definitely is not a blocker for
contributions.
## Running an Acceptance Test
Acceptance tests can be run using the `testacc` target in the Terraform
`Makefile`. The individual tests to run can be controlled using a regular
expression. Prior to running the tests provider configuration details such as
access keys must be made available as environment variables.
For example, to run an acceptance test against the Amazon Web Services
provider, the following environment variables must be set:
```sh
# Using a profile
export AWS_PROFILE=...
# Otherwise
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_DEFAULT_REGION=...
```
Please note that the default region for the testing is `us-west-2` and must be
overridden via the `AWS_DEFAULT_REGION` environment variable, if necessary. This
is especially important for testing AWS GovCloud (US), which requires:
```sh
export AWS_DEFAULT_REGION=us-gov-west-1
```
Tests can then be run by specifying a regular expression defining the tests to
run and the package in which the tests are defined:
```sh
$ make testacc TESTS=TestAccCloudWatchDashboard_updateName PKG=cloudwatch
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/cloudwatch/... -v -count 1 -parallel 20 -run=TestAccCloudWatchDashboard_updateName -timeout 180m
=== RUN TestAccCloudWatchDashboard_updateName
=== PAUSE TestAccCloudWatchDashboard_updateName
=== CONT TestAccCloudWatchDashboard_updateName
--- PASS: TestAccCloudWatchDashboard_updateName (25.33s)
PASS
ok github.com/hashicorp/terraform-provider-aws/internal/service/cloudwatch 25.387s
```
Entire resource test suites can be targeted by using the naming convention to
write the regular expression. For example, to run all tests of the
`aws_cloudwatch_dashboard` resource rather than just the updateName test, you
can start testing like this:
```sh
$ make testacc TESTS=TestAccCloudWatchDashboard PKG=cloudwatch
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/cloudwatch/... -v -count 1 -parallel 20 -run=TestAccCloudWatchDashboard -timeout 180m
=== RUN TestAccCloudWatchDashboard_basic
=== PAUSE TestAccCloudWatchDashboard_basic
=== RUN TestAccCloudWatchDashboard_update
=== PAUSE TestAccCloudWatchDashboard_update
=== RUN TestAccCloudWatchDashboard_updateName
=== PAUSE TestAccCloudWatchDashboard_updateName
=== CONT TestAccCloudWatchDashboard_basic
=== CONT TestAccCloudWatchDashboard_updateName
=== CONT TestAccCloudWatchDashboard_update
--- PASS: TestAccCloudWatchDashboard_basic (15.83s)
--- PASS: TestAccCloudWatchDashboard_updateName (26.69s)
--- PASS: TestAccCloudWatchDashboard_update (27.72s)
PASS
ok github.com/hashicorp/terraform-provider-aws/internal/service/cloudwatch 27.783s
```
Running acceptance tests requires version 0.12.26 or higher of the Terraform CLI to be installed.
For advanced developers, the acceptance testing framework accepts some additional environment variables that can be used to control Terraform CLI binary selection, logging, and other behaviors. See the [SDKv2 documentation](https://www.terraform.io/plugin/sdkv2/testing/acceptance-tests#environment-variables) for more information.
Please Note: On macOS 10.14 and later (and some Linux distributions), the default user open file limit is 256. This may cause unexpected issues when running the acceptance testing since this can prevent various operations from occurring such as opening network connections to AWS. To view this limit, the `ulimit -n` command can be run. To update this limit, run `ulimit -n 1024` (or higher).
### Running Cross-Account Tests
Certain testing requires multiple AWS accounts. This additional setup is not typically required and the testing will return an error (shown below) if your current setup does not have the secondary AWS configuration:
```console
$ make testacc TESTS=TestAccRDSInstance_DBSubnetGroupName_ramShared PKG=rds
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 20 -run=TestAccRDSInstance_DBSubnetGroupName_ramShared -timeout 180m
=== RUN TestAccRDSInstance_DBSubnetGroupName_ramShared
=== PAUSE TestAccRDSInstance_DBSubnetGroupName_ramShared
=== CONT TestAccRDSInstance_DBSubnetGroupName_ramShared
acctest.go:674: skipping test because at least one environment variable of [AWS_ALTERNATE_PROFILE AWS_ALTERNATE_ACCESS_KEY_ID] must be set. Usage: credentials for running acceptance testing in alternate AWS account.
--- SKIP: TestAccRDSInstance_DBSubnetGroupName_ramShared (0.85s)
PASS
ok github.com/hashicorp/terraform-provider-aws/internal/service/rds 0.888s
```
Running these acceptance tests is the same as before, except the following additional AWS credential information is required:
```sh
# Using a profile
export AWS_ALTERNATE_PROFILE=...
# Otherwise
export AWS_ALTERNATE_ACCESS_KEY_ID=...
export AWS_ALTERNATE_SECRET_ACCESS_KEY=...
```
### Running Cross-Region Tests
Certain testing requires multiple AWS regions. Additional setup is not typically required because the testing defaults the second AWS region to `us-east-1` and the third AWS region to `us-east-2`.
Running these acceptance tests is the same as before, but if you wish to override the second and third regions:
```sh
export AWS_ALTERNATE_REGION=...
export AWS_THIRD_REGION=...
```
### Running Only Short Tests
Some tests have been manually marked as long-running (longer than 300 seconds) and can be skipped using the `-short` flag. However, we are adding long-running guards little by little and many services have no guarded tests.
Where guards have been implemented, do not always skip long-running tests. However, for intermediate test runs during development, or to verify functionality unrelated to the specific long-running tests, skipping long-running tests makes work more efficient. We recommend that for the final test run before submitting a PR that you run affected tests without the `-short` flag.
If you want to run only short-running tests, you can use either one of these equivalent statements. Note the use of `-short`.
For example:
```console
$ make testacc TESTS='TestAccECSTaskDefinition_' PKG=ecs TESTARGS=-short
```
Or:
```console
$ TF_ACC=1 go test ./internal/service/ecs/... -v -count 1 -parallel 20 -run='TestAccECSTaskDefinition_' -short -timeout 180m
```
## Writing an Acceptance Test
Terraform has a framework for writing acceptance tests which minimizes the
amount of boilerplate code necessary to use common testing patterns. This guide is meant to augment the general [SDKv2 documentation](https://www.terraform.io/plugin/sdkv2/testing/acceptance-tests) with Terraform AWS Provider specific conventions and helpers.
### Anatomy of an Acceptance Test
This section describes in detail how the Terraform acceptance testing framework operates with respect to the Terraform AWS Provider. We recommend those unfamiliar with this provider, or Terraform resource testing in general, take a look here first to generally understand how we interact with AWS and the resource code to verify functionality.
The entry point to the framework is the `resource.ParallelTest()` function. This wraps our testing to work with the standard Go testing framework, while also preventing unexpected usage of AWS by requiring the `TF_ACC=1` environment variable. This function accepts a `TestCase` parameter, which has all the details about the test itself. For example, this includes the test steps (`TestSteps`) and how to verify resource deletion in the API after all steps have been run (`CheckDestroy`).
Each `TestStep` proceeds by applying some
Terraform configuration using the provider under test, and then verifying that
results are as expected by making assertions using the provider API. It is
common for a single test function to exercise both the creation of and updates
to a single resource. Most tests follow a similar structure.
1. Pre-flight checks are made to ensure that sufficient provider configuration
is available to be able to proceed - for example in an acceptance test
targeting AWS, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` must be set prior
to running acceptance tests. This is common to all tests exercising a single
provider.
Most assertion
functions are defined out of band with the tests. This keeps the tests
readable, and allows reuse of assertion functions across different tests of the
same type of resource. The definition of a complete test looks like this:
```go
func TestAccCloudWatchDashboard_basic(t *testing.T) {
var dashboard cloudwatch.GetDashboardOutput
rInt := acctest.RandInt()
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, cloudwatch.EndpointsID),
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories,
CheckDestroy: testAccCheckDashboardDestroy,
Steps: []resource.TestStep{
{
Config: testAccDashboardConfig(rInt),
Check: resource.ComposeTestCheckFunc(
testAccCheckDashboardExists("aws_cloudwatch_dashboard.foobar", &dashboard),
resource.TestCheckResourceAttr("aws_cloudwatch_dashboard.foobar", "dashboard_name", testAccDashboardName(rInt)),
),
},
},
})
}
```
When executing the test, the following steps are taken for each `TestStep`:
1. The Terraform configuration required for the test is applied. This is
responsible for configuring the resource under test, and any dependencies it
may have. For example, to test the `aws_cloudwatch_dashboard` resource, a valid configuration with the requisite fields is required. This results in configuration which looks like this:
```terraform
resource "aws_cloudwatch_dashboard" "foobar" {
dashboard_name = "terraform-test-dashboard-%d"
dashboard_body = <<EOF
{
"widgets": [{
"type": "text",
"x": 0,
"y": 0,
"width": 6,
"height": 6,
"properties": {
"markdown": "Hi there from Terraform: CloudWatch"
}
}]
}
EOF
}
```
1. Assertions are run using the provider API. These use the provider API
directly rather than asserting against the resource state. For example, to
verify that the `aws_cloudwatch_dashboard` described above was created
successfully, a test function like this is used:
```go
func testAccCheckDashboardExists(n string, dashboard *cloudwatch.GetDashboardOutput) resource.TestCheckFunc {
return func(s *terraform.State) error {
rs, ok := s.RootModule().Resources[n]
if !ok {
return fmt.Errorf("Not found: %s", n)
}
conn := acctest.Provider.Meta().(*conns.AWSClient).CloudWatchConn
params := cloudwatch.GetDashboardInput{
DashboardName: aws.String(rs.Primary.ID),
}
resp, err := conn.GetDashboard(¶ms)
if err != nil {
return err
}
*dashboard = *resp
return nil
}
}
```
Notice that the only information used from the Terraform state is the ID of
the resource. For computed properties, we instead assert that the value saved in the Terraform state was the
expected value if possible. The testing framework provides helper functions
for several common types of check - for example:
```go
resource.TestCheckResourceAttr("aws_cloudwatch_dashboard.foobar", "dashboard_name", testAccDashboardName(rInt)),
```
1. The resources created by the test are destroyed. This step happens
automatically, and is the equivalent of calling `terraform destroy`.
1. Assertions are made against the provider API to verify that the resources
have indeed been removed. If these checks fail, the test fails and reports
"dangling resources". The code to ensure that the `aws_cloudwatch_dashboard` shown
above has been destroyed looks like this:
```go
func testAccCheckDashboardDestroy(s *terraform.State) error {
conn := acctest.Provider.Meta().(*conns.AWSClient).CloudWatchConn
for _, rs := range s.RootModule().Resources {
if rs.Type != "aws_cloudwatch_dashboard" {
continue
}
params := cloudwatch.GetDashboardInput{
DashboardName: aws.String(rs.Primary.ID),
}
_, err := conn.GetDashboard(¶ms)
if err == nil {
return fmt.Errorf("Dashboard still exists: %s", rs.Primary.ID)
}
if !isDashboardNotFoundErr(err) {
return err
}
}
return nil
}
```
These functions usually test only for the resource directly under test.
### Resource Acceptance Testing
Most resources that implement standard Create, Read, Update, and Delete functionality should follow the pattern below. Each test type has a section that describes them in more detail:
- **basic**: This represents the bare minimum verification that the resource can be created, read, deleted, and optionally imported.
- **disappears**: A test that verifies Terraform will offer to recreate a resource if it is deleted outside of Terraform (e.g., via the Console) instead of returning an error that it cannot be found.
- **Per Attribute**: A test that verifies the resource with a single additional argument can be created, read, optionally updated (or force resource recreation), deleted, and optionally imported.
The leading sections below highlight additional recommended patterns.
#### Test Configurations
Most of the existing test configurations you will find in the Terraform AWS Provider are written in the following function-based style:
```go
func TestAccExampleThing_basic(t *testing.T) {
// ... omitted for brevity ...
resource.ParallelTest(t, resource.TestCase{
// ... omitted for brevity ...
Steps: []resource.TestStep{
{
Config: testAccExampleThingConfig(),
// ... omitted for brevity ...
},
},
})
}
func testAccExampleThingConfig() string {
return `
resource "aws_example_thing" "test" {
# ... omitted for brevity ...
}
`
}
```
Even when no values need to be passed in to the test configuration, we have found this setup to be the most flexible for allowing that to be easily implemented. Any configurable values are handled via `fmt.Sprintf()`. Using `text/template` or other templating styles is explicitly forbidden.
For consistency, resources in the test configuration should be named `resource "..." "test"` unless multiple of that resource are necessary.
We discourage re-using test configurations across test files (except for some common configuration helpers we provide) as it is much harder to discover potential testing regressions.
Please also note that the newline on the first line of the configuration (before `resource`) and the newline after the last line of configuration (after `}`) are important to allow test configurations to be easily combined without generating Terraform configuration language syntax errors.
#### Combining Test Configurations
We include a helper function, `acctest.ConfigCompose()` for iteratively building and chaining test configurations together. It accepts any number of configurations to combine them. This simplifies a single resource's testing by allowing the creation of a "base" test configuration for all the other test configurations (if necessary) and also allows the maintainers to curate common configurations. Each of these is described in more detail in below sections.
Please note that we do discourage _excessive_ chaining of configurations such as implementing multiple layers of "base" configurations. Usually these configurations are harder for maintainers and other future readers to understand due to the multiple levels of indirection.
##### Base Test Configurations
If a resource requires the same Terraform configuration as a prerequisite for all test configurations, then a common pattern is implementing a "base" test configuration that is combined with each test configuration.
For example:
```go
func testAccExampleThingConfigBase() string {
return `
resource "aws_iam_role" "test" {
# ... omitted for brevity ...
}
resource "aws_iam_role_policy" "test" {
# ... omitted for brevity ...
}
`
}
func testAccExampleThingConfig() string {
return acctest.ConfigCompose(
testAccExampleThingConfigBase(),
`
resource "aws_example_thing" "test" {
# ... omitted for brevity ...
}
`)
}
```
##### Available Common Test Configurations
These test configurations are typical implementations we have found or allow testing to implement best practices easier, since the Terraform AWS Provider testing is expected to run against various AWS Regions and Partitions.
- `acctest.AvailableEC2InstanceTypeForRegion("type1", "type2", ...)`: Typically used to replace hardcoded EC2 Instance Types. Uses `aws_ec2_instance_type_offering` data source to return an available EC2 Instance Type in preferred ordering. Reference the instance type via: `data.aws_ec2_instance_type_offering.available.instance_type`. Use `acctest.AvailableEC2InstanceTypeForRegionNamed("name", "type1", "type2", ...)` to specify a name for the data source
- `acctest.ConfigLatestAmazonLinuxHVMEBSAMI()`: Typically used to replace hardcoded EC2 Image IDs (`ami-12345678`). Uses `aws_ami` data source to find the latest Amazon Linux image. Reference the AMI ID via: `data.aws_ami.amzn-ami-minimal-hvm-ebs.id`
#### Randomized Naming
For AWS resources that require unique naming, the tests should implement a randomized name, typically coded as a `rName` variable in the test and passed as a parameter to creating the test configuration.
For example:
```go
func TestAccExampleThing_basic(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
// ... omitted for brevity ...
resource.ParallelTest(t, resource.TestCase{
// ... omitted for brevity ...
Steps: []resource.TestStep{
{
Config: testAccExampleThingConfigName(rName),
// ... omitted for brevity ...
},
},
})
}
func testAccExampleThingConfigName(rName string) string {
return fmt.Sprintf(`
resource "aws_example_thing" "test" {
name = %[1]q
}
`, rName)
}
```
Typically the `rName` is always the first argument to the test configuration function, if used, for consistency.
Note that if `rName` (or any other variable) is used multiple times in the `fmt.Sprintf()` statement, _do not_ repeat `rName` in the `fmt.Sprintf()` arguments. Using `fmt.Sprintf(..., rName, rName)`, for example, would not be correct. Instead, use the indexed `%[1]q` (or `%[x]q`, `%[x]s`, `%[x]t`, or `%[x]d`, where `x` represents the index number) verb multiple times. For example:
```go
func testAccExampleThingConfigName(rName string) string {
return fmt.Sprintf(`
resource "aws_example_thing" "test" {
name = %[1]q
tags = {
Name = %[1]q
}
}
`, rName)
}
```
#### Other Recommended Variables
We also typically recommend saving a `resourceName` variable in the test that contains the resource reference, e.g., `aws_example_thing.test`, which is repeatedly used in the checks.
For example:
```go
func TestAccExampleThing_basic(t *testing.T) {
// ... omitted for brevity ...
resourceName := "aws_example_thing.test"
resource.ParallelTest(t, resource.TestCase{
// ... omitted for brevity ...
Steps: []resource.TestStep{
{
// ... omitted for brevity ...
Check: resource.ComposeTestCheckFunc(
testAccCheckExampleThingExists(resourceName),
acctest.CheckResourceAttrRegionalARN(resourceName, "arn", "example", fmt.Sprintf("thing/%s", rName)),
resource.TestCheckResourceAttr(resourceName, "description", ""),
resource.TestCheckResourceAttr(resourceName, "name", rName),
),
},
{
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
},
},
})
}
// below all TestAcc functions
func testAccExampleThingConfigName(rName string) string {
return fmt.Sprintf(`
resource "aws_example_thing" "test" {
name = %[1]q
}
`, rName)
}
```
#### Basic Acceptance Tests
Usually this test is implemented first. The test configuration should contain only required arguments (`Required: true` attributes) and it should check the values of all read-only attributes (`Computed: true` without `Optional: true`). If the resource supports it, it verifies import. It should _NOT_ perform other `TestStep` such as updates or verify recreation.
These are typically named `TestAcc{SERVICE}{THING}_basic`, e.g., `TestAccCloudWatchDashboard_basic`
For example:
```go
func TestAccExampleThing_basic(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_example_thing.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, service.EndpointsID),
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories,
CheckDestroy: testAccCheckExampleThingDestroy,
Steps: []resource.TestStep{
{
Config: testAccExampleThingConfigName(rName),
Check: resource.ComposeTestCheckFunc(
testAccCheckExampleThingExists(resourceName),
acctest.CheckResourceAttrRegionalARN(resourceName, "arn", "example", fmt.Sprintf("thing/%s", rName)),
resource.TestCheckResourceAttr(resourceName, "description", ""),
resource.TestCheckResourceAttr(resourceName, "name", rName),
),
},
{
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
},
},
})
}
// below all TestAcc functions
func testAccExampleThingConfigName(rName string) string {
return fmt.Sprintf(`
resource "aws_example_thing" "test" {
name = %[1]q
}
`, rName)
}
```
#### PreChecks
Acceptance test cases have a PreCheck. The PreCheck ensures that the testing environment meets certain preconditions. If the environment does not meet the preconditions, Go skips the test. Skipping a test avoids reporting a failure and wasting resources where the test cannot succeed.
Here is an example of the default PreCheck:
```go
func TestAccExampleThing_basic(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_example_thing.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
// ... additional checks follow ...
})
}
```
Extend the default PreCheck by adding calls to functions in the anonymous PreCheck function. The functions can be existing functions in the provider or custom functions you add for new capabilities.
##### Standard Provider PreChecks
If you add a new test that has preconditions which are checked by an existing provider function, use that standard PreCheck instead of creating a new one. Some existing tests are missing standard PreChecks and you can help by adding them where appropriate.
These are some of the standard provider PreChecks:
* `acctest.PreCheckPartitionHasService(serviceId string, t *testing.T)` checks whether the current partition lists the service as part of its offerings. Note: AWS may not add new or public preview services to the service list immediately. This function will return a false positive in that case.
* `acctest.PreCheckOrganizationsAccount(t *testing.T)` checks whether the current account can perform AWS Organizations tests.
* `acctest.PreCheckAlternateAccount(t *testing.T)` checks whether the environment is set up for tests across accounts.
* `acctest.PreCheckMultipleRegion(t *testing.T, regions int)` checks whether the environment is set up for tests across regions.
This is an example of using a standard PreCheck function. For an established service, such as WAF or FSx, use `acctest.PreCheckPartitionHasService()` and the service endpoint ID to check that a partition supports the service.
```go
func TestAccExampleThing_basic(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_example_thing.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t); acctest.PreCheckPartitionHasService(waf.EndpointsID, t) },
// ... additional checks follow ...
})
}
```
##### Custom PreChecks
In situations where standard PreChecks do not test for the required preconditions, create a custom PreCheck.
Below is an example of adding a custom PreCheck function. For a new or preview service that AWS does not include in the partition service list yet, you can verify the existence of the service with a simple read-only request (e.g., list all X service things). (For acceptance tests of established services, use `acctest.PreCheckPartitionHasService()` instead.)
```go
func TestAccExampleThing_basic(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_example_thing.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t), testAccPreCheckExample(t) },
// ... additional checks follow ...
})
}
func testAccPreCheckExample(t *testing.T) {
conn := acctest.Provider.Meta().(*conns.AWSClient).ExampleConn
input := &example.ListThingsInput{}
_, err := conn.ListThings(input)
if testAccPreCheckSkipError(err) {
t.Skipf("skipping acceptance testing: %s", err)
}
if err != nil {
t.Fatalf("unexpected PreCheck error: %s", err)
}
}
```
#### ErrorChecks
Acceptance test cases have an ErrorCheck. The ErrorCheck provides a chance to take a look at errors before the test fails. While most errors should result in test failure, some should not. For example, an error that indicates an API operation is not supported in a particular region should cause the test to skip instead of fail. Since errors should flow through the ErrorCheck, do not handle the vast majority of failing conditions. Instead, in ErrorCheck, focus on the rare errors that should cause a test to skip, or in other words, be ignored.
##### Common ErrorCheck
In many situations, the common ErrorCheck is sufficient. It will skip tests for several normal occurrences such as when AWS reports a feature is not supported in the current region.
Here is an example of the common ErrorCheck:
```go
func TestAccExampleThing_basic(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_example_thing.test"
resource.ParallelTest(t, resource.TestCase{
// PreCheck
ErrorCheck: acctest.ErrorCheck(t, service.EndpointsID),
// ... additional checks follow ...
})
}
```
##### Service-Specific ErrorChecks
However, some services have special conditions that aren't caught by the common ErrorCheck. In these cases, you can create a service-specific ErrorCheck.
To add a service-specific ErrorCheck, follow these steps:
1. Make sure there is not already an ErrorCheck for the service you have in mind. For example, search the codebase for `acctest.RegisterServiceErrorCheckFunc(service.EndpointsID` replacing "service" with the package name of the service you're working on (e.g., `ec2`). If there is already an ErrorCheck for the service, add to the existing service-specific ErrorCheck.
2. Create the service-specific ErrorCheck in an `_test.go` file for the service. See the example below.
3. Register the new service-specific ErrorCheck in the `init()` at the top of the `_test.go` file. See the example below.
An example of adding a service-specific ErrorCheck:
```go
// just after the imports, create or add to the init() function
func init() {
acctest.RegisterServiceErrorCheck(service.EndpointsID, testAccErrorCheckSkipService)
}
// ... additional code and tests ...
// this is the service-specific ErrorCheck
func testAccErrorCheckSkipService(t *testing.T) resource.ErrorCheckFunc {
return acctest.ErrorCheckSkipMessagesContaining(t,
"Error message specific to the service that indicates unsupported features",
"You can include from one to many portions of error messages",
"Be careful to not inadvertently capture errors that should not be skipped",
)
}
```
#### Long-Running Test Guards
For any acceptance tests that typically run longer than 300 seconds (5 minutes), add a `-short` test guard at the top of the test function.
For example:
```go
func TestAccExampleThing_longRunningTest(t *testing.T) {
if testing.Short() {
t.Skip("skipping long-running test in short mode")
}
// ... omitted for brevity ...
resource.ParallelTest(t, resource.TestCase{
// ... omitted for brevity ...
})
}
```
When running acceptances tests, tests with these guards can be skipped using the Go `-short` flag. See [Running Only Short Tests](#running-only-short-tests) for examples.
#### Disappears Acceptance Tests
This test is generally implemented second. It is straightforward to setup once the basic test is passing since it can reuse that test configuration. It prevents a common bug report with Terraform resources that error when they can not be found (e.g., deleted outside Terraform).
These are typically named `TestAcc{SERVICE}{THING}_disappears`, e.g., `TestAccCloudWatchDashboard_disappears`
For example:
```go
func TestAccExampleThing_disappears(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_example_thing.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, service.EndpointsID),
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories,
CheckDestroy: testAccCheckExampleThingDestroy,
Steps: []resource.TestStep{
{
Config: testAccExampleThingConfigName(rName),
Check: resource.ComposeTestCheckFunc(
testAccCheckExampleThingExists(resourceName, &job),
acctest.CheckResourceDisappears(acctest.Provider, ResourceExampleThing(), resourceName),
),
ExpectNonEmptyPlan: true,
},
},
})
}
```
If this test does fail, the fix for this is generally adding error handling immediately after the `Read` API call that catches the error and tells Terraform to remove the resource before returning the error:
```go
output, err := conn.GetThing(input)
if isAWSErr(err, example.ErrCodeResourceNotFound, "") {
log.Printf("[WARN] Example Thing (%s) not found, removing from state", d.Id())
d.SetId("")
return nil
}
if err != nil {
return fmt.Errorf("reading Example Thing (%s): %w", d.Id(), err)
}
```
For children resources that are encapsulated by a parent resource, it is also preferable to verify that removing the parent resource will not generate an error either. These are typically named `TestAcc{SERVICE}{THING}_disappears_{PARENT}`, e.g., `TestAccRoute53ZoneAssociation_disappears_Vpc`
```go
func TestAccExampleChildThing_disappears_ParentThing(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
parentResourceName := "aws_example_parent_thing.test"
resourceName := "aws_example_child_thing.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, service.EndpointsID),
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories,
CheckDestroy: testAccCheckExampleChildThingDestroy,
Steps: []resource.TestStep{
{
Config: testAccExampleThingConfigName(rName),
Check: resource.ComposeTestCheckFunc(
testAccCheckExampleThingExists(resourceName),
acctest.CheckResourceDisappears(acctest.Provider, ResourceExampleParentThing(), parentResourceName),
),
ExpectNonEmptyPlan: true,
},
},
})
}
```
#### Per Attribute Acceptance Tests
These are typically named `TestAcc{SERVICE}{THING}_{ATTRIBUTE}`, e.g., `TestAccCloudWatchDashboard_Name`
For example:
```go
func TestAccExampleThing_Description(t *testing.T) {
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_example_thing.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, service.EndpointsID),
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories,
CheckDestroy: testAccCheckExampleThingDestroy,
Steps: []resource.TestStep{
{
Config: testAccExampleThingConfigDescription(rName, "description1"),
Check: resource.ComposeTestCheckFunc(
testAccCheckExampleThingExists(resourceName),
resource.TestCheckResourceAttr(resourceName, "description", "description1"),
),
},
{
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
},
{
Config: testAccExampleThingConfigDescription(rName, "description2"),
Check: resource.ComposeTestCheckFunc(
testAccCheckExampleThingExists(resourceName),
resource.TestCheckResourceAttr(resourceName, "description", "description2"),
),
},
},
})
}
// below all TestAcc functions
func testAccExampleThingConfigDescription(rName string, description string) string {
return fmt.Sprintf(`
resource "aws_example_thing" "test" {
description = %[2]q
name = %[1]q
}
`, rName, description)
}
```
#### Cross-Account Acceptance Tests
When testing requires AWS infrastructure in a second AWS account, the below changes to the normal setup will allow the management or reference of resources and data sources across accounts:
- In the `PreCheck` function, include `acctest.PreCheckOrganizationsAccount(t)` to ensure a standardized set of information is required for cross-account testing credentials
- Switch usage of `ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories` to `ProtoV5ProviderFactories: acctest.ProtoV5FactoriesAlternate(t)`
- Add `acctest.ConfigAlternateAccountProvider()` to the test configuration and use `provider = awsalternate` for cross-account resources. The resource that is the focus of the acceptance test should _not_ use the alternate provider identification to simplify the testing setup.
- For any `TestStep` that includes `ImportState: true`, add the `Config` that matches the previous `TestStep` `Config`
An example acceptance test implementation can be seen below:
```go
func TestAccExample_basic(t *testing.T) {
resourceName := "aws_example.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() {
acctest.PreCheck(t)
acctest.PreCheckOrganizationsAccount(t)
},
ErrorCheck: acctest.ErrorCheck(t, service.EndpointsID),
ProtoV5ProviderFactories: acctest.ProtoV5FactoriesAlternate(t),
CheckDestroy: testAccCheckExampleDestroy,
Steps: []resource.TestStep{
{
Config: testAccExampleConfig(),
Check: resource.ComposeTestCheckFunc(
testAccCheckExampleExists(resourceName),
// ... additional checks ...
),
},
{
Config: testAccExampleConfig(),
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
},
},
})
}
func testAccExampleConfig() string {
return acctest.ConfigAlternateAccountProvider() + fmt.Sprintf(`
# Cross account resources should be handled by the cross account provider.
# The standardized provider block to use is awsalternate as seen below.
resource "aws_cross_account_example" "test" {
provider = awsalternate
# ... configuration ...
}
# The resource that is the focus of the testing should be handled by the default provider,
# which is automatically done by not specifying the provider configuration in the resource.
resource "aws_example" "test" {
# ... configuration ...
}
`)
}
```
Searching for usage of `acctest.PreCheckOrganizationsAccount` in the codebase will yield real world examples of this setup in action.
#### Cross-Region Acceptance Tests
When testing requires AWS infrastructure in a second or third AWS region, the below changes to the normal setup will allow the management or reference of resources and data sources across regions:
- In the `PreCheck` function, include `acctest.PreCheckMultipleRegion(t, ###)` to ensure a standardized set of information is required for cross-region testing configuration. If the infrastructure in the second AWS region is also in a second AWS account also include `acctest.PreCheckOrganizationsAccount(t)`
- Switch usage of `ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories` to `ProtoV5ProviderFactories: acctest.ProtoV5FactoriesMultipleRegions(t, 2)` (where the last parameter is number of regions, 2 or 3)
- Add `acctest.ConfigMultipleRegionProvider(###)` to the test configuration and use `provider = awsalternate` (and potentially `provider = awsthird`) for cross-region resources. The resource that is the focus of the acceptance test should _not_ use the alternative providers to simplify the testing setup. If the infrastructure in the second AWS region is also in a second AWS account use `testAccAlternateAccountAlternateRegionProviderConfig()` (EC2) instead
- For any `TestStep` that includes `ImportState: true`, add the `Config` that matches the previous `TestStep` `Config`
An example acceptance test implementation can be seen below:
```go
func TestAccExample_basic(t *testing.T) {
var providers []*schema.Provider
resourceName := "aws_example.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() {
acctest.PreCheck(t)
acctest.PreCheckMultipleRegion(t, 2)
},
ErrorCheck: acctest.ErrorCheck(t, service.EndpointsID),
ProtoV5ProviderFactories: acctest.ProtoV5FactoriesMultipleRegions(t, 2),
CheckDestroy: testAccCheckExampleDestroy,
Steps: []resource.TestStep{
{
Config: testAccExampleConfig(),
Check: resource.ComposeTestCheckFunc(
testAccCheckExampleExists(resourceName),
// ... additional checks ...
),
},
{
Config: testAccExampleConfig(),
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
},
},
})
}
func testAccExampleConfig() string {
return acctest.ConfigMultipleRegionProvider(2) + fmt.Sprintf(`
# Cross region resources should be handled by the cross region provider.
# The standardized provider is awsalternate as seen below.
resource "aws_cross_region_example" "test" {
provider = awsalternate
# ... configuration ...
}
# The resource that is the focus of the testing should be handled by the default provider,
# which is automatically done by not specifying the provider configuration in the resource.
resource "aws_example" "test" {
# ... configuration ...
}
`)
}
```
Searching for usage of `acctest.PreCheckMultipleRegion` in the codebase will yield real world examples of this setup in action.
#### Service-Specific Region Acceptance Testing
Certain AWS service APIs are only available in specific AWS regions. For example as of this writing, the `pricing` service is available in `ap-south-1` and `us-east-1`, but no other regions or partitions. When encountering these types of services, the acceptance testing can be setup to automatically detect the correct region(s), while skipping the testing in unsupported partitions.
To prepare the shared service functionality, create a file named `internal/service/{SERVICE}/acc_test.go`. A starting example with the Pricing service (`internal/service/pricing/acc_test.go`):
```go
package aws
import (
"context"
"sync"
"testing"
"github.com/aws/aws-sdk-go/aws/endpoints"
"github.com/aws/aws-sdk-go/service/pricing"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-plugin-sdk/v2/terraform"
"github.com/hashicorp/terraform-provider-aws/internal/acctest"
"github.com/hashicorp/terraform-provider-aws/internal/provider"
)
// testAccPricingRegion is the chosen Pricing testing region
//
// Cached to prevent issues should multiple regions become available.
var testAccPricingRegion string
// testAccProviderPricing is the Pricing provider instance
//
// This Provider can be used in testing code for API calls without requiring
// the use of saving and referencing specific ProviderFactories instances.
//
// testAccPreCheckPricing(t) must be called before using this provider instance.
var testAccProviderPricing *schema.Provider
// testAccProviderPricingConfigure ensures the provider is only configured once
var testAccProviderPricingConfigure sync.Once
// testAccPreCheckPricing verifies AWS credentials and that Pricing is supported
func testAccPreCheckPricing(t *testing.T) {
acctest.PreCheckPartitionHasService(pricing.EndpointsID, t)
// Since we are outside the scope of the Terraform configuration we must
// call Configure() to properly initialize the provider configuration.
testAccProviderPricingConfigure.Do(func() {
testAccProviderPricing = provider.Provider()
config := map[string]interface{}{
"region": testAccGetPricingRegion(),
}
diags := testAccProviderPricing.Configure(context.Background(), terraform.NewResourceConfigRaw(config))
if diags != nil && diags.HasError() {