Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support changing provider configurations during test steps #779

Closed
paddycarver opened this issue Jul 7, 2021 · 2 comments · Fixed by #972
Closed

Support changing provider configurations during test steps #779

paddycarver opened this issue Jul 7, 2021 · 2 comments · Fixed by #972
Assignees
Labels
enhancement New feature or request subsystem/tests Issues and feature requests related to the testing framework.
Milestone

Comments

@paddycarver
Copy link
Contributor

SDK version

v2.7.0

Use-cases

To be able to test upgrading providers between different versions, it would be helpful if we could specify a different set of providers per test step, not just in the test case.

Attempted Solutions

Proposal

Adding the ProviderFactories, ProtoV5ProviderFactories, ProtoV6ProviderFactories, and ExternalProviders properties from resource.TestCase to resource.TestStep.

References

@paddycarver paddycarver added enhancement New feature or request subsystem/tests Issues and feature requests related to the testing framework. labels Jul 7, 2021
@bflad bflad self-assigned this May 26, 2022
@bflad bflad added this to the v2.17.0 milestone May 26, 2022
bflad added a commit that referenced this issue May 26, 2022
Reference: #253
Reference: #628
Reference: #779

Provider developers can now select whether to configure providers for acceptance testing at the `TestCase` or `TestStep` level. Only one level may be used in this current implementation, however it may be possible to allow merged `TestCase` and `TestStep` configuration with additional validation logic to ensure a single provider is not specified multiple times across the merge result of all fields.

This change also introduces some upfront `TestCase` and `TestStep` configuration validation when calling any of the `Test` functions, failing the test early if a problem is detected. There are other validations that are possible, however these are considered out of scope.
bflad added a commit that referenced this issue May 31, 2022
Reference: #253
Reference: #628
Reference: #779

Provider developers can now select whether to configure providers for acceptance testing at the `TestCase` or `TestStep` level. Only one level may be used in this current implementation, however it may be possible to allow merged `TestCase` and `TestStep` configuration with additional validation logic to ensure a single provider is not specified multiple times across the merge result of all fields.

This change also introduces some upfront `TestCase` and `TestStep` configuration validation when calling any of the `Test` functions, failing the test early if a problem is detected. There are other validations that are possible, however these are considered out of scope.
@bflad
Copy link
Contributor

bflad commented May 31, 2022

The acceptance testing framework (helper/resource) in v2.17.0 now allows provider developers to specify providers at the TestStep level rather than just TestCase. This means that real world state migration/upgrade testing can now be implemented acceptance tests. e.g.

resource.Test(t, resource.TestCase{
	Steps: []resource.TestStep{
		{
			ExternalProviders: map[string]resource.ExternalProvider{
				"random": {
					VersionConstraint: "...", // last version of old schema version
					Source:            "example-namespace/example",
				},
			},
			Config: "...",
			Check:  /* ... */,
		},
		{
			ProviderFactories: testAccProviders,
			Config:            "...",
			Check:             /* ... */,
		},
	},
})

@github-actions
Copy link

github-actions bot commented Jul 1, 2022

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request subsystem/tests Issues and feature requests related to the testing framework.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants