-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Standard Integration Or Reference For Use With Go VCR #190
Comments
Hi @tgoodsell-tempus 👋 Thank you for raising this feature request. It would certainly be useful if there was additional documentation about the general pattern of using a mock API client for provider acceptance testing. To help with the testing code related portion of your proposal it might help to provide a code snippet to highlight what you think can/should be done in this regard. One important thing to note is that newer testing code ( |
Hi @bflad, Yes this is a bit of a odd / difficult one to think about concretely. I thought up of a theoretical example based on overriding the import (
"context"
"fmt"
"net/http"
"testing"
"github.com/hashicorp/terraform-plugin-testing/helper/resource"
fwresource "github.com/hashicorp/terraform-plugin-framework/resource"
)
type OverideResource interface {
Configure(ctx context.Context, req fwresource.ConfigureRequest, resp *fwresource.ConfigureResponse)
}
type OverideResourceImpl struct {
ExampleResource
}
func (r *OverideResourceImpl) Configure(ctx context.Context, req fwresource.ConfigureRequest, resp *fwresource.ConfigureResponse) {
// Prevent panic if the provider has not been configured.
if req.ProviderData == nil {
return
}
client, ok := req.ProviderData.(*http.Client)
client.Transport = &http.Transport{} // EXAMPLE REPLACEMENT OF TRANSPORT WITH MOCK OR OTHER CUSTOM TRANSPORT
if !ok {
resp.Diagnostics.AddError(
"Unexpected Resource Configure Type",
fmt.Sprintf("Expected *http.Client, got: %T. Please report this issue to the provider developers.", req.ProviderData),
)
return
}
r.client = client
} Then perhaps have a new field in the func TestAccExampleResource(t *testing.T) {
resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,
// Replace the instance of the resource with the OverideResourceImpl used for this test
// ResourceOverride: map[string]fwresource.Resource{
// "scaffolding_example": &OverideResourceImpl{},
// },
Steps: []resource.TestStep{
// Create and Read testing
{
Config: testAccExampleResourceConfig("one"),
Check: resource.ComposeAggregateTestCheckFunc(
resource.TestCheckResourceAttr("scaffolding_example.test", "configurable_attribute", "one"),
resource.TestCheckResourceAttr("scaffolding_example.test", "defaulted", "example value when not configured"),
resource.TestCheckResourceAttr("scaffolding_example.test", "id", "example-id"),
),
},
// ImportState testing
{
ResourceName: "scaffolding_example.test",
ImportState: true,
ImportStateVerify: true,
// This is not normally necessary, but is here because this
// example code does not have an actual upstream service.
// Once the Read method is able to refresh information from
// the upstream service, this can be removed.
ImportStateVerifyIgnore: []string{"configurable_attribute", "defaulted"},
},
// Update and Read testing
{
Config: testAccExampleResourceConfig("two"),
Check: resource.ComposeAggregateTestCheckFunc(
resource.TestCheckResourceAttr("scaffolding_example.test", "configurable_attribute", "two"),
),
},
// Delete testing automatically occurs in TestCase
},
})
} While I came up with this at the |
terraform-plugin-testing version
Use cases
Running acceptance tests requires access to a live provider which involves making API calls that may or may not cost a lot of money. Additionally, some platforms which offer or have had Terraform providers built, may have certain features or functions locked behind flags or paywall gates that aren't accessible to common users/developers.
Providers, such as
terraform-provider-google
andterraform-provider-okta
, have implemented the use of Go VCR, to allow for cheaper, more efficient, and a more accessible contributor experience. Other providers have implemented other tools as well, some more specific to the particular API/service being modeled in Terraform.Attempted solutions
This has already been accomplished in a ad-hoc manner in a variety of "similar" but also slightly different ways:
Configure
function for "framework" and theConfigureContextFunc
to handle this HTTP configuration. Which then are passed into the variousProtoProviderFactories
inputs on the the test case. This must be done since each test case will start a new instance of the configuration / providers by convention.For the most part, this works out decently well provided the SDK being used is nice about replacing the HTTP client it uses, as well as good code cleanliness and standards. However, in it's current form, this does require a pretty significant effort and self-motivated desire to get this done.
Proposal
My personal statement would be that the benefits of a VCR/playback based acceptance test system are significant enough that the testing framework for providers should attempt to offer some level of "reduced boilerplate" tools to making the necessary changes during acceptance tests and some level of "encouragement" to include these types of acceptance tests tools in the provider.
My proposal for this would be.
References
Google provider: https://github.com/hashicorp/terraform-provider-google/blob/main/google/acctest/vcr_utils.go
Okta provider: https://github.com/okta/terraform-provider-okta/blob/master/okta/provider_test.go
The text was updated successfully, but these errors were encountered: