Add google_composer_environment data source #4238

KernelPryanic · 2020-11-16T20:12:34Z

Resolved hashicorp/terraform-provider-google#7826

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

composer: added datasource for `google_composer_environment`

google-cla · 2020-11-16T20:12:39Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

modular-magician · 2020-11-16T20:13:38Z

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@ScottSuarez\melinath, please review this PR or find an appropriate assignee.

KernelPryanic · 2020-11-16T20:17:40Z

@googlebot I signed it!

modular-magician · 2020-11-16T20:20:38Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 167 insertions(+))
Terraform Beta: Diff ( 5 files changed, 167 insertions(+))

nat-henderson · 2020-11-16T21:47:00Z

Looks great, let me get the tests run. /gcbrun.

modular-magician · 2020-11-16T21:47:15Z

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=158233"

modular-magician · 2020-11-16T21:56:03Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 167 insertions(+))
Terraform Beta: Diff ( 5 files changed, 167 insertions(+))

modular-magician · 2020-11-16T23:05:52Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDataSourceComposerEnvironment_basic|TestAccDataSourceSpannerInstance_basic|TestAccApiGatewayApiConfigIamBindingGenerated|TestAccApiGatewayApiConfigIamMemberGenerated|TestAccApiGatewayApiConfigIamPolicyGenerated|TestAccApiGatewayGatewayIamBindingGenerated|TestAccApiGatewayGatewayIamMemberGenerated|TestAccApiGatewayGatewayIamPolicyGenerated|TestAccApiGatewayApiConfig_apigatewayApiConfigBasicExample|TestAccApiGatewayApiConfig_apigatewayApiConfigBasicExampleUpdated|TestAccApiGatewayGateway_apigatewayGatewayBasicExample|TestAccApiGatewayGateway_apigatewayGatewayBasicExampleUpdated|TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample|TestAccBigqueryDataTransferConfig|TestAccComposerEnvironment_withUpdateOnCreate|TestAccFilestoreInstance_filestoreInstanceBasicExample|TestAccFilestoreInstance_update|TestAccLoggingOrganizationSink_described|TestAccLoggingOrganizationSink_disabled|TestAccOSLoginSSHPublicKey_osLoginSshKeyExpiry You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=158242"

KernelPryanic · 2020-11-17T18:46:41Z

@ndmckinley please take a look at the PR if you have spare time :)

nat-henderson · 2020-11-17T20:36:11Z

Were you able to get the new test to pass on your machine?

------- Stdout: -------
=== RUN   TestAccDataSourceComposerEnvironment_basic
=== PAUSE TestAccDataSourceComposerEnvironment_basic
=== CONT  TestAccDataSourceComposerEnvironment_basic
TestAccDataSourceComposerEnvironment_basic: provider_test.go:266: Step 1/1 error: Error running pre-apply refresh: 2020/11/16 23:15:31 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0
Error: "name" ("data_google_composer_environment_test") doesn't match regexp "^(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?)$"
on config710596132/terraform_plugin_test.tf line 3, in data "google_composer_environment" "composer_env":
3:   name = "data_google_composer_environment_test"
--- FAIL: TestAccDataSourceComposerEnvironment_basic (3.56s)
FAIL

modular-magician · 2020-11-18T09:46:53Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 167 insertions(+))
Terraform Beta: Diff ( 5 files changed, 167 insertions(+))

KernelPryanic · 2020-11-18T10:35:58Z

Were you able to get the new test to pass on your machine?

------- Stdout: -------
=== RUN   TestAccDataSourceComposerEnvironment_basic
=== PAUSE TestAccDataSourceComposerEnvironment_basic
=== CONT  TestAccDataSourceComposerEnvironment_basic
TestAccDataSourceComposerEnvironment_basic: provider_test.go:266: Step 1/1 error: Error running pre-apply refresh: 2020/11/16 23:15:31 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0
Error: "name" ("data_google_composer_environment_test") doesn't match regexp "^(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?)$"
on config710596132/terraform_plugin_test.tf line 3, in data "google_composer_environment" "composer_env":
3:   name = "data_google_composer_environment_test"
--- FAIL: TestAccDataSourceComposerEnvironment_basic (3.56s)
FAIL

Oops, sorry, I've changed the name of the test environment after I executed tests on my own environment.

KernelPryanic · 2020-11-23T19:29:08Z

@ndmckinley hey! Could you please take a look when you've got some time?

nat-henderson · 2020-11-23T21:10:35Z

Ah, thanks for the ping! Let me rerun those. /gcbrun

modular-magician · 2020-11-23T21:10:51Z

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=159490"

modular-magician · 2020-11-23T21:19:16Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 167 insertions(+))
Terraform Beta: Diff ( 5 files changed, 167 insertions(+))

modular-magician · 2020-11-23T22:41:21Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDataSourceComposerEnvironment_basic|TestAccDataSourceSpannerInstance_basic|TestAccApiGatewayApiConfigIamBindingGenerated|TestAccApiGatewayApiConfigIamMemberGenerated|TestAccApiGatewayApiConfigIamPolicyGenerated|TestAccApiGatewayGatewayIamBindingGenerated|TestAccApiGatewayGatewayIamMemberGenerated|TestAccApiGatewayGatewayIamPolicyGenerated|TestAccApiGatewayApiConfig_apigatewayApiConfigBasicExample|TestAccApiGatewayApiConfig_apigatewayApiConfigBasicExampleUpdated|TestAccApiGatewayGateway_apigatewayGatewayBasicExample|TestAccApiGatewayGateway_apigatewayGatewayBasicExampleUpdated|TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample|TestAccBigqueryDataTransferConfig|TestAccComposerEnvironment_withUpdateOnCreate|TestAccComputeTargetInstance_targetInstanceCustomNetworkExample|TestAccFilestoreInstance_filestoreInstanceBasicExample|TestAccFilestoreInstance_update|TestAccProjectIamCustomRole_basic|TestAccLoggingOrganizationSink_described|TestAccLoggingOrganizationSink_disabled|TestAccOSLoginSSHPublicKey_osLoginSshKeyExpiry|TestAccSpannerDatabase_basic|TestAccStorageBucket_cors You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=159495"

nat-henderson · 2020-11-23T22:57:52Z

@ndmckinley hey! Could you please take a look when you've got some time?

Hey, sorry:

------- Stdout: -------
=== RUN   TestAccDataSourceComposerEnvironment_basic
=== PAUSE TestAccDataSourceComposerEnvironment_basic
=== CONT  TestAccDataSourceComposerEnvironment_basic
TestAccDataSourceComposerEnvironment_basic: testing_new_config.go:71: no "id" found in attributes
TestAccDataSourceComposerEnvironment_basic: testing_new.go:48: no "id" found in attributes
--- FAIL: TestAccDataSourceComposerEnvironment_basic (7.41s)
FAIL

KernelPryanic · 2020-11-24T19:09:20Z

@ndmckinley hm, that's weird, locally tests are passing

==> Checking source code against gofmt...
==> Checking that code complies with gofmt requirements...
go generate  ./...
TF_ACC=1 TF_SCHEMA_PANIC_ON_ERROR=1 go test ./google -v -run=TestAccDataSourceComposerEnvironment_basic -timeout 240m -ldflags="-X=github.com/hashicorp/terraform-provider-google/version.ProviderVersion=acc"
=== RUN   TestAccDataSourceComposerEnvironment_basic
=== PAUSE TestAccDataSourceComposerEnvironment_basic
=== CONT  TestAccDataSourceComposerEnvironment_basic
--- PASS: TestAccDataSourceComposerEnvironment_basic (6.62s)
PASS
ok  	github.com/hashicorp/terraform-provider-google/google	6.638s

Also I found some weird issue with composer environment creation when I was trying to use data-google-composer-environment-test name for the environment, so I've changed the name of test env once again.

Use of this bucket name is restricted: 'us-central1-data-google-com-cb1e6a07-bucket'

modular-magician · 2020-11-24T19:10:27Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 167 insertions(+))
Terraform Beta: Diff ( 5 files changed, 167 insertions(+))

nat-henderson · 2020-11-24T20:53:17Z

Let's try one more time. If it fails again I'll go ahead and run it locally and try to debug. /gcbrun

modular-magician · 2020-11-24T20:53:31Z

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=159684"

modular-magician · 2020-11-24T21:01:27Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 167 insertions(+))
Terraform Beta: Diff ( 5 files changed, 167 insertions(+))

modular-magician · 2020-11-24T22:14:08Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDataSourceComposerEnvironment_basic|TestAccDataSourceSpannerInstance_basic|TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample|TestAccBigqueryDataTransferConfig|TestAccComposerEnvironment_withUpdateOnCreate|TestAccFilestoreInstance_filestoreInstanceBasicExample|TestAccFilestoreInstance_update|TestAccOSLoginSSHPublicKey_osLoginSshKeyExpiry You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=159689"

nat-henderson · 2020-11-24T23:49:23Z

Same problem unfortunately! I'll clone it locally and see what I can see.

nat-henderson · 2020-11-25T20:16:52Z

Same thing happens on my machine, exact same. Never seen this error before, but it only happens on your PR. The tests are passing for you?

nat-henderson · 2020-11-25T23:57:50Z

Okay, our theory is that the issue is that the config does not create the resource that the data source reads. Take a look at https://github.com/GoogleCloudPlatform/magic-modules/blob/master/third_party/terraform/tests/data_source_cloud_run_service_test.go#L53, for example, where the data source reads from a resource that is also created. We think it's passing for you since that resource exists in your environment (but not in ours), and we think the error message is just confusing and if that resolves your problem we'll ask HashiCorp to edit it to make it clearer.

KernelPryanic · 2020-11-26T23:11:48Z

@ndmckinley Gotcha, didn't know how they are hooked up, thank you! Updated the PR.

modular-magician · 2020-11-26T23:20:06Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 202 insertions(+))
Terraform Beta: Diff ( 5 files changed, 202 insertions(+))

modular-magician · 2020-11-27T00:00:06Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 208 insertions(+))
Terraform Beta: Diff ( 5 files changed, 208 insertions(+))

KernelPryanic · 2020-11-30T18:36:53Z

@ndmckinley hey) Could you please take a look and run /gcbrun if you've got a time?

nat-henderson · 2020-11-30T18:39:41Z

Hi there - /gcbrun - thanks, we were off for the thanksgiving holiday. :)

modular-magician · 2020-11-30T18:39:56Z

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=160653"

KernelPryanic · 2020-11-30T18:46:31Z

Ah, right, happy belated thanksgiving! :)

modular-magician · 2020-11-30T18:47:55Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 208 insertions(+))
Terraform Beta: Diff ( 5 files changed, 208 insertions(+))

nat-henderson · 2020-11-30T19:06:41Z

Looks good to me!

modular-magician · 2020-11-30T19:59:10Z

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccDataSourceComposerEnvironment_basic|TestAccDataSourceSpannerInstance_basic|TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample|TestAccBigqueryDataTransferConfig|TestAccComposerEnvironment_withUpdateOnCreate|TestAccContainerNodePool_basic|TestAccContainerNodePool_nodeLocations|TestAccContainerNodePool_maxPodsPerNode|TestAccContainerNodePool_withInvalidUpgradeSettings|TestAccContainerNodePool_withNodeConfig|TestAccContainerNodePool_withKubeletConfig|TestAccContainerNodePool_withGPU|TestAccContainerNodePool_withLinuxNodeConfig|TestAccContainerNodePool_withNodeConfigScopeAlias|TestAccContainerNodePool_withUpgradeSettings|TestAccContainerNodePool_regionalAutoscaling|TestAccContainerNodePool_withWorkloadIdentityConfig|TestAccContainerNodePool_withSandboxConfig|TestAccContainerNodePool_withManagement|TestAccContainerNodePool_regionalClusters|TestAccContainerNodePool_resize|TestAccContainerNodePool_shieldedInstanceConfig|TestAccContainerNodePool_autoscaling|TestAccContainerNodePool_012_ConfigModeAttr|TestAccContainerNodePool_EmptyGuestAccelerator|TestAccFilestoreInstance_filestoreInstanceBasicExample|TestAccFilestoreInstance_update|TestAccOSLoginSSHPublicKey_osLoginSshKeyExpiry You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=160656"

Add google_composer_environment data source

13d4aea

google-cla bot added the cla: no label Nov 16, 2020

google-cla bot added cla: yes and removed cla: no labels Nov 16, 2020

KernelPryanic mentioned this pull request Nov 16, 2020

Add google_composer_environment data source hashicorp/terraform-provider-google#7794

Closed

nat-henderson self-requested a review November 16, 2020 21:46

Fix test environment name

42afd8f

Fix test environment name

ef5d8b2

Fix tests

097537e

Update docs

79908a0

nat-henderson approved these changes Nov 30, 2020

View reviewed changes

nat-henderson merged commit 9ec557d into GoogleCloudPlatform:master Nov 30, 2020

This was referenced Nov 30, 2020

Add google_composer_environment data source hashicorp/terraform-provider-google-beta#2745

Merged

Add google_composer_environment data source hashicorp/terraform-provider-google#7902

Merged

Add google_composer_environment data source #4238

Add google_composer_environment data source #4238

Conversation

KernelPryanic commented Nov 16, 2020 • edited Loading

google-cla bot commented Nov 16, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

modular-magician commented Nov 16, 2020

KernelPryanic commented Nov 16, 2020

modular-magician commented Nov 16, 2020

Diff report:

nat-henderson commented Nov 16, 2020

modular-magician commented Nov 16, 2020

modular-magician commented Nov 16, 2020

Diff report:

modular-magician commented Nov 16, 2020

KernelPryanic commented Nov 17, 2020

nat-henderson commented Nov 17, 2020

modular-magician commented Nov 18, 2020

Diff report:

KernelPryanic commented Nov 18, 2020 • edited Loading

KernelPryanic commented Nov 23, 2020

nat-henderson commented Nov 23, 2020

modular-magician commented Nov 23, 2020

modular-magician commented Nov 23, 2020

Diff report:

modular-magician commented Nov 23, 2020

nat-henderson commented Nov 23, 2020

KernelPryanic commented Nov 24, 2020 • edited Loading

modular-magician commented Nov 24, 2020

Diff report:

nat-henderson commented Nov 24, 2020

modular-magician commented Nov 24, 2020

modular-magician commented Nov 24, 2020

Diff report:

modular-magician commented Nov 24, 2020

nat-henderson commented Nov 24, 2020

nat-henderson commented Nov 25, 2020

nat-henderson commented Nov 25, 2020

KernelPryanic commented Nov 26, 2020

modular-magician commented Nov 26, 2020

Diff report:

modular-magician commented Nov 27, 2020

Diff report:

KernelPryanic commented Nov 30, 2020

nat-henderson commented Nov 30, 2020

modular-magician commented Nov 30, 2020

KernelPryanic commented Nov 30, 2020 • edited Loading

modular-magician commented Nov 30, 2020

Diff report:

nat-henderson commented Nov 30, 2020

modular-magician commented Nov 30, 2020

KernelPryanic commented Nov 16, 2020 •

edited

Loading

KernelPryanic commented Nov 18, 2020 •

edited

Loading

KernelPryanic commented Nov 24, 2020 •

edited

Loading

KernelPryanic commented Nov 30, 2020 •

edited

Loading