Add status to bigquery_job

[kanterov]

Fixes hashicorp/terraform-provider-google#8357

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

bigquery: added `status` field to `google_bigquery_job`

[google-cla]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[modular-magician]

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@slevenick, please review this PR or find an appropriate assignee.

[kanterov]

Not sure if this is needed. Because of that terraform generator doesn't set maxItems: 1

[slevenick]

I believe this is needed because the user cannot set the value of the status object. output: true is not necessarily needed on the sub-fields of this object, as if the top-level is output only then all sub fields will be as well

[kanterov]

@googlebot I signed it!

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 138 insertions(+))
Terraform Beta: Diff ( 2 files changed, 138 insertions(+))

[modular-magician]

Oops! It looks like you're using an unknown release-note type in your changelog entries:

• REPLACEME

Please only use the types listed in https://github.com/GoogleCloudPlatform/magic-modules/blob/master/.ci/RELEASE_NOTES_GUIDE.md.

[slevenick]

/gcbrun

Rerunning jobs as release note is fixed now

[slevenick]

I believe this is needed because the user cannot set the value of the status object. output: true is not necessarily needed on the sub-fields of this object, as if the top-level is output only then all sub fields will be as well

[slevenick]

/gcbrun

Apparently review comments don't trigger this

[modular-magician]

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=170550"

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 138 insertions(+))
Terraform Beta: Diff ( 2 files changed, 138 insertions(+))

[modular-magician]

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample|TestAccBigQueryJob_bigqueryJobQueryTableReferenceExample|TestAccBigQueryJob_bigqueryJobQueryExample|TestAccBigQueryJob_bigqueryJobLoadExample|TestAccBigQueryJob_withLocation|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthorityCmekExample You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=170552"

[kanterov]

I fixed my setup so I can run tests locally and now I can see what's wrong. Fixing it now and going to update the code soon.

[slevenick]

Looks like this causes a panic during the bigquery tests:

panic: status.0.error_result: '' expected type 'string', got unconvertible type 'map[string]interface {}'

Looks like the errorResult is this structure: https://cloud.google.com/bigquery/docs/reference/rest/v2/ErrorProto

rather than just a string

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 197 insertions(+))
Terraform Beta: Diff ( 2 files changed, 197 insertions(+))

[kanterov]

/gcbrun

[kanterov]

@slevenick sorry, totally confused it. Now tests pass locally, is it possible to re-trigger GCB?

[slevenick]

/gcbrun

[modular-magician]

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=170555"

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 197 insertions(+))
Terraform Beta: Diff ( 2 files changed, 197 insertions(+))

[modular-magician]

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample|TestAccBigQueryJob_bigqueryJobLoadExample|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthorityCmekExample You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=170640"

[kanterov]

I can't reproduce the failure of TestAccBigQueryJob_bigqueryJobLoadExample locally, is there any way to see the logs?

TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample and TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthorityCmekExample seem to be failing in the main branch as well.

[slevenick]

I can't reproduce the failure of TestAccBigQueryJob_bigqueryJobLoadExample locally, is there any way to see the logs?

TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample and TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthorityCmekExample seem to be failing in the main branch as well.

Yeah, the other two tests are unrelated to your changes, ignore them.

I'm still seeing this failure for the bigquery job:

TestAccBigQueryJob_bigqueryJobLoadExample: provider_test.go:270: ImportStateVerify attributes not equivalent. Difference is shown below. Top is actual, bottom is expected.
(map[string]string) (len=1) {
(string) (len=14) "status.0.state": (string) (len=4) "DONE"
}
(map[string]string) (len=1) {
(string) (len=14) "status.0.state": (string) (len=7) "RUNNING"
}

it looks like the job status is changing between creating the job & the import step running. This may be flaky due to the time it takes the job to finish. We probably want to ignore the status when running the import step due to this, as it should never be set by the user.

You can add status.0.state to the ImportStateVerifyIgnore for the bigquery job tests and it should fix this

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 206 insertions(+), 9 deletions(-))
Terraform Beta: Diff ( 5 files changed, 207 insertions(+), 10 deletions(-))

[kanterov]

@slevenick thanks for the clarifications, I've updated tests, including auto-generated ones. Should we /gcbrun?

[slevenick]

/gcbrun

[modular-magician]

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=170713"

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 206 insertions(+), 9 deletions(-))
Terraform Beta: Diff ( 5 files changed, 207 insertions(+), 10 deletions(-))

[modular-magician]

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthorityCmekExample You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=170729"

[slevenick]

Looks good to me! Thanks

[kanterov]

@slevenick thanks a lot for your support, I learned a lot!