Add skip_wait_on_job_termination option for dataflow job resources

[n-oden]

This addresses #10559

See also hashicorp/terraform-provider-google-beta#3868

[n-oden]

@ScottSuarez Happy new year! Did you have any thoughts on this one?

[ScottSuarez]

I think I understand the use case but could you elaborate or go into more detail on your change?

[n-oden]

@ScottSuarez #10559 goes into a lot more detail; the 5s summary is "sometimes draining streaming dataflow jobs takes a ridiculously long time" -- minutes to hours depending on what sort of internal windowing/watermark strategy your job is using. This means that potentially your terraform workspace is locked for that entire time while waiting for the job to drain.

With this in place, operators will have the option of waiting only until the job reaches the DRAINING state before moving on. Obviously this has the caveat that if you try to create a replacement job with the same name before the job actually exits it will fail, hence the digression about random_id in the doc examples.

[n-oden]

@rileykarson happy new year! Thoughts on this?

[n-oden]

@ScottSuarez @rileykarson nudge? If this isn't an approach to #10559 that you like, please let me know.

[ScottSuarez]

taking over review here. I'll give this a look over soon

[ScottSuarez]

Could you also add a test case to the testsuite for this resource and ensure it passes

https://github.com/hashicorp/terraform-provider-google/blob/main/google/resource_dataflow_job_test.go

[ScottSuarez]

This logic is repeated several times. Can you extrapolate this to a function

func shouldStopDataflowJobDeleteQuery(state string, skipWait bool) bool {
	_, stopQuery := dataflowTerminalStatesMap[state]
	if !stopQuery && skipWait {
		_, stopQuery = dataflowTerminatingStatesMap[d.Get("state").(string)]
	}
	return stopQuery
}

[n-oden]

All reasonable requests! I'll try to get it sorted out both here and in the beta-provider PR on monday or tuesday.

[ScottSuarez]

Can you make the changes against the upstream repository https://github.com/GoogleCloudPlatform/magic-modules

We use this to generate the two versions of the provider. Tag me on the PR when you open it

[n-oden]

continued in GoogleCloudPlatform/magic-modules#5844

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.