Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Track2 Poller crash #14729

Closed
yupwei68 opened this issue Jun 1, 2021 · 8 comments
Closed

Track2 Poller crash #14729

yupwei68 opened this issue Jun 1, 2021 · 8 comments
Assignees
@jhendrixMSFT
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved.

Comments

@yupwei68
Copy link
Contributor

yupwei68 commented Jun 1, 2021

Bug Report

  • import path of package in question, e.g. .../services/compute/mgmt/2018-06-01/compute
  • SDK version e.g. master, latest, 18.1.0
    • Specify the exact commit if possible; one way to get this is the REVISION
      column output by dep status "github.com/Azure/azure-sdk-for-go.
  • output of go version

  • What happened?
    When trying track2, the plugin crash on poller.

  • What did you expect or want to happen?
    No crash

  • How can we reproduce it?
    Use track2 sdk built Terrform plugin to create a resource avs, and after 1.5 hrs, it crashed.

  • Anything we should know about your environment.

It crashed at https://github.com/Azure/azure-sdk-for-go/blob/master/sdk/armcore/poller.go#L198, looking alike it's a nil pointer problem.

When it sends request and receives no response, it crashed.

[DEBUG] Request to https://management.azure.com/subscriptions/.../resourceGroups/acctestRG-Vmware-yup/providers/Microsoft.AVS/privateClouds/acctest-PC-yup/operationstatuses/1be210d6-d357-42a0-bf7e-f8da4de108ea?api-version=2020-03-20 completed with no response

crash log:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xe86f04]

goroutine 75 [running]:
github.com/Azure/azure-sdk-for-go/sdk/armcore.(*poller).Poll(0xc0010aafb0, 0x5edd618, 0xc0016ae180, 0xc0016e0200, 0x9, 0x10, 0x1, 0x1, 0x0)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/Azure/azure-sdk-for-go/sdk/armcore/poller.go:198 +0xc4
github.com/Azure/azure-sdk-for-go/sdk/armcore.(*poller).PollUntilDone(0xc0010aafb0, 0x5edd618, 0xc0016ae180, 0x77359400, 0xc0016e0200, 0x9, 0x10, 0x4e53660, 0xc001180480, 0xc0003a80e0, ...)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/Azure/azure-sdk-for-go/sdk/armcore/poller.go:222 +0x17e
github.com/Azure/azure-sdk-for-go/sdk/arm/avs/2020-03-20/armavs.(*privateCloudPoller).pollUntilDone(0xc00168d140, 0x5edd618, 0xc0016ae180, 0x77359400, 0xc00168d140, 0xc0016e0200, 0xc0010aafb0, 0x5ee4698)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/Azure/azure-sdk-for-go/sdk/arm/avs/2020-03-20/armavs/zz_generated_pollers.go:182 +0xa4
github.com/Azure/azure-sdk-for-go/sdk/arm/avs/2020-03-20/armavs.(*PrivateCloudsClient).BeginCreateOrUpdate.func1(0x5edd618, 0xc0016ae180, 0x77359400, 0xc0011137a0, 0x14, 0xc001128ab0, 0xe)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/Azure/azure-sdk-for-go/sdk/arm/avs/2020-03-20/armavs/zz_generated_privateclouds_client.go:52 +0x48
github.com/terraform-providers/terraform-provider-azurerm/azurerm/internal/services/vmware.resourceVmwarePrivateCloudCreate(0xc000d0bd50, 0x5065c60, 0xc00030c300, 0x0, 0x0)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/azurerm/internal/services/vmware/vmware_private_cloud_resource.go:249 +0xb6f
github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).Apply(0xc000fad4d0, 0xc00148bd10, 0xc00184b4e0, 0x5065c60, 0xc00030c300, 0x5092901, 0xc001671298, 0xc0015a5260)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/resource.go:310 +0x375
github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Provider).Apply(0xc0000b1880, 0xc0012f3a38, 0xc00148bd10, 0xc00184b4e0, 0xc001671228, 0xc000b418b0, 0x50950e0)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/provider.go:294 +0x99
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ApplyResourceChange(0xc0000104f0, 0x5edd650, 0xc0010b8990, 0xc000d0b880, 0xc0000104f0, 0xc0010b8990, 0xc0014c9ba0)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin/grpc_provider.go:885 +0x8a5
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x5539480, 0xc0000104f0, 0x5edd650, 0xc0010b8990, 0xc001119ec0, 0x0, 0x5edd650, 0xc0010b8990, 0xc001694500, 0x49f)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5/tfplugin5.pb.go:3305 +0x214
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000186540, 0x5f20118, 0xc0003bd680, 0xc0016e1400, 0xc0000ba540, 0x90b4dc0, 0x0, 0x0, 0x0)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:1180 +0x52b
google.golang.org/grpc.(*Server).handleStream(0xc000186540, 0x5f20118, 0xc0003bd680, 0xc0016e1400, 0x0)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:1503 +0xd0c
google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc000bece00, 0xc000186540, 0x5f20118, 0xc0003bd680, 0xc0016e1400)
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:843 +0xab
created by google.golang.org/grpc.(*Server).serveStreams.func1
/home/weiyuping/work/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:841 +0x1fd

Error: The terraform-provider-azurerm plugin crashed!
@ghost ghost added the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jun 1, 2021
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jun 1, 2021
@ArcturusZhang ArcturusZhang added bug This issue requires a change to an existing behavior in the product in order to be resolved. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Jun 1, 2021
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jun 1, 2021
@ArcturusZhang
Copy link
Member

Thanks @yupwei68 for this issue!

This looks weird. How could we know a request was complete with no response?

Hi @jhendrixMSFT could you please take a look at this issue?

@jhendrixMSFT jhendrixMSFT self-assigned this Jun 1, 2021
@jhendrixMSFT jhendrixMSFT mentioned this issue Jun 2, 2021
Closed
5 tasks
@jhendrixMSFT
Copy link
Member

jhendrixMSFT commented Jun 2, 2021 •
edited
Loading

What version of azcore are you using?

Generally speaking, when opening issues, it would be great to include link(s) to the fork/branch of the code you're using.

@yupwei68
Copy link
Contributor Author

yupwei68 commented Jun 3, 2021

azcore is v0.16.0, the branch link is here https://github.com/yupwei68/terraform-provider-azurerm/blob/storage-track2/go.sum

@jhendrixMSFT
Copy link
Member

As an aside, I don't think your logging policy accurately distinguishes between a nil response and an error response. I think it should look something like this.

if resp != nil {
	// dump response to wire format
	if dump, err2 := httputil.DumpResponse(resp.Response, true); err2 == nil {
		log.Printf("[DEBUG] %s Response for %s: \n%s\n", p.providerName, r.URL, dump)
	} else {
		// fallback to basic message
		log.Printf("[DEBUG] %s Response: %s for %s\n", p.providerName, resp.Status, r.URL)
	}
} else if err != nil {
	// log error....
} else {
	log.Printf("[DEBUG] Request to %s completed with no response and no error", r.URL)
}

@yupwei68
Copy link
Contributor Author

yupwei68 commented Jun 4, 2021

Thanks @jhendrixMSFT I think it's to be fixed in track2.

The usage to crash is here https://github.com/yupwei68/terraform-provider-azurerm/blob/storage-track2/azurerm/internal/services/vmware/vmware_private_cloud_resource.go#L249 .

You could confirm if our usage is correct.

@jhendrixMSFT
Copy link
Member

Your usage is fine, this is a bug in our polling logic that I hope to have fixed soon.

Are you saying you'll be fixing your logging policy in track 2?

@yupwei68
Copy link
Contributor Author

yupwei68 commented Jun 4, 2021

Yes

@jhendrixMSFT jhendrixMSFT mentioned this issue Jun 7, 2021
Merged
5 tasks
@jhendrixMSFT
Copy link
Member

This has been fixed. You'll need to rebuild the affected modules with v4.0.0-preview.22 of the code generator.

@yupwei68 yupwei68 mentioned this issue Jun 15, 2021
Merged
@RickWinter RickWinter removed the Track2 label Sep 13, 2021
@github-actions github-actions bot locked and limited conversation to collaborators Apr 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jhendrixMSFT jhendrixMSFT

Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@RickWinter @jhendrixMSFT @ArcturusZhang @yupwei68