Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

compute/metadata: retryer tries to get the status code from the resource that does not exist #4713

Closed
lukatendai opened this issue Sep 1, 2021 · 3 comments · Fixed by #4714
Closed

compute/metadata: retryer tries to get the status code from the resource that does not exist #4713

lukatendai opened this issue Sep 1, 2021 · 3 comments · Fixed by #4714
Assignees
@codyoss
Labels
api: compute Issues related to the Compute Engine API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. status: investigating The issue is under investigation, which is determined to be non-trivial. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@lukatendai
Copy link

In/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go on line 318

if delay, shouldRetry := retryer.Retry(res.StatusCode, err); shouldRetry {
	if err := gax.Sleep(ctx, delay); err != nil {
		return "", "", err
	}
	continue
}

status code wants to be retrived from "res" but in case of an error res is nil and it panics

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xa5a9b7]

goroutine 1 [running]:
cloud.google.com/go/compute/metadata.(*Client).getETag(0x24eba88, 0xc00011c480, 0x20, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go:318 +0x417
cloud.google.com/go/compute/metadata.(*Client).Get(...)
/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go:349
cloud.google.com/go/compute/metadata.(*Client).InstanceAttributeValue(0x24eba88, 0x172e0b2, 0xc, 0x0, 0x0, 0x203000, 0x60)
/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go:454 +0x79
cloud.google.com/go/compute/metadata.InstanceAttributeValue(...)
/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go:248
cloud.google.com/go/logging.isKubernetesEngine(0x40f800)
/go/pkg/mod/cloud.google.com/go/logging@v1.4.2/resource.go:143 +0x45
cloud.google.com/go/logging.detectResource.func1()
/go/pkg/mod/cloud.google.com/go/logging@v1.4.2/resource.go:222 +0xd1
sync.(*Once).doSlow(0x251a1d8, 0x17af868)
/usr/local/go/src/sync/once.go:68 +0xec
sync.(*Once).Do(...)
/usr/local/go/src/sync/once.go:59
cloud.google.com/go/logging.detectResource(0xc00039fd48)
/go/pkg/mod/cloud.google.com/go/logging@v1.4.2/resource.go:212 +0x59
cloud.google.com/go/logging.(*Client).Logger(0xc000289080, 0x1718138, 0x2, 0x0, 0x0, 0x0, 0x2)
/go/pkg/mod/cloud.google.com/go/logging@v1.4.2/logging.go:347 +0x34
bitbucket.org/endai/gobg/modules/bg/logging.Init(0x1d36e78, 0xc00053c340, 0x171ebda, 0x6)
/opt/atlassian/pipelines/agent/build/modules/bg/logging/logging.go:27 +0x208
main.main()
/opt/atlassian/pipelines/agent/build/bg.go:46 +0x2bd
time="Sep 1 11:23:48.513" level=info msg="BG process [585942] starting at 2021-09-01 15:23:48.505785215 +0000 UTC UTC"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xa5a9b7]

goroutine 1 [running]:
cloud.google.com/go/compute/metadata.(*Client).getETag(0x24eba88, 0xc00056e5a0, 0x20, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go:318 +0x417
cloud.google.com/go/compute/metadata.(*Client).Get(...)
/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go:349
cloud.google.com/go/compute/metadata.(*Client).InstanceAttributeValue(0x24eba88, 0x172e0b2, 0xc, 0x0, 0x0, 0x203000, 0x60)
/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go:454 +0x79
cloud.google.com/go/compute/metadata.InstanceAttributeValue(...)
/go/pkg/mod/cloud.google.com/go@v0.94.0/compute/metadata/metadata.go:248
cloud.google.com/go/logging.isKubernetesEngine(0x40f800)
/go/pkg/mod/cloud.google.com/go/logging@v1.4.2/resource.go:143 +0x45
cloud.google.com/go/logging.detectResource.func1()
/go/pkg/mod/cloud.google.com/go/logging@v1.4.2/resource.go:222 +0xd1
sync.(*Once).doSlow(0x251a1d8, 0x17af868)
/usr/local/go/src/sync/once.go:68 +0xec
sync.(*Once).Do(...)
/usr/local/go/src/sync/once.go:59
cloud.google.com/go/logging.detectResource(0xc0003b7d48)
/go/pkg/mod/cloud.google.com/go/logging@v1.4.2/resource.go:212 +0x59
cloud.google.com/go/logging.(*Client).Logger(0xc0005453e0, 0x1718138, 0x2, 0x0, 0x0, 0x0, 0x2)
/go/pkg/mod/cloud.google.com/go/logging@v1.4.2/logging.go:347 +0x34
bitbucket.org/endai/gobg/modules/bg/logging.Init(0x1d36e78, 0xc0001fc140, 0x171ebda, 0x6)
/opt/atlassian/pipelines/agent/build/modules/bg/logging/logging.go:27 +0x208
main.main()
/opt/atlassian/pipelines/agent/build/bg.go:46 +0x2bd
@lukatendai lukatendai added the triage me I really want to be triaged. label Sep 1, 2021
@codyoss codyoss changed the title retryer tries to get the status code from the resource that does not exist compute/metadata: retryer tries to get the status code from the resource that does not exist Sep 1, 2021
@codyoss codyoss added status: investigating The issue is under investigation, which is determined to be non-trivial. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. and removed triage me I really want to be triaged. labels Sep 1, 2021
@codyoss codyoss self-assigned this Sep 1, 2021
@codyoss codyoss added the priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. label Sep 1, 2021
@codyoss codyoss mentioned this issue Sep 1, 2021
Merged
@codyoss
Copy link
Member

codyoss commented Sep 1, 2021

@lukatendai Thank you for the detailed report! I have raised a PR to fix this issue. Once merged I will cut a new release to fix this.

@product-auto-label product-auto-label bot added the api: compute Issues related to the Compute Engine API. label Sep 2, 2021
gcf-merge-on-green bot pushed a commit that referenced this issue Sep 2, 2021
@codyoss
fix(compute/metadata): fix retry logic to not panic on error (#4714)
75c63b9 
In the case of an error res will be nil. Also, we should not break on nil err
as it might be eligible for a retry from the status code.

Fixes: #4713
@codyoss
Copy link
Member

codyoss commented Sep 2, 2021

Released in v0.94.1. Thanks again for the quick and detailed report!

@lukatendai
Copy link
Author

lukatendai commented Sep 2, 2021 via email

@Deiz Deiz mentioned this issue Nov 2, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@codyoss codyoss

Labels
api: compute Issues related to the Compute Engine API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. status: investigating The issue is under investigation, which is determined to be non-trivial. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(compute/metadata): fix retry logic to not panic on error
2 participants
@lukatendai @codyoss