Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aws: Retrieve instance info only when max pods is not set #16216

Merged
merged 2 commits into from
Jan 4, 2024
Merged

aws: Retrieve instance info only when max pods is not set #16216

merged 2 commits into from
Jan 4, 2024

Conversation

hakman
Copy link
Member

@hakman hakman commented Jan 4, 2024 •
edited
Loading

/cc @zetaab @rifelpet

https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kops/16204/presubmit-kops-aws-scale-amazonvpc-using-cl2/1742822642840244224/artifacts/i-0005fc01059a34f29/kops-configuration.log

I0104 08:28:05.708287    1346 request_logger.go:45] AWS request: ec2/DescribeInstances
I0104 08:28:05.833709    1346 request_logger.go:45] AWS request: ec2/DescribeInstanceTypes
W0104 08:28:06.034361    1346 retry_handler.go:99] Got RequestLimitExceeded error on AWS request (ec2::DescribeInstanceTypes)
I0104 08:28:06.034430    1346 logging_retryer.go:60] Retryable error (RequestLimitExceeded: Request limit exceeded.
	status code: 503, request id: 7bd37a68-e0a7-4edc-b5aa-ccd2f5d144a1) from ec2/DescribeInstanceTypes - will retry after delay of 854.685499ms
W0104 08:28:06.889975    1346 retry_handler.go:55] Inserting delay before AWS request (ec2::DescribeInstanceTypes) to avoid RequestLimitExceeded: 8s
I0104 08:28:14.894278    1346 request_logger.go:45] AWS request: ec2/DescribeInstanceTypes
W0104 08:28:15.061892    1346 retry_handler.go:99] Got RequestLimitExceeded error on AWS request (ec2::DescribeInstanceTypes)
I0104 08:28:15.061944    1346 logging_retryer.go:60] Retryable error (RequestLimitExceeded: Request limit exceeded.
	status code: 503, request id: 0aef4c53-2951-4b51-a506-951f43927e93) from ec2/DescribeInstanceTypes - will retry after delay of 1.742493586s
W0104 08:28:16.804677    1346 retry_handler.go:55] Inserting delay before AWS request (ec2::DescribeInstanceTypes) to avoid RequestLimitExceeded: 15s
I0104 08:28:31.805061    1346 request_logger.go:45] AWS request: ec2/DescribeInstanceTypes
W0104 08:28:31.953427    1346 retry_handler.go:99] Got RequestLimitExceeded error on AWS request (ec2::DescribeInstanceTypes)
I0104 08:28:31.953480    1346 logging_retryer.go:60] Retryable error (RequestLimitExceeded: Request limit exceeded.
	status code: 503, request id: 45548340-1449-45f0-8e29-5411d3fb19f7) from ec2/DescribeInstanceTypes - will retry after delay of 3.780755776s
W0104 08:28:35.735176    1346 retry_handler.go:55] Inserting delay before AWS request (ec2::DescribeInstanceTypes) to avoid RequestLimitExceeded: 19s
I0104 08:28:54.735527    1346 request_logger.go:45] AWS request: ec2/DescribeInstanceTypes
W0104 08:28:54.865307    1346 retry_handler.go:99] Got RequestLimitExceeded error on AWS request (ec2::DescribeInstanceTypes)
I0104 08:28:54.865359    1346 logging_retryer.go:60] Retryable error (RequestLimitExceeded: Request limit exceeded.
	status code: 503, request id: 208c1745-e41f-43bb-89c4-5b2f67bd11d5) from ec2/DescribeInstanceTypes - will retry after delay of 7.848118296s
W0104 08:29:02.715293    1346 retry_handler.go:55] Inserting delay before AWS request (ec2::DescribeInstanceTypes) to avoid RequestLimitExceeded: 23s
I0104 08:29:25.719757    1346 request_logger.go:45] AWS request: ec2/DescribeInstanceTypes
W0104 08:29:25.900643    1346 retry_handler.go:99] Got RequestLimitExceeded error on AWS request (ec2::DescribeInstanceTypes)
I0104 08:29:25.900707    1346 logging_retryer.go:60] Retryable error (RequestLimitExceeded: Request limit exceeded.
	status code: 503, request id: a78ff931-1b7c-4fca-bdac-f9dde8aa6aa1) from ec2/DescribeInstanceTypes - will retry after delay of 15.35870848s
W0104 08:29:41.260398    1346 retry_handler.go:55] Inserting delay before AWS request (ec2::DescribeInstanceTypes) to avoid RequestLimitExceeded: 26s
I0104 08:30:07.261188    1346 request_logger.go:45] AWS request: ec2/DescribeInstanceTypes

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/nodeup labels Jan 4, 2024
@hakman
Copy link
Member Author

hakman commented Jan 4, 2024

/retest

@hakman
Copy link
Member Author

hakman commented Jan 4, 2024

/retest

@hakman
Copy link
Member Author

hakman commented Jan 4, 2024

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 4, 2024
@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 4, 2024
hakman
hakman commented Jan 4, 2024
View reviewed changes
nodeup/pkg/model/kubelet.go Outdated Show resolved Hide resolved
nodeup/pkg/model/kubelet.go Outdated Show resolved Hide resolved
nodeup/pkg/model/kubelet.go Outdated Show resolved Hide resolved
nodeup/pkg/model/kubelet.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 4, 2024
justinsb
justinsb reviewed Jan 4, 2024
View reviewed changes
nodeup/pkg/model/kubelet.go
})
if err != nil {
return nil, fmt.Errorf("error describing instances: %v", err)
if localHostname, err := metadata.GetMetadata("local-hostname"); err == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to have an issue (IIRC) where local-hostname did not match PrivateDNSName, when users had a VPC with a custom DNS suffix. However, I think that we're now OK because (1) we use the instance ID as the node name now, rather than PrivateDNSName and (2) kubeletNames seems to mostly be ignored now anyway, as we usually issue the cert via kops-controller.

@justinsb
Copy link
Member

justinsb commented Jan 4, 2024

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 4, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: justinsb, rifelpet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hakman
Copy link
Member Author

hakman commented Jan 4, 2024 

I0104 16:22:19.140847    1327 kubelet.go:776] Local Hostname: i-004a0d15dfc0dc709.ap-southeast-2.compute.internal
I0104 16:22:19.141646    1327 kubelet.go:780] Local IPv4: 172.20.50.163
I0104 16:22:19.142047    1327 kubelet.go:784] Public IPv4: 52.65.193.146

/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 4, 2024
@hakman hakman mentioned this pull request Jan 4, 2024
Merged
@k8s-ci-robot k8s-ci-robot merged commit 57b457c into kubernetes:master Jan 4, 2024
22 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.29 milestone Jan 4, 2024
@hakman hakman deleted the aws-vpc-cni-max-pods branch January 4, 2024 17:07
This was referenced Jan 19, 2024
Closed
Merged
k8s-ci-robot added a commit that referenced this pull request Jan 19, 2024
@k8s-ci-robot
Merge pull request #16260 from hakman/automated-cherry-pick-of-#16216-#…
3725172 
…16253-upstream-release-1.28

Automated cherry pick of #16216: aws: Retrieve instance info only when max pods is not set
#16253: aws: fix maxPods when cilium ipam is used
@BrewTestBot BrewTestBot mentioned this pull request May 16, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@justinsb justinsb justinsb left review comments

@rifelpet rifelpet rifelpet approved these changes

@zetaab zetaab Awaiting requested review from zetaab

Assignees

@justinsb justinsb

@rifelpet rifelpet

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/nodeup cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
v1.29
Development

Successfully merging this pull request may close these issues.
4 participants
@hakman @justinsb @k8s-ci-robot @rifelpet