-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AWS EC2 API Rate exceeded #364
Comments
It's probably solved by jenkinsci/jenkins#7658 (perfect improvement) added on Jenkins 2.403 |
Thanks for your response, @benipeled! Unfortunately, that's not included into an LTS release at this time but once it is, I'll try it. |
Related Issue: #347 |
Issues addressed as part of #385. Please, re-open if experiencing any additional issues. |
Released in ec2-fleet-2.7.1 |
Issue Details
Describe the bug
Whenever the "Manage Jenkins" -> "Manage Nodes and Clouds" -> "Configure Clouds" page is being opened, the EC2 fleet plugin loads all the fleets that were added. As far as I can tell, this load procedure, involves loading all the AWS Auto Scaling Groups on one's AWS account using the EC2 API for all the fleets that were added ordered by the name of the fleet (not by the order that can be seen on the page itself).
If someone has 100 EC2 fleets, the plugin will make 100 API calls in order to retrieve the ASG lists (which doesn't change from one fleet to another). Therefore, at some point, the API's requests/second rate can be exceeded and the following error occurs:
WARNING com.amazon.jenkins.ec2fleet.EC2FleetCloud$DescriptorImpl doFillFleetItems
Cannot describe fleets in eu-central-1 or by endpoint
com.amazonaws.services.autoscaling.model.AmazonAutoScalingException: Rate exceeded (Service: AmazonAutoScaling; Status Code: 400; Error Code: Throttling; Request ID: f4886158-05d0-4bfb-80e2-db990dfb300b; Proxy: null)
Whenever this error occurs, the plugin is using the first (alphabetically sorted) ASG that it can find in a previous list for the "EC2 Fleet
" field. This fill happens only on the Jenkins page, but if someone is not careful about it and saves the changes, then multiple EC2 fleets can be mismatched (instead of using the right ASG, they will use the first ASG that was found by the plugin).
To Reproduce
Environment Details
Plugin Version?
2.5.1
Jenkins Version?
2.319.2
Spot Fleet or ASG?
ASG
Label based fleet?
Yes - but I don't think it's relevant in this case
Linux or Windows?
Both
EC2Fleet Configuration as Code
N/A
Anything else unique about your setup?
Not really
The text was updated successfully, but these errors were encountered: