Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws : Adds configurations to the default retryer #375

Merged
merged 25 commits into from
Sep 11, 2019

Conversation

skotambkar
Copy link
Contributor

@skotambkar skotambkar commented Aug 29, 2019

Provides more retryer customization options by adding a constructor for default Retryer which accepts functional options. Adds NoOpRetryer to support no retry behavior. Exposes members of default retryer.

Updates the underlying logic used by the default retryer to calculate jittered delay for retry. Also updates the custom retry logic for service/ec2. Handles int overflow for retry delay.

Includes changes for PR #373 . The changes will be squashed once the PR #373 is merged in.

Fixes #370

@skotambkar skotambkar self-assigned this Aug 29, 2019
@skotambkar skotambkar requested a review from jasdel August 29, 2019 18:19
aws/client.go Outdated Show resolved Hide resolved
aws/client.go Outdated Show resolved Hide resolved
aws/config.go Outdated Show resolved Hide resolved
aws/default_retryer.go Outdated Show resolved Hide resolved
aws/default_retryer.go Outdated Show resolved Hide resolved
aws/default_retryer.go Outdated Show resolved Hide resolved
private/model/api/shape_marshal.go Show resolved Hide resolved
service/ec2/customizations.go Outdated Show resolved Hide resolved
service/ec2/customizations.go Outdated Show resolved Hide resolved
service/ec2/customizations.go Outdated Show resolved Hide resolved
@skotambkar skotambkar requested a review from jasdel September 9, 2019 20:31
Copy link
Contributor

@jasdel jasdel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Couple suggestions.

We probably want to target the PR's name and description to focus on updating the SDK's DefaultRetryer instead of EC2's customization.

aws/default_retryer.go Show resolved Hide resolved
aws/default_retryer.go Outdated Show resolved Hide resolved
aws/default_retryer.go Outdated Show resolved Hide resolved
aws/default_retryer.go Outdated Show resolved Hide resolved
aws/default_retryer.go Show resolved Hide resolved
aws/request_test.go Outdated Show resolved Hide resolved
internal/awstesting/client.go Outdated Show resolved Hide resolved
internal/awstesting/unit/unit.go Outdated Show resolved Hide resolved
service/ec2/customizations.go Outdated Show resolved Hide resolved
service/ec2/customizations.go Outdated Show resolved Hide resolved
@jasdel jasdel added the pr/needs-review This PR needs a review from a Member. label Sep 10, 2019
@skotambkar skotambkar changed the title service/ec2: Adds custom retryer logic for service/ec2 aws : Updates retry handling logic used by the sdk Sep 11, 2019
@skotambkar skotambkar changed the title aws : Updates retry handling logic used by the sdk aws : Updates default/custom retry handling logic used by the sdk Sep 11, 2019
@skotambkar skotambkar changed the title aws : Updates default/custom retry handling logic used by the sdk aws : Updates retry handling logic used by the sdk Sep 11, 2019
@skotambkar skotambkar requested a review from jasdel September 11, 2019 07:40
Copy link
Contributor

@jasdel jasdel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, minor suggestions with updates.

aws/default_retryer.go Show resolved Hide resolved
service/dynamodb/customizations.go Outdated Show resolved Hide resolved
CHANGELOG_PENDING.md Outdated Show resolved Hide resolved
@skotambkar skotambkar changed the title aws : Updates retry handling logic used by the sdk aws : Adds configurations to the default retryer Sep 11, 2019
@jasdel jasdel merged commit e57892e into aws:master Sep 11, 2019
skotambkar added a commit that referenced this pull request Sep 12, 2019
Updates expected test result that validates the default retryer logic for jittered delay, added in #375
skotambkar added a commit to skotambkar/aws-sdk-go-v2 that referenced this pull request Sep 17, 2019
===

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Enhancements
---
* `aws/endpoints`: Expose DNSSuffix for partitions ([aws#369](aws#369))
  * Exposes the underlying partition metadata's DNSSuffix value via the `DNSSuffix` method on the endpoint's `Partition` type. This allows access to the partition's DNS suffix, e.g. "amazon.com".
  * Fixes [aws#347](aws#347)
* `private/protocol`: Add support for parsing fractional timestamp ([aws#367](aws#367))
  * Fixes the SDK's ability to parse fractional unix timestamp values and adds tests.
  * Fixes [aws#365](aws#365)
* `aws/ec2metadata`: Add marketplaceProductCodes to EC2 Instance Identity Document ([aws#374](aws#374))
  * Adds `MarketplaceProductCodes` to the EC2 Instance Metadata's Identity Document. The ec2metadata client will now retrieve these values if they are available.
  * Related to: [aws/aws-sdk-go#2781](aws/aws-sdk-go#2781)
* `aws`: Adds configurations to the default retryer ([aws#375](aws#375))
  * Provides more customization options for retryer by adding a constructor for default Retryer which accepts functional options. Adds NoOpRetryer to support no retry behavior. Exposes members of default retryer.
  * Updates the underlying logic used by the default retryer to calculate jittered delay for retry. Handles int overflow for retry delay.
  * Fixes [aws#370](aws#370)
* `aws` : Refactors request retry behavior path logic ([aws#384](aws#384))
  * Retry utilities now follow a consistent code path. aws.IsErrorRetryable is the primary entry point to determine if a request is retryable.
  * Corrects sdk's behavior by not retrying errors with status code 501. Adds support for retrying the Kinesis API error, LimitExceededException.
  * Fixes [aws#372](aws#372), [aws#145](aws#145)

SDK Bugs
---
* `aws`: Fixes bug in calculating throttled retry delay ([aws#373](aws#373))
  * The `Retry-After` duration specified in the request is now added to the Retry delay for throttled exception. Adds test for retry delays for throttled exceptions. Fixes bug where the throttled retry's math was off.
  * Fixes [aws#45](aws#45)
* `aws` : Adds missing sdk error checking when seeking readers ([aws#379](aws#379))
  * Adds support for nonseekable io.Reader. Adds support for streamed payloads for unsigned body request.
  * Fixes [aws#371](aws#371)
* `service/s3` : Fixes unexpected EOF error by s3manager ([aws#386](aws#386))
  * Fixes bug which threw unexpected EOF error when s3 upload is performed for a file of maximum allowed size
  * Fixes [aws#316](aws#316)
* `private/model` : Fixes generated API Reference docs links being invalid ([387](aws#387))
  * Fixes [aws#327](aws#327)
skotambkar added a commit that referenced this pull request Sep 17, 2019
Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Enhancements
---
* `aws/endpoints`: Expose DNSSuffix for partitions ([#369](#369))
  * Exposes the underlying partition metadata's DNSSuffix value via the `DNSSuffix` method on the endpoint's `Partition` type. This allows access to the partition's DNS suffix, e.g. "amazon.com".
  * Fixes [#347](#347)
* `private/protocol`: Add support for parsing fractional timestamp ([#367](#367))
  * Fixes the SDK's ability to parse fractional unix timestamp values and adds tests.
  * Fixes [#365](#365)
* `aws/ec2metadata`: Add marketplaceProductCodes to EC2 Instance Identity Document ([#374](#374))
  * Adds `MarketplaceProductCodes` to the EC2 Instance Metadata's Identity Document. The ec2metadata client will now retrieve these values if they are available.
  * Related to: [aws/aws-sdk-go#2781](aws/aws-sdk-go#2781)
* `aws`: Adds configurations to the default retryer ([#375](#375))
  * Provides more customization options for retryer by adding a constructor for default Retryer which accepts functional options. Adds NoOpRetryer to support no retry behavior. Exposes members of default retryer.
  * Updates the underlying logic used by the default retryer to calculate jittered delay for retry. Handles int overflow for retry delay.
  * Fixes [#370](#370)
* `aws` : Refactors request retry behavior path logic ([#384](#384))
  * Retry utilities now follow a consistent code path. aws.IsErrorRetryable is the primary entry point to determine if a request is retryable.
  * Corrects sdk's behavior by not retrying errors with status code 501. Adds support for retrying the Kinesis API error, LimitExceededException.
  * Fixes [#372](#372), [#145](#145)

SDK Bugs
---
* `aws`: Fixes bug in calculating throttled retry delay ([#373](#373))
  * The `Retry-After` duration specified in the request is now added to the Retry delay for throttled exception. Adds test for retry delays for throttled exceptions. Fixes bug where the throttled retry's math was off.
  * Fixes [#45](#45)
* `aws` : Adds missing sdk error checking when seeking readers ([#379](#379))
  * Adds support for nonseekable io.Reader. Adds support for streamed payloads for unsigned body request.
  * Fixes [#371](#371)
* `service/s3` : Fixes unexpected EOF error by s3manager ([#386](#386))
  * Fixes bug which threw unexpected EOF error when s3 upload is performed for a file of maximum allowed size
  * Fixes [#316](#316)
* `private/model` : Fixes generated API Reference docs links being invalid ([387](#387))
  * Fixes [#327](#327)
@skotambkar skotambkar deleted the issue#370 branch December 3, 2019 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/needs-review This PR needs a review from a Member.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

service/ec2: Fix int overflow in minTime on 386 and arm
2 participants