Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data race in v1.0.9 #509

Closed
imkira opened this issue Jan 15, 2016 · 3 comments
Closed

data race in v1.0.9 #509

imkira opened this issue Jan 15, 2016 · 3 comments
Assignees
Labels
bug This issue is a bug.

Comments

@imkira
Copy link

imkira commented Jan 15, 2016

Apparently it is not safe to read from the http.Transport as we get a data race bug when testing (go test -v -race -cpu=1,2,4 -covermode=atomic).

==================
WARNING: DATA RACE
Read by goroutine 81:
  github.com/aws/aws-sdk-go/aws/ec2metadata.NewClient()
      /home/ubuntu/go/src/github.com/aws/aws-sdk-go/aws/ec2metadata/service.go:54 +0xb5f
  github.com/aws/aws-sdk-go/aws/defaults.CredChain()
      /home/ubuntu/go/src/github.com/aws/aws-sdk-go/aws/defaults/defaults.go:94 +0x245
  github.com/aws/aws-sdk-go/aws/defaults.Get()
      /home/ubuntu/go/src/github.com/aws/aws-sdk-go/aws/defaults/defaults.go:34 +0xb14
  github.com/aws/aws-sdk-go/aws/session.New()
      /home/ubuntu/go/src/github.com/aws/aws-sdk-go/aws/session/session.go:54 +0x37
  example.com/sample.TestS3()
      /home/ubuntu/go/src/example.com/sample/s3_test.go:86 +0x82
  testing.tRunner()
      /tmp/workdir/go/src/testing/testing.go:456 +0xdc

Previous write by goroutine 79:
  sync/atomic.AddInt32()
      /tmp/workdir/go/src/runtime/race_amd64.s:255 +0xb
  sync.(*Mutex).Unlock()
      /tmp/workdir/go/src/sync/mutex.go:106 +0x55
  net/http.(*Transport).putIdleConn()
      /tmp/workdir/go/src/net/http/transport.go:412 +0xa8d
  net/http.(*persistConn).readLoop()
      /tmp/workdir/go/src/net/http/transport.go:997 +0xf0f

Goroutine 81 (running) created at:
  testing.RunTests()
      /tmp/workdir/go/src/testing/testing.go:561 +0xaa3
  testing.(*M).Run()
      /tmp/workdir/go/src/testing/testing.go:494 +0xe4
  main.main()
      example.com/sample/_test/_testmain.go:140 +0x384

Goroutine 79 (running) created at:
  net/http.(*Transport).dialConn()
      /tmp/workdir/go/src/net/http/transport.go:685 +0x11bf
  net/http.(*Transport).getConn.func4()
      /tmp/workdir/go/src/net/http/transport.go:549 +0x73
==================

The problem happens in:

transport := *t

The fact that the transport is managed internally by net/http, makes me think it is unsafe and inappropriate any attempt to make a copy of it (especially when there are unexported fields and mutexes all around). Why not leave that to the user?

@jasdel
Copy link
Contributor

jasdel commented Jan 15, 2016

Thanks for reporting this @imkira This looks to be caused by the change implemented for #504. I think you're correct the SDK should not be copying http.DefaultTransport. All this is done in order to provide shorter timeouts for the EC2RoleProvider in the CredentialsChain. Without the shorter timeout, using the SDK's default CredentialChain on a Non-EC2 host would take a over a minute to fail and return no credentials due to retries.

I think the #504 change and previous implementation is approaching the problem incorrectly. Instead of modifying the Transport the SDK should set the net/http#Client.Timeout instead. This will simplify the concept of EC2 Metadata custom timeouts, and only use this functionality if the client hasn't been modified. Adding a flag to turn off this logic all together is also probably needed.

jasdel added a commit that referenced this issue Jan 16, 2016
The EC2Metadata client was taking a copy of http.Client's Transporter
which is not safe. This also complicated the implementation preventing
custom configurations by users. This alternative implementation is
concurrency safe and is much more flexible with custom transports and
dialers.

The EC2Metadata Timeout override can also be disabled by setting the
EC2MetadataDisableTimeoutOverride `aws.Config` flag. This flag needs to
be passed into the Session or Config prior to creating the credentials
change and EC2RoleProvider.

Fix #509
@jasdel jasdel added the bug This issue is a bug. label Jan 16, 2016
@jasdel jasdel self-assigned this Jan 16, 2016
@jasdel
Copy link
Contributor

jasdel commented Jan 16, 2016

I've posted a PR addresses this data race and improves how the timeout override is configured. The user now has the option of disabling the override completely, in addition to using safe method of configuring the timeout if not disabled.

jasdel added a commit that referenced this issue Jan 16, 2016
The EC2Metadata client was taking a copy of http.Client's Transporter
which is not safe. This also complicated the implementation preventing
custom configurations by users. This alternative implementation is
concurrency safe and is much more flexible with custom transports and
dialers.

The EC2Metadata Timeout override can also be disabled by setting the
EC2MetadataDisableTimeoutOverride `aws.Config` flag. This flag needs to
be passed into the Session or Config prior to creating the credentials
change and EC2RoleProvider.

Fix #509
@imkira
Copy link
Author

imkira commented Jan 16, 2016

Thanks a lot for the prompt support @jasdel

jasdel added a commit that referenced this issue Jan 19, 2016
skotambkar pushed a commit to skotambkar/aws-sdk-go that referenced this issue May 20, 2021
skotambkar pushed a commit to skotambkar/aws-sdk-go that referenced this issue May 20, 2021
Breaking Change
---
* `aws/endpoints`: Several functions and types have been removed
  * Removes `DecodeModel` and `DecodeModelOptions` from the package ([aws#509](aws/aws-sdk-go-v2#509))
  * Remove Region Constants, Partition Constants, and types use for exploring the endpoint data model ([aws#512](aws/aws-sdk-go-v2#512))
* `service/s3/s3crypto`: Package and associated encryption/decryption clients have been removed from the SDK ([aws#511](aws/aws-sdk-go-v2#511))
* `aws/external`: Removes several export constants and types ([aws#508](aws/aws-sdk-go-v2#508))
  * No longer exports AWS environment constants used by the external environment configuration loader
  * `DefaultSharedConfigProfile` is now defined an exported constant
* `aws`: `ErrMissingRegion`, `ErrMissingEndpoint`, `ErrStaticCredentialsEmpty` are now concrete error types ([aws#510](aws/aws-sdk-go-v2#510))

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws/signer/v4`: New methods `SignHTTP` and `PresignHTTP` have been added ([aws#519](aws/aws-sdk-go-v2#519))
  * `SignHTTP` replaces `Sign`, and usage of `Sign` should be migrated before it's removal at a later date
  * `PresignHTTP` replaces `Presign`, and usage of `Presign` should be migrated before it's removal at a later date
  * `DisableRequestBodyOverwrite` and `UnsignedPayload` are now deprecated options and have no effect on `SignHTTP` or `PresignHTTP`. These options will be removed at a later date.
* `aws/external`: Add Support for setting a default fallback region and resolving region from EC2 IMDS ([aws#523](aws/aws-sdk-go-v2#523))
  * `WithDefaultRegion` helper has been added which can be passed to `LoadDefaultAWSConfig`
    * This helper can be used to configure a default fallback region in the event a region fails to be resolved from other sources
  * Support has been added to resolve region using EC2 IMDS when available
    * The IMDS region will be used if region as not found configured in either the shared config or the process environment.
  * Fixes [aws#244](aws/aws-sdk-go-v2#244)
  * Fixes [aws#515](aws/aws-sdk-go-v2#515)

SDK Enhancements
---
* `service/dynamodb/expression`: Add IsSet helper for ConditionBuilder and KeyConditionBuilder ([aws#494](aws/aws-sdk-go-v2#494))
  * Adds a IsSet helper for ConditionBuilder and KeyConditionBuilder to make it easier to determine if the condition builders have any conditions added to them.
  * Implements [aws#493](aws/aws-sdk-go-v2#493).
* `internal/ini`: Normalize Section keys to lowercase ([aws#495](aws/aws-sdk-go-v2#495))
  * Update's SDK's ini utility to store all keys as lowercase. This brings the SDK inline with the AWS CLI's behavior.


SDK Bugs
---
* `internal/sdk`: Fix SDK's UUID utility to handle partial read ([aws#536](aws/aws-sdk-go-v2#536))
  * Fixes the SDK's UUID utility to correctly handle partial reads from its crypto rand source. This error was sometimes causing the SDK's InvocationID value to fail to be obtained, due to a partial read from crypto.Rand.
  * Fix [aws#534](aws/aws-sdk-go-v2#534)
* `aws/defaults`: Fix request metadata headers causing signature errors ([aws#536](aws/aws-sdk-go-v2#536))
    * Fixes the SDK's adding the request metadata headers in the wrong location within the request handler stack. This created a situation where a request that was retried would sign the new attempt using the old value of the header. The header value would then be changed before sending the request.
    * Fix [aws#533](aws/aws-sdk-go-v2#533)
    * Fix [aws#521](aws/aws-sdk-go-v2#521)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug.
Projects
None yet
Development

No branches or pull requests

2 participants