TemporaryRedirect on recently created bucket

[kr]

Version of AWS SDK for Go?

57572ec625c9aa8bf5c45453efa923bacabc0afe

Version of Go (go version)?

go version go1.8 linux/amd64

What issue did you see?

On a recently created bucket:

TemporaryRedirect: Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.
	status code: 307, request id: 04F41768B41AED8F, host id: VU4EJ7ZZKxU0bRi0vobrdfHes6YCKDcgsPOGoWaW8BYhbrxeauwWJTxxUZSDdoTV24o4inlLXas=

This bucket operates normally on the AWS console and via the AWS CLI. Of the things I've tried, only the AWS Go SDK is returning this error.

Steps to reproduce

Create a bucket and then call PutObjectWithContext.

[jasdel]

Thanks for contacting us @kr Out of curiosity is your code attempting to put to the bucket as soon as the CreateBucket API call returns? I believe there is a small timing issue that can arise because addressing the bucket by its domain name isn't available yet. e.g. mybucket.s3-us-west-2.amazonaws.com.

The SDK provides the WaitUntilBucketExists utility on the S3 service client that you can use to ensure that the bucket is fully created and is available to use. Both the CLI and Console may be performing additional logic hiding the need to wait until the bucket exists, or the timing is just enough to not experience the issue. In my experience the time until a bucket is ready is very very small, but its non-zero. Using the WaitUntilBucketExists method will help you ensure the bucket is ready to be used before its used.

sess := session.Must(session.NewSession())

svc := s3.New(sess, &aws.Config{Region: aws.String(myRegion)})

// Create the bucket
resp, err := svc.CreateBucket(&CreateBucketInput{
    Bucket: aws.String(myBucket),
})

// Wait until bucket is setup and ready to use before using it.
err = svc.WaitUntilBucketExists(&s3.HeadBucket{
    Bucket: aws.String(myBucket),
})

// Now the bucket is ready to use.

[kr]

Out of curiosity is your code attempting to put to the bucket as soon as the CreateBucket API call returns?

No, I created the bucket manually on the AWS console, then a few minutes later attempted to put an object into it. As I mentioned, putting an object on the console or with the CLI worked fine, but subsequent attempts to put an object using the AWS Go SDK still failed over the course of many minutes.

I tried the same PUT call on the same bucket with the Go SDK again the next day (today) and it worked.

[jasdel]

Thanks for the update and clarification @kr. Are you only creating the bucket via the AWS console? Then attempting to upload to it from the SDK, or CLI?

I've not experienced this issue before. I'll try to reproduce this. What region are you using when creating the bucket? Does this happen every time a new bucket is created, or only some fraction of the time?

[jasdel]

So after a bit of investigation the redirect issue looks to be a known condition of creating buckets outside of the s3.amazonaws.com endpoint. http://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.html

The SDK's http client will not follow this 307 redirect because the method is a PUT. and the SDK does set the http.Request.GetBody value that was added in Go 1.8 enabling support for following PUT redirects.

[jasdel]

Updating this to a bug since the SDK should definitely support these redirects, and there isn't really a good way to work around this issue.

[jasdel]

Thanks for reporting this issue @kr I've updated the SDK to enable support for retrying PUT requests that are redirected. This should resolve the errors that you are seeing. Let us know if you run into any additional issues, or have feedback.

[kr]

Awesome, thank you so much!

(It looks like you don't need the RequestID, but in case you ever do for whatever reason, there's one in my original message above. And yep, I was/am using Go 1.8.)

[jasdel]

Thanks for verifying this corrects the redirect you were running into.