Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

native multi-part upload #2

Closed
gaul opened this issue Jul 27, 2014 · 0 comments
Closed

native multi-part upload #2

gaul opened this issue Jul 27, 2014 · 0 comments

Comments

@gaul
Copy link
Owner

gaul commented Jul 27, 2014

AWS-S3 limits single-part uploads to 5 GB and provides multi-part uploads to allow larger blobs. This API consists of initiate, uploadPart, complete, abort, and listParts. How to translate these multiple calls into the single jclouds method BlobStore.putBlob(blob, multipart())? Can S3Proxy provide a dynamic Payload variant which demuxes these calls?
@gaul gaul mentioned this issue Aug 18, 2014
Closed
gaul added a commit that referenced this issue Jan 3, 2015
@gaul
Emulate multipart upload with single-part uploads
e7c81e4 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
two still fail:

test_multipart_upload_size_too_small
test_list_multipart_upload

References #2.
@gaul gaul mentioned this issue Jan 3, 2015
Merged
gaul added a commit that referenced this issue Jan 3, 2015
@gaul
Emulate multipart upload with single-part uploads
21b6d8f 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
two still fail:

test_multipart_upload_size_too_small
test_list_multipart_upload

References #2.
gaul added a commit that referenced this issue Jan 12, 2015
@gaul
Emulate multipart upload with single-part uploads
b3fbfb6 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
two still fail:

test_multipart_upload_size_too_small
test_list_multipart_upload

References #2.
gaul added a commit that referenced this issue Jan 14, 2015
@gaul
Emulate multipart upload with single-part uploads
c82825a 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
two still fail:

test_multipart_upload_size_too_small
test_list_multipart_upload

References #2.
gaul added a commit that referenced this issue Jan 21, 2015
@gaul
Emulate multipart upload with single-part uploads
44841bb 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
two still fail:

test_multipart_upload_size_too_small
test_list_multipart_upload

References #2.
@gaul gaul mentioned this issue Feb 19, 2015
Closed
gaul added a commit that referenced this issue Feb 21, 2015
@gaul
Emulate multipart upload with single-part uploads
3363687 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
one still fails:

test_list_multipart_upload

References #2.
gaul added a commit that referenced this issue Mar 1, 2015
@gaul
Emulate multipart upload with single-part uploads
5d2ab56 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
test_list_multipart_upload still fails:

References #2.
gaul added a commit that referenced this issue Mar 1, 2015
@gaul
Emulate multipart upload with single-part uploads
e38fb02 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
test_list_multipart_upload still fails:

References #2.
gaul added a commit that referenced this issue Mar 2, 2015
@gaul
Emulate multipart upload with single-part uploads
b0b1f4e 
This approach requires three times as many operations as the optimal
approach.  Implementing this correctly requires exposing the
underlying multipart operations in jclouds.  Most s3-tests pass but
test_list_multipart_upload still fails:

References #2.
gaul added a commit that referenced this issue Mar 3, 2015
@gaul
Explicitly add Content-Length
71d3a67 
Previously InitiateMultipartUpload used the incorrect length.  Found
with s3fs integration tests.  References #2.
@gaul gaul changed the title multi-part uploads native multi-part upload Mar 5, 2015
gaul added a commit that referenced this issue Apr 28, 2015
@gaul
Native multipart upload support
1c99858 
Fixes #2.

Do not commit; requires:

jclouds/jclouds#737
gaul added a commit that referenced this issue Apr 29, 2015
@gaul
Native multipart upload support
f9fc8f7 
Fixes #2.

Do not commit; requires:

jclouds/jclouds#737
gaul added a commit that referenced this issue Apr 30, 2015
@gaul
Native multipart upload support
a548da3 
Fixes #2.

Do not commit; requires:

jclouds/jclouds#737
gaul added a commit that referenced this issue Apr 30, 2015
@gaul
Native multipart upload support
9d566c9 
Fixes #2.

Do not commit; requires:

jclouds/jclouds#737
@gaul gaul closed this as completed in 4a49f32 May 1, 2015
@gaul gaul mentioned this issue May 1, 2015
Closed
reluxa added a commit to reluxa/s3proxy that referenced this issue Oct 10, 2021
@reluxa
buildx gaul#2
8f2f71c
reluxa added a commit to reluxa/s3proxy that referenced this issue Oct 11, 2021
@reluxa
buildx gaul#4 cache location added gaul#2
ce13760
@rajivml rajivml mentioned this issue May 21, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gaul