Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace hand-rolled upload/download impl. w/ apitools utilities. #407

Merged
merged 5 commits into from
Dec 10, 2014
Merged

Replace hand-rolled upload/download impl. w/ apitools utilities. #407

merged 5 commits into from
Dec 10, 2014

Conversation

tseaver
Copy link
Contributor

@tseaver tseaver commented Dec 4, 2014

Note that in order to use the 'Upload' / 'Download' utilities provided
by apitools without its 'base_api' module (not vendored), we have to unwind
some of the generated request setup code.

Fixes #334.

@tseaver
Replace hand-rolled upload/download impl. w/ apitools utilities.
08e0c9e 
Note that in order to use the 'Upload' / 'Download' utilities provided
by apitools without its 'base_api' module (not vendored), we have to unwind
some of the generated request setup code.

Fixes #334.
silvolu
silvolu reviewed Dec 4, 2014
View reviewed changes
gcloud/storage/key.py
upload = Upload(file_obj, content_type or 'application/unknown',
total_bytes, auto_transfer=False,
chunksize=self.CHUNK_SIZE)
upload.strategy = _RESUMABLE_UPLOAD

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 71780e3 on tseaver:334-use_apitools_transfer_for_upload_download into 03fc101 on GoogleCloudPlatform:master.

@tseaver
Garden in 'python-modernize' updates to apitools.
0416e93 
Catches up our vendored-in source to commit:
5a1bab1df7a474cff57f7f5cc066b19a1c286a21.
@tseaver
Copy link
Contributor Author

tseaver commented Dec 8, 2014

0416e93 pulls in @craigcitro's merge of my 'python-modernize' PR to the vendored-in sources.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 0416e93 on tseaver:334-use_apitools_transfer_for_upload_download into 03fc101 on GoogleCloudPlatform:master.

@tseaver
Copy link
Contributor Author

tseaver commented Dec 9, 2014

@silvolu, @craigcitro, @dhermes: any further review? Do we need to add regression tests which explicitly target either "simple" or "resumable" uploads?

@silvolu
Copy link
Contributor

silvolu commented Dec 9, 2014

LGTM

Do we need to add regression tests which explicitly target either "simple" or "resumable" uploads?

Please do.

@tseaver
Add a regression test for small-file writes.
9a68d90 
In conjunction with the existing large-file test, this one ensures
that we exercise both the 'simple'/'media' upload API, and the
'resumable' one.
@tseaver
Copy link
Contributor Author

tseaver commented Dec 9, 2014

@silvolu I added a "small file" regression test in 9a68d90, exercising the "simple" / "media" upload API. @dhermes already had a "large file" one, which now exercises the "resumable" upload API.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 9a68d90 on tseaver:334-use_apitools_transfer_for_upload_download into 03fc101 on GoogleCloudPlatform:master.

tseaver added a commit that referenced this pull request Dec 10, 2014
@tseaver
Merge pull request #407 from tseaver/334-use_apitools_transfer_for_up…
41286a8 
…load_download

Replace hand-rolled upload/download impl. w/ apitools utilities.
@tseaver tseaver merged commit 41286a8 into googleapis:master Dec 10, 2014
@tseaver tseaver deleted the 334-use_apitools_transfer_for_upload_download branch December 10, 2014 02:32
@tseaver
Copy link
Contributor Author

tseaver commented Dec 10, 2014

Ugh, another set of likely-transient regression test failures post-merge. I have restarted the build on Travis..

@silvolu
Copy link
Contributor

silvolu commented Dec 10, 2014

@tseaver I can reproduce the failures. Looking into fixing the tests.

@dhermes dhermes mentioned this pull request Dec 30, 2014
Closed
@dhermes dhermes added the api: storage Issues related to the Cloud Storage API. label Dec 31, 2015
parthea pushed a commit that referenced this pull request Jun 4, 2023
@renovate-bot
chore(deps): update dependency google-cloud-documentai to v2.1.0 (#407)
3fcd645
parthea pushed a commit that referenced this pull request Jun 4, 2023
@gcf-owl-bot
chore: resolve issue with prerelease presubmit [autoapprove] (#407)
930a54e 
Source-Link: googleapis/synthtool@1b9ad76
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:9db98b055a7f8bd82351238ccaacfd3cda58cdf73012ab58b8da146368330021
parthea added a commit that referenced this pull request Jun 4, 2023
@gcf-owl-bot @parthea
feat: Add files field to finding's list of attributes (#407)
b246140 
* chore: update to gapic-generator-python 1.5.0

feat: add support for `google.cloud.<api>.__version__`
PiperOrigin-RevId: 484665853

Source-Link: googleapis/googleapis@8eb249a

Source-Link: googleapis/googleapis-gen@c8aa327
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzhhYTMyN2I1ZjQ3ODg2NWZjM2ZkOTFlM2MyNzY4ZTU0ZTI2YWQ0NCJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* update version in gapic_version.py

* add .release-please-manifest.json with correct version

* add owlbot.py to exclude generated gapic_version.py

* set manifest to true in .github/release-please.yml

* add release-please-config.json

* chore: Update to gapic-generator-python 1.6.0

feat(python): Add typing to proto.Message based class attributes

feat(python): Snippetgen handling of repeated enum field

PiperOrigin-RevId: 487326846

Source-Link: googleapis/googleapis@da380c7

Source-Link: googleapis/googleapis-gen@61ef576
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjFlZjU3NjJlZTY3MzFhMGNiYmZlYTIyZmQwZWVjZWU1MWFiMWM4ZSJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: new APIs added to reflect updates to the filestore service

- Add ENTERPRISE Tier
- Add snapshot APIs: RevertInstance, ListSnapshots, CreateSnapshot, DeleteSnapshot, UpdateSnapshot
- Add multi-share APIs: ListShares, GetShare, CreateShare, DeleteShare, UpdateShare
- Add ConnectMode to NetworkConfig (for Private Service Access support)
- New status codes (SUSPENDED/SUSPENDING, REVERTING/RESUMING)
- Add SuspensionReason (for KMS related suspension)
- Add new fields to Instance information: max_capacity_gb, capacity_step_size_gb, max_share_count, capacity_gb, multi_share_enabled

PiperOrigin-RevId: 487492758

Source-Link: googleapis/googleapis@5be5981

Source-Link: googleapis/googleapis-gen@ab0e217
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYWIwZTIxN2Y1NjBjYzJjMWFmYzExNDQxYzJlYWI2YjY5NTBlZmQyYiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* update path to snippet metadata json

* chore: Update gapic-generator-python to v1.6.1

PiperOrigin-RevId: 488036204

Source-Link: googleapis/googleapis@08f275f

Source-Link: googleapis/googleapis-gen@555c094
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNTU1YzA5NDVlNjA2NDllMzg3MzlhZTY0YmM0NTcxOWNkZjcyMTc4ZiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: Add files field to finding's list of attributes

PiperOrigin-RevId: 488290111

Source-Link: googleapis/googleapis@7349e1c

Source-Link: googleapis/googleapis-gen@721049d
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNzIxMDQ5ZDUxMmRlZWJhYTk2ZDQzOGFhMjNlOTcwMjk5ZDk3NzEyYiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
parthea pushed a commit that referenced this pull request Jun 4, 2023
@gcf-owl-bot
chore(deps): Update nox in .kokoro/requirements.in [autoapprove] (#407)
8964345 
Source-Link: https://github.com/googleapis/synthtool/commit/92006bb3cdc84677aa93c7f5235424ec2b157146
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:2e247c7bf5154df7f98cce087a20ca7605e236340c7d6d1a14447e5c06791bd6
parthea pushed a commit that referenced this pull request Sep 20, 2023
@renovate-bot
chore(deps): update all dependencies (#407)
8ab06fb
vchudnov-g pushed a commit that referenced this pull request Sep 20, 2023
@release-please
chore: release 2.9.0 (#407)
9d45771 
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this pull request Sep 22, 2023
@gcf-owl-bot
chore: add prerelease nox session (#407)
0494e8c 
Source-Link: googleapis/synthtool@050953d
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:65e656411895bff71cffcae97246966460160028f253c2e45b7a25d805a5b142

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea added a commit that referenced this pull request Sep 22, 2023
@gcf-owl-bot @parthea
chore(python): upgrade gcp-releasetool in .kokoro [autoapprove] (#407)
60b12d3 
* chore(python): upgrade gcp-releasetool in .kokoro [autoapprove]

Source-Link: googleapis/synthtool@5f2a608
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:8555f0e37e6261408f792bfd6635102d2da5ad73f8f09bcb24f25e6afb5fac97

* trigger ci

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
parthea pushed a commit that referenced this pull request Oct 21, 2023
@gcf-owl-bot
build(deps): [autoapprove] bump certifi from 2022.12.7 to 2023.7.22 (#…
7b6663d 
…407)

Source-Link: https://github.com/googleapis/synthtool/commit/395d53adeeacfca00b73abf197f65f3c17c8f1e9
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:6c1cbc75c74b8bdd71dada2fa1677e9d6d78a889e9a70ee75b93d1d0543f96e1
parthea pushed a commit that referenced this pull request Oct 21, 2023
@gcf-owl-bot
chore: add prerelease nox session (#407)
640ad16 
Source-Link: googleapis/synthtool@050953d
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:65e656411895bff71cffcae97246966460160028f253c2e45b7a25d805a5b142

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this pull request Oct 21, 2023
@gcf-owl-bot
build(deps): bump cryptography to 41.0.0 [autoapprove] (#407)
ee0b865 
Source-Link: https://github.com/googleapis/synthtool/commit/d0f51a0c2a9a6bcca86911eabea9e484baadf64b
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:240b5bcc2bafd450912d2da2be15e62bc6de2cf839823ae4bf94d4f392b451dc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
api: storage Issues related to the Cloud Storage API.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Use apitools for uploads/downloads
5 participants
@tseaver @coveralls @silvolu @craigcitro @dhermes