Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[storage.blob] Remove requests as dependency for storage.blob #25017

Merged
merged 21 commits into from
Jul 29, 2022
Merged
1 change: 1 addition & 0 deletions sdk/storage/azure-storage-blob/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
### Features Added

### Bugs Fixed
- Updated exception catching of `StorageStreamDownloader`'s retry mechanism.
- Adjusted type hints for `upload_blob` and `StorageStreamDownloader.readall`.

## 12.13.0 (2022-07-07)
Expand Down
11 changes: 5 additions & 6 deletions sdk/storage/azure-storage-blob/azure/storage/blob/_download.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@
from io import BytesIO
from typing import Generic, Iterator, TypeVar

import requests
from azure.core.exceptions import HttpResponseError, ServiceResponseError
from azure.core.exceptions import DecodeError, HttpResponseError, IncompleteReadError
from azure.core.tracing.common import with_current_context

from ._shared.request_handlers import validate_and_format_range_headers
Expand Down Expand Up @@ -213,10 +212,10 @@ def _download_chunk(self, chunk_start, chunk_end):
try:
chunk_data = process_content(response, offset[0], offset[1], self.encryption_options)
retry_active = False
except (requests.exceptions.ChunkedEncodingError, requests.exceptions.ConnectionError) as error:
jalauzon-msft marked this conversation as resolved.
Show resolved Hide resolved
except (IncompleteReadError, HttpResponseError, DecodeError) as error:
retry_total -= 1
if retry_total <= 0:
raise ServiceResponseError(error, error=error)
raise HttpResponseError(error, error=error)
time.sleep(1)

# This makes sure that if_match is set so that we can validate
Expand Down Expand Up @@ -474,10 +473,10 @@ def _initial_request(self):
self._encryption_options
)
retry_active = False
except (requests.exceptions.ChunkedEncodingError, requests.exceptions.ConnectionError) as error:
except (IncompleteReadError, HttpResponseError, DecodeError) as error:
retry_total -= 1
if retry_total <= 0:
raise ServiceResponseError(error, error=error)
raise HttpResponseError(error, error=error)
time.sleep(1)

# get page ranges to optimize downloading sparse page blob
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ interactions:
Content-Length:
- '0'
User-Agent:
- azsdk-python-storage-blob/12.12.1 Python/3.10.4 (Windows-10-10.0.17763-SP0)
- azsdk-python-storage-blob/12.14.0b1 Python/3.7.13 (Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid)
x-ms-date:
- Mon, 06 Jun 2022 23:40:27 GMT
- Wed, 27 Jul 2022 20:08:23 GMT
x-ms-version:
- '2021-06-08'
- '2021-08-06'
method: PUT
uri: https://storagename.blob.core.windows.net/utcontainer403f19fe?restype=container
response:
Expand All @@ -25,15 +25,15 @@ interactions:
content-length:
- '0'
date:
- Mon, 06 Jun 2022 23:40:27 GMT
- Wed, 27 Jul 2022 20:08:25 GMT
etag:
- '"0x8DA4815EFD21552"'
- '"0x8DA700BC42EC346"'
last-modified:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:26 GMT
server:
- Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-version:
- '2021-06-08'
- '2021-08-06'
status:
code: 201
message: Created
Expand All @@ -53,13 +53,13 @@ interactions:
If-None-Match:
- '*'
User-Agent:
- azsdk-python-storage-blob/12.12.1 Python/3.10.4 (Windows-10-10.0.17763-SP0)
- azsdk-python-storage-blob/12.14.0b1 Python/3.7.13 (Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid)
x-ms-blob-type:
- BlockBlob
x-ms-date:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:23 GMT
x-ms-version:
- '2021-06-08'
- '2021-08-06'
method: PUT
uri: https://storagename.blob.core.windows.net/utcontainer403f19fe/encryption_block_blob403f19fe
response:
Expand All @@ -71,19 +71,19 @@ interactions:
content-md5:
- E1bGfXrRY42Ba/uCLdLCXQ==
date:
- Mon, 06 Jun 2022 23:40:27 GMT
- Wed, 27 Jul 2022 20:08:25 GMT
etag:
- '"0x8DA4815EFE0DF93"'
- '"0x8DA700BC4342311"'
last-modified:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:26 GMT
server:
- Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-content-crc64:
- 7Wml2VbcwgU=
x-ms-request-server-encrypted:
- 'true'
x-ms-version:
- '2021-06-08'
- '2021-08-06'
status:
code: 201
message: Created
Expand All @@ -97,11 +97,11 @@ interactions:
Connection:
- keep-alive
User-Agent:
- azsdk-python-storage-blob/12.12.1 Python/3.10.4 (Windows-10-10.0.17763-SP0)
- azsdk-python-storage-blob/12.14.0b1 Python/3.7.13 (Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid)
x-ms-date:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:23 GMT
x-ms-version:
- '2021-06-08'
- '2021-08-06'
method: HEAD
uri: https://storagename.blob.core.windows.net/utcontainer403f19fe/encryption_block_blob403f19fe
response:
Expand All @@ -117,31 +117,29 @@ interactions:
content-type:
- application/octet-stream
date:
- Mon, 06 Jun 2022 23:40:27 GMT
- Wed, 27 Jul 2022 20:08:25 GMT
etag:
- '"0x8DA4815EFE0DF93"'
- '"0x8DA700BC4342311"'
last-modified:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:26 GMT
server:
- Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
vary:
- Origin
x-ms-access-tier:
- Hot
x-ms-access-tier-inferred:
- 'true'
x-ms-blob-type:
- BlockBlob
x-ms-creation-time:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:26 GMT
x-ms-lease-state:
- available
x-ms-lease-status:
- unlocked
x-ms-server-encrypted:
- 'true'
x-ms-version:
- '2021-06-08'
- '2021-08-06'
status:
code: 200
message: OK
Expand All @@ -155,13 +153,13 @@ interactions:
Connection:
- keep-alive
User-Agent:
- azsdk-python-storage-blob/12.12.1 Python/3.10.4 (Windows-10-10.0.17763-SP0)
- azsdk-python-storage-blob/12.14.0b1 Python/3.7.13 (Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid)
x-ms-date:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:23 GMT
x-ms-range:
- bytes=0-1023
x-ms-version:
- '2021-06-08'
- '2021-08-06'
method: GET
uri: https://storagename.blob.core.windows.net/utcontainer403f19fe/encryption_block_blob403f19fe
response:
Expand All @@ -177,29 +175,139 @@ interactions:
content-type:
- application/octet-stream
date:
- Mon, 06 Jun 2022 23:40:27 GMT
- Wed, 27 Jul 2022 20:08:25 GMT
etag:
- '"0x8DA4815EFE0DF93"'
- '"0x8DA700BC4342311"'
last-modified:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:26 GMT
server:
- Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
vary:
- Origin
x-ms-blob-content-md5:
- E1bGfXrRY42Ba/uCLdLCXQ==
x-ms-blob-type:
- BlockBlob
x-ms-creation-time:
- Mon, 06 Jun 2022 23:40:28 GMT
- Wed, 27 Jul 2022 20:08:26 GMT
x-ms-lease-state:
- available
x-ms-lease-status:
- unlocked
x-ms-server-encrypted:
- 'true'
x-ms-version:
- '2021-06-08'
- '2021-08-06'
status:
code: 206
message: Partial Content
- request:
body: null
headers:
Accept:
- application/xml
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- azsdk-python-storage-blob/12.14.0b1 Python/3.7.13 (Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid)
x-ms-date:
- Wed, 27 Jul 2022 20:08:24 GMT
x-ms-range:
- bytes=0-1023
x-ms-version:
- '2021-08-06'
method: GET
uri: https://storagename.blob.core.windows.net/utcontainer403f19fe/encryption_block_blob403f19fe
response:
body:
string: Foo
headers:
accept-ranges:
- bytes
content-length:
- '3'
content-range:
- bytes 0-2/3
content-type:
- application/octet-stream
date:
- Wed, 27 Jul 2022 20:08:26 GMT
etag:
- '"0x8DA700BC4342311"'
last-modified:
- Wed, 27 Jul 2022 20:08:26 GMT
server:
- Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-blob-content-md5:
- E1bGfXrRY42Ba/uCLdLCXQ==
x-ms-blob-type:
- BlockBlob
x-ms-creation-time:
- Wed, 27 Jul 2022 20:08:26 GMT
x-ms-lease-state:
- available
x-ms-lease-status:
- unlocked
x-ms-server-encrypted:
- 'true'
x-ms-version:
- '2021-08-06'
status:
code: 206
message: Partial Content
- request:
body: null
headers:
Accept:
- application/xml
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- azsdk-python-storage-blob/12.14.0b1 Python/3.7.13 (Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid)
x-ms-date:
- Wed, 27 Jul 2022 20:08:25 GMT
x-ms-range:
- bytes=0-1023
x-ms-version:
- '2021-08-06'
method: GET
uri: https://storagename.blob.core.windows.net/utcontainer403f19fe/encryption_block_blob403f19fe
response:
body:
string: Foo
headers:
accept-ranges:
- bytes
content-length:
- '3'
content-range:
- bytes 0-2/3
content-type:
- application/octet-stream
date:
- Wed, 27 Jul 2022 20:08:27 GMT
etag:
- '"0x8DA700BC4342311"'
last-modified:
- Wed, 27 Jul 2022 20:08:26 GMT
server:
- Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-blob-content-md5:
- E1bGfXrRY42Ba/uCLdLCXQ==
x-ms-blob-type:
- BlockBlob
x-ms-creation-time:
- Wed, 27 Jul 2022 20:08:26 GMT
x-ms-lease-state:
- available
x-ms-lease-status:
- unlocked
x-ms-server-encrypted:
- 'true'
x-ms-version:
- '2021-08-06'
status:
code: 206
message: Partial Content
Expand Down
Loading