Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[storage.blob] Remove requests as dependency for storage.blob #25017

Merged
merged 21 commits into from
Jul 29, 2022
Merged

[storage.blob] Remove requests as dependency for storage.blob #25017

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
787ea07
fix retry
Jun 29, 2022
228eeee
catch ServiceResponseError
Jun 29, 2022
dc1b481
changelog
Jun 29, 2022
477ff17
undo retry changes
Jul 1, 2022
872be12
changelog formatting
Jul 1, 2022
102c516
Merge branch 'main' into fix/storage-requests-dependency
lmazuel Jul 1, 2022
31075d9
merge main
Jul 21, 2022
1517113
Merge branch 'fix/storage-requests-dependency' of github.com:Azure/az…
Jul 21, 2022
f388cf6
add tests
Jul 25, 2022
9c597a4
naming
Jul 25, 2022
61082af
pr review
Jul 25, 2022
77266e5
typo
Jul 25, 2022
dc3198f
some formatting
Jul 25, 2022
66436de
streaming retry recording
Jul 26, 2022
5bda09a
unmark stream retry test as live-only
Jul 26, 2022
a366a86
fix extra headers
Jul 27, 2022
46239bf
raise HttpResponseError on failed retries
Jul 27, 2022
b9d0274
some rerecordings
Jul 27, 2022
6bb4574
stricter retry tests
Jul 27, 2022
35514f0
unused import
Jul 27, 2022
5c0be43
changelog
Jul 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions sdk/storage/azure-storage-blob/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
### Features Added

### Bugs Fixed
- Removed `requests` import from `azure.storage.blob._download`. #25017
- Fixed retry mechanism in `azure.storage.blob._download`. #25017
lmazuel marked this conversation as resolved.
Show resolved Hide resolved

## 12.13.0b1 (2022-06-15)

Expand Down
55 changes: 31 additions & 24 deletions sdk/storage/azure-storage-blob/azure/storage/blob/_download.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@
from io import BytesIO
from typing import Iterator, Union

import requests
from azure.core.exceptions import HttpResponseError, ServiceResponseError
from azure.core.exceptions import HttpResponseError, IncompleteReadError, ServiceResponseError
from azure.core.tracing.common import with_current_context

from ._shared.request_handlers import validate_and_format_range_headers
Expand Down Expand Up @@ -205,17 +204,17 @@ def _download_chunk(self, chunk_start, chunk_end):
download_stream_current=self.progress_total,
**self.request_options
)
except HttpResponseError as error:
process_storage_error(error)

try:
chunk_data = process_content(response, offset[0], offset[1], self.encryption_options)
retry_active = False
except (requests.exceptions.ChunkedEncodingError, requests.exceptions.ConnectionError) as error:
jalauzon-msft marked this conversation as resolved.
Show resolved Hide resolved
except (IncompleteReadError, ServiceResponseError) as error:
Stevenjin8 marked this conversation as resolved.
Show resolved Hide resolved
retry_total -= 1
if retry_total <= 0:
raise ServiceResponseError(error, error=error)
time.sleep(1)
except HttpResponseError as error:
process_storage_error(error)

chunk_data = process_content(response, offset[0], offset[1], self.encryption_options)


# This makes sure that if_match is set so that we can validate
# that subsequent downloads are to an unmodified blob
Expand Down Expand Up @@ -434,6 +433,14 @@ def _initial_request(self):
self.size = self._file_size - self._start_range
else:
self.size = self._file_size
retry_active = False

except (IncompleteReadError, ServiceResponseError) as error:
retry_total -= 1
if retry_total <= 0:
raise ServiceResponseError(error, error=error)
time.sleep(1)
continue

except HttpResponseError as error:
if self._start_range is None and error.response.status_code == 416:
Expand All @@ -447,6 +454,13 @@ def _initial_request(self):
download_stream_current=0,
**self._request_options
)
retry_active = False
except (IncompleteReadError, ServiceResponseError) as error:
retry_total -= 1
if retry_total <= 0:
raise ServiceResponseError(error, error=error)
time.sleep(1)
continue
except HttpResponseError as error:
process_storage_error(error)

Expand All @@ -456,22 +470,15 @@ def _initial_request(self):
else:
process_storage_error(error)

try:
if self.size == 0:
self._current_content = b""
else:
self._current_content = process_content(
response,
self._initial_offset[0],
self._initial_offset[1],
self._encryption_options
)
retry_active = False
except (requests.exceptions.ChunkedEncodingError, requests.exceptions.ConnectionError) as error:
retry_total -= 1
if retry_total <= 0:
raise ServiceResponseError(error, error=error)
time.sleep(1)
if self.size == 0:
self._current_content = b""
else:
self._current_content = process_content(
response,
self._initial_offset[0],
self._initial_offset[1],
self._encryption_options
)

# get page ranges to optimize downloading sparse page blob
if response.properties.blob_type == 'PageBlob':
Expand Down