You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm using the s3fs library that makes use of aiobotocore underneath. I'm running the library like this:
fs = s3fs.S3FileSystem(s3_additional_kwargs={'ChecksumAlgorithm': 'SHA256'})
with fs.open(f'some_bucket/some_object', 'w', block_size=10 * 1024 * 1024) as f:
f.write("hello")
Not supplying the kwargs works fine, yet I'm expecting it to work with the latest ChecksumAlgorithm feature enabled.
I'm receiving several errors from aiobotocore:
Traceback (most recent call last):
File "Python39\lib\site-packages\aiohttp\client_reqrep.py", line 510, in update_body_from_data
body = payload.PAYLOAD_REGISTRY.get(body, disposition=None)
File "Python39\lib\site-packages\aiohttp\payload.py", line 118, in get
raise LookupError()
aiohttp.payload.LookupError
Traceback (most recent call last):
File "Python39\lib\site-packages\aiobotocore\httpsession.py", line 178, in send
response = await self._session.request(
File "Python39\lib\site-packages\aiohttp\client.py", line 507, in _request
req = self._request_class(
File "Python39\lib\site-packages\aiohttp\client_reqrep.py", line 313, in __init__
self.update_body_from_data(data)
File "Python39\lib\site-packages\aiohttp\client_reqrep.py", line 512, in update_body_from_data
body = FormData(body)()
File "Python39\lib\site-packages\aiohttp\formdata.py", line 37, in __init__
self.add_fields(*fields)
File "Python39\lib\site-packages\aiohttp\formdata.py", line 107, in add_fields
raise TypeError(
TypeError: Only io.IOBase, multidict and (name, file) pairs allowed, use .add_field() for passing more complex parameters, got <botocore.httpchecksum.AwsChunkedWrapper object at 0x000001187CDEC0A0>
Traceback (most recent call last):
< my code snipped >
f.write("hello")
File "Python39\lib\site-packages\fsspec\spec.py", line 1664, in close
self.flush(force=True)
File "Python39\lib\site-packages\fsspec\spec.py", line 1535, in flush
if self._upload_chunk(final=force) is not False:
File "Python39\lib\site-packages\s3fs\core.py", line 2095, in _upload_chunk
self.commit()
File "Python39\lib\site-packages\s3fs\core.py", line 2110, in commit
write_result = self._call_s3(
File "Python39\lib\site-packages\s3fs\core.py", line 1962, in _call_s3
return self.fs.call_s3(method, self.s3_additional_kwargs, *kwarglist, **kwargs)
File "Python39\lib\site-packages\fsspec\asyn.py", line 86, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "Python39\lib\site-packages\fsspec\asyn.py", line 66, in sync
raise return_result
File "Python39\lib\site-packages\fsspec\asyn.py", line 26, in _runner
result[0] = await coro
File "Python39\lib\site-packages\s3fs\core.py", line 332, in _call_s3
return await _error_wrapper(
File "Python39\lib\site-packages\s3fs\core.py", line 137, in _error_wrapper
raise err
File "Python39\lib\site-packages\s3fs\core.py", line 110, in _error_wrapper
return await func(*args, **kwargs)
File "Python39\lib\site-packages\aiobotocore\client.py", line 251, in _make_api_call
http, parsed_response = await self._make_request(
File "Python39\lib\site-packages\aiobotocore\client.py", line 271, in _make_request
return await self._endpoint.make_request(operation_model, request_dict)
File "Python39\lib\site-packages\aiobotocore\endpoint.py", line 80, in _send_request
while await self._needs_retry(attempts, operation_model,
File "Python39\lib\site-packages\aiobotocore\endpoint.py", line 213, in _needs_retry
responses = await self._event_emitter.emit(
File "Python39\lib\site-packages\aiobotocore\hooks.py", line 50, in _emit
response = await resolve_awaitable(handler(**kwargs))
File "Python39\lib\site-packages\aiobotocore\_helpers.py", line 11, in resolve_awaitable
return await obj
File "Python39\lib\site-packages\aiobotocore\retryhandler.py", line 89, in _call
if await resolve_awaitable(self._checker(**checker_kwargs)):
File "Python39\lib\site-packages\aiobotocore\_helpers.py", line 11, in resolve_awaitable
return await obj
File "Python39\lib\site-packages\aiobotocore\retryhandler.py", line 107, in _call
should_retry = await self._should_retry(attempt_number, response,
File "Python39\lib\site-packages\aiobotocore\retryhandler.py", line 129, in _should_retry
return await resolve_awaitable(
File "Python39\lib\site-packages\aiobotocore\_helpers.py", line 11, in resolve_awaitable
return await obj
File "Python39\lib\site-packages\aiobotocore\retryhandler.py", line 146, in _call
checker(attempt_number, response,
File "Python39\lib\site-packages\botocore\retryhandler.py", line 247, in __call__
return self._check_caught_exception(
File "Python39\lib\site-packages\botocore\retryhandler.py", line 416, in _check_caught_exception
raise caught_exception
File "Python39\lib\site-packages\aiobotocore\endpoint.py", line 149, in _do_get_response
http_response = await self._send(request)
File "Python39\lib\site-packages\aiobotocore\endpoint.py", line 229, in _send
return await self.http_session.send(request)
File "Python39\lib\site-packages\aiobotocore\httpsession.py", line 220, in send
raise HTTPClientError(error=e)
botocore.exceptions.HTTPClientError: An HTTP Client raised an unhandled exception: Only io.IOBase, multidict and (name, file) pairs allowed, use .add_field() for passing more complex parameters, got <botocore.httpchecksum.AwsChunkedWrapper object at 0x00000197488C50A0>
Process finished with exit code 1
Checklist
I have reproduced in environment where pip check passes without errors
I have provided pip freeze results
I have provided sample code or detailed way to reproduce
I have tried the same code in botocore to ensure this is an aiobotocore specific issue
I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue
I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection
Describe the bug
I'm using the s3fs library that makes use of aiobotocore underneath. I'm running the library like this:
Not supplying the kwargs works fine, yet I'm expecting it to work with the latest ChecksumAlgorithm feature enabled.
I'm receiving several errors from aiobotocore:
Checklist
pip check
passes without errorspip freeze
resultspip freeze results
Environment:
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: