Decompress gzipped responses

[kalaspuff]

With aiobotocore 0.5.0 it now also accepts content encoding 'gzip' but doesn't use aiohttp auto_decompress which makes larger responses from AWS end up compressed when passed to botocore parsers.

In my case I got this issue after upgrading to 0.5.0 and when doing a list_topics() call on an SNS aiobotocore client and had lots of topics in the response data.

File ".../lib/python3.5/site-packages/aiobotocore/client.py", line 80, in _make_api_call
  operation_model, request_dict)
File ".../lib/python3.5/site-packages/aiobotocore/endpoint.py", line 264, in _send_request
  request, operation_model, attempts)
File ".../lib/python3.5/site-packages/aiobotocore/endpoint.py", line 356, in _get_response
  response_dict, operation_model.output_shape)
File ".../lib/python3.5/site-packages/botocore/parsers.py", line 212, in parse
  parsed = self._do_parse(response, shape)
File ".../lib/python3.5/site-packages/botocore/parsers.py", line 454, in _do_parse
  root = self._parse_xml_string_to_dom(xml_contents)
File ".../lib/python3.5/site-packages/botocore/parsers.py", line 390, in _parse_xml_string_to_dom
  "invalid XML received:\n%s" % (e, xml_string))
botocore.parsers.ResponseParserError: Unable to parse response (not well-formed (invalid token): line 1, column 0), invalid XML received:
b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00\xcd\x9aMs\x9b0\x10\x86\xef\xf9\x15\x99 ... \xbf\x01[G\xbd\xdf\x11)\x00\x00'

[codecov]

Codecov Report

Merging #430 into master will decrease coverage by 0.12%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master     #430      +/-   ##
==========================================
- Coverage   87.24%   87.11%   -0.13%     
==========================================
  Files           8        8              
  Lines         486      489       +3     
==========================================
+ Hits          424      426       +2     
- Misses         62       63       +1

Impacted Files	Coverage Δ
aiobotocore/endpoint.py	90.24% <66.66%> (-0.44%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 29b2bd1...43b3ccd. Read the comment docs.

[jettify]

Good catch, previous versions disabled gzip explicitly by adding headers['Accept-Encoding'] = 'identity'. What about streaming output? should we decompress there too?

     elif operation_model.has_streaming_output:
         response_dict['body'] = http_response.raw

[jettify]

Original issue was crc32 checks on dynamo see comment here https://github.com/aio-libs/aiobotocore/pull/425/files

[thehesiod]

we need to ensure the following:

1. making sure that S3 get_object of key that has 'Content-Encoding' metadata set to gzip returns wrong data type #295 works and add unit test
2. I believe there's a bug in Update aiohttp #425, it should not have removed the identity part yet (see the comment I just added), botocore does not support compressed responses and we need to follow what they do.

[kalaspuff]

From my own tests it doesn't seem to be conflicting with #295 at least, as it seems to be a streaming output that sets response_dict['body'] to a FlowControlStreamReader instead of bytes data.

[thehesiod]

@kalaspuff could to try by putting back this line: https://github.com/aio-libs/aiobotocore/pull/425/files#diff-e941ba728e64131911465db18f803157L278 looks like jettify is working on a revert

[jettify]

@kalaspuff please try https://pypi.python.org/pypi/aiobotocore/0.5.1 feel free to open new issue

[kalaspuff]

@jettify Works like a charm, thanks!