'NoneType' object has no attribute '__aexit__' aiobotcore (from release 1.4.2 and onwards)

[chrisBLIT]

Async AWS SDK for Python version: aioboto3: 9.3.1, aiobotocore 1.4.2 (and releases after)
Python version: 3.9

Description

Trying to use the latest version of aioboto3 and botocore, in the latest version of aioboto3 resource is no longer used instead we switched over to using below as described in its documentation

session = aioboto3.Session()
    async with session.resource("s3") as s3

we have some async tests, using something similar to mock server and conftest described in the test examples in aioboto3. we are seeing an issue in those async tests, which I think is related to a change added in previous release of botocore to do with "Fix missing close() method on http session". the error we are seeing is below (in aiobotocore/httpsession.py:111:):

AttributeError: 'NoneType' object has no attribute '__aexit__'

the tests work fine, the aioboto3.Session() it seems cleans up and closes the session, however in the mock server it also tries to clean up the session and we get the error above. we are handling the error now for the meantime. wondering If its a good idea for that async def close(self): that a check is added to see if the session is already None.

[chrisBLIT]

just to add the mock server uses the aioboto3.Session() and in the functionality we are testing it also has aioboto3.Session(). one thing we can do is refactor the existing logic so everything below the line async with session.resource("s3") as s3 is in a function and instead we test directly against that. but it seems because the mock server has the aioboto3.Session() and the functionality we are testing has it, that it tries to close the session twice. so there is ways we can change things about to get around it, but thought maybe also good to have a check if the session was already closed.

[thehesiod]

did you mean to log this against https://github.com/terrycain/aioboto3 ?

[chrisBLIT]

the issue we are seeing is in the httpsession in aiobotocore, specifically to do with the following function:

    async def close(self):
        await self._session.__aexit__(None, None, None)
        self._session = None

could be our async test is slightly wrong, but i followed the aioboto3 samples listed above in aioboto3 link and this only started breaking after switching to a version of aiobotocore core past 1.4.2.

incidentally if i do the following to that close method and check for the session before trying to close it, as such:

    async def close(self):
        if self._session:
            await self._session.__aexit__(None, None, None)
            self._session = None

it works fine. the issue we seem to have is the async tests we have clean up the session after they are finished, and then with the mock conftest server tries to clean up the session also, because the session has already been cleaned up the above async def close throws an error. we are handling this at the moment by handling the exception, but wondering if its possibly a good idea to add the above check to see if the session is there before closing it.

[thehesiod]

ya we should probably do an if check, will create a PR asap, thanks!

[thehesiod]

for future reference:

aiobotocore/aiobotocore/httpsession.py

Line 112 in e98c842

await self._session.__aexit__(None, None, None)

[thehesiod]

btw typically I don't call close explicitly, and instead rely on aenter/aexit, how are you getting to the close call?

[chrisBLIT]

btw typically I don't call close explicitly, and instead rely on aenter/aexit, how are you getting to the close call?

It seems to be made from aioboto3, in the AIOBoto3ServiceResource.

async def __aexit__(self, exc_type, exc_val, exc_tb):
     await self.meta.client.close()
     return False

which then makes the link to aiobotocore and in the client.py calls to the close method.

Anyway thanks for adding the check for the session, cleans up my test config :). thanks again

[thehesiod]

i c, @terrycain any reason for not chaining the __aexit__ ?

[terricain]

errr..... nope, 😄 Have added it to my todo list unless someone beats me to it.