ServerDisconnectedError is trigger happy

[digitaldavenyc]

I'm getting this error a lot when using the aiohttp library when making http requests. It's firing on over 50% of the domains I visit. The documentation is extremely light on what this error actually is why it is firing at all. Can someone clarify what it is?

[digitaldavenyc]

I figured out what the issue was, if the session connection is closed before you try to read the results of the requests ServerDisconnectedError fires. This must've been a functional change is an update, I did not receive this error using and older version of aiohttp

Is it necessary to close the connection on all requests?

[asvetlov]

ServerDisconnectedError is caused when server side closes connection in non-expected manner.

Response should be release explicitly (yield from resp.release()) to avoid resource warning -- it's best practice. After Python 3.5 release I'll add context manager for that.

[digitaldavenyc]

Thanks! Was wondering what the best practice was for opening and closing connections.

Calling response and connector close() before calling read() was also causing a ServerDisconnectedError. I modified the code so that didn't happen and it resolved the issue. Does this make any sense?

[asvetlov]

I think it's a bug.
If you did not see it in older aiohttp versions it's regression.
We improved disconnection handling and probably made this error.
I suggest the following behavior: after direct connector/client session closing all requests and responses produced by connector should be closed silently.

@fafhrd91 that do you think about?

[digitaldavenyc]

Running yield from resp.release() actually introduced another error. I've since removed it and reverted to not closing any connections and my application is performing well.

I probably should have noted that this was being used for aiohttp.request GET requests and not serving content.

[asvetlov]

Running yield from resp.release() actually introduced another error.

Please elaborate.

[digitaldavenyc]

I upgraded to the latest aiohttp and this is no longer an issue.

[fafhrd91]

Good

[wvxvw]

@asvetlov how do you imagine a combination of response.raise_for_status() and response.release()?

[asvetlov]

async with client.get(url) as resp:
    resp.raise_for_status()

[wvxvw]

@asvetlov are you even serious? What if response must outlive the function in which it is created?

[asvetlov]

I'm deadly serious.
What prevent you from something like:

async with client.get(url) as resp:
    process_response(resp)  # do all your dirty work here, even resp.raise_for_status() call

[wvxvw]

If I could chose everything about my code, I wouldn't chose Python, and surely not asyncio, and none of its minions. The reason I'm here is that I cannot chose. One thing I cannot chose is what to do with response generated in this way. It must outlive the scope of the context manager it is in.

Sorry for the tone: most things I'm upset about aren't your fault. The retard who conceived the idea of context managers is probably the most guilty one, but Python core developers must share in this responsibility, obviously. You are only guilty of thinking that it is a good idea to use one.

Now, technically, you probably realize there is a problem. You probably also suspect that there isn't a good solution, (because, what you offer is certainly not a solution) that that was a mistake, but the change to API is too extreme. You'd rather live with bad API than try to deal with change. I can understand this too.

[wvxvw]

In addition to above, if response outlives the session, then aiohttp hangs forever, not even errors. I cannot really think of a worse interface.

[asvetlov]

Please keep your emotions for other places.
Sure, Python and especially asyncio maybe not fit well to people with alternative brain organization.
If you hate aiohttp, asyncio and Python -- please don't use it.
Save your and our time.
Otherwise please keep constructive tone in conversation.

The thread is frozen, it falled into counterproductive series of blames.