Server.wait_closed() doesn't always wait for its transports to fihish

[1st1]

BPO	33727
Nosy	@asvetlov, @1st1, @tmewett

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-06-01.05:11:49.326>
labels = ['3.7', '3.8', 'expert-asyncio']
title = "Server.wait_closed() doesn't always wait for its transports to fihish"
updated_at = <Date 2020-08-14.13:48:19.890>
user = 'https://github.com/1st1'

bugs.python.org fields:

activity = <Date 2020-08-14.13:48:19.890>
actor = 'tmewett'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2018-06-01.05:11:49.326>
creator = 'yselivanov'
dependencies = []
files = []
hgrepos = []
issue_num = 33727
keywords = []
message_count = 3.0
messages = ['318360', '326724', '375406']
nosy_count = 4.0
nosy_names = ['asvetlov', 'aymeric.augustin', 'yselivanov', 'tmewett']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue33727'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[1st1]

Server.wait_closed() currently does two checks:

1. if _sockets is None -- means that Server.close() was called
2. if self._waiters is None -- means that Server._wakeup() was called

if (1) *or* (2) is true, wait_closed() just returns without waiting on anything.

However, when Server.close() is called there might be still active transports serving requests. Server.wait_closed() should wait until all of them are detached, even if Server._sockets is already reset.

So the below implementation:

    async def wait_closed(self):
        if self._sockets is None or self._waiters is None:
            return
        waiter = self._loop.create_future()
        self._waiters.append(waiter)
        await waiter

should be changed to:

    async def wait_closed(self):
        if self._waiters is None:
            assert self._active_count == 0
            return
        waiter = self._loop.create_future()
        self._waiters.append(waiter)
        await waiter

[aymericaugustin]

I believe this is by design: the documentation says:

The sockets that represent existing incoming client connections are left open.

Server doesn't keep track of active transports serving requests.

(That said, I haven't figured out what _waiters is here so I could be wrong.)

[tmewett]

I ran into this while working on an asyncio application using
asyncio.start_server.

From the documentation, I expected the combination of close and wait_closed
to wait until all connection handlers have finished. Instead, handlers remaining
running with open connections as background tasks. I wanted to be able to
"gracefully" close the server, with all processing done, so I could inspect some
results for a test case.

Could there be a method for this? One suggestion would be:

• Clarify the current behaviour of close and wait_closed
  (https://bugs.python.org/issue34852)
• Add new coro wait_finished which waits until all handler tasks are done

I'm afraid I'm not familiar with low-level asyncio APIs like transports and
protocols, so I don't know how/if this fits in with those.

[kumaraditya303]

Duplicate of #79033