Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent behavior of asyncio.Server.wait_closed in Python 3.12 versus earlier releases #120866

Open
tovrstra opened this issue Jun 22, 2024 · 1 comment
Labels
topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@tovrstra
Copy link

tovrstra commented Jun 22, 2024

Bug report

Bug description:

The issue fixed in #111424 is only available in Python 3.12. As a result, the example below prematurely closes a connection when using an older Python version. The old behavior may lead to bugs that are hard to catch, and the difference in behavior implies that applications must use workarounds to support multiple Python versions.

If backporting the fix is not an option, it would be helpful to add a note to the documentation of asyncio.Server.wait_closed that it changed in Python 3.12.

Related issues: #104344 (same cause, going from 3.11 to 3.12), #79033

minimal example

server.py

import asyncio
from functools import partial

async def handler(stop_event, reader, writer):
    print("Client connected")
    msg = (await reader.readline()).decode()
    print(f"Received: {msg.strip()}")
    await asyncio.sleep(1)
    if "STOP" in msg:
        print("Client requests stop. Setting stop event")
        stop_event.set()
        await asyncio.sleep(1)
    print("----> Respond with same message <----")
    writer.write(msg.encode())
    await writer.drain()
    writer.close()
    await writer.wait_closed()
    print("----> Closing connection <----")


async def main():
    print("Starting server")
    stop_event = asyncio.Event()
    socket_path = "socket"
    server = await asyncio.start_unix_server(partial(handler, stop_event), socket_path)
    print(f"Server listening at: {socket_path}")
    async with server:
        await stop_event.wait()
        print("Server stopping")
    print("Server stopped")


if __name__ == "__main__":
    asyncio.run(main())

client.py

import asyncio


async def talk(msg):
    socket_path = "socket"
    print(f"Connecting to: {socket_path}")
    reader, writer = await asyncio.open_unix_connection(socket_path)
    print("Connected")
    print(f"Sending message: {msg}")
    writer.write(f"{msg}\n".encode())
    await writer.drain()
    res = (await reader.readline()).decode()
    print(f"Received response: {res[:-1]}")
    print("Closing connection")
    writer.close()
    await writer.wait_closed()


async def main():
    print("Starting client")
    await talk("Hello")
    await talk("STOP")
    print("Stopping client")


if __name__ == "__main__":
    asyncio.run(main())

This is a minimal working example distilled from a more complex use case where the stop_event is set in a function that receives the request and creates the response.

One possible workaround is to postpone setting the stop_event to the end of the handler function. That would be easy in this example, but more involved in the case where I ran into this issue.

Python 3.12 and 3.13 output

server.py

Starting server
Server listening at: socket
Client connected
Received: Hello
----> Respond with same message <----
----> Closing connection <----
Client connected
Received: STOP
Client requests stop. Setting stop event
Server stopping
----> Respond with same message <----
----> Closing connection <----
Server stopped

client.py

Starting client
Connecting to: socket
Connected
Sending message: Hello
Received response: Hello
Closing connection
Connecting to: socket
Connected
Sending message: STOP
Received response: STOP
Closing connection
Stopping client

Python 3.9, 3.10 and 3.11 output

Note that the handler does not complete when it receives STOP.

server.py

Starting server
Server listening at: socket
Client connected
Received: Hello
----> Respond with same message <----
----> Closing connection <----
Client connected
Received: STOP
Client requests stop. Setting stop event
Server stopping
Server stopped

client.py

Starting client
Connecting to: socket
Connected
Sending message: Hello
Received response: Hello
Closing connection
Connecting to: socket
Connected
Sending message: STOP
Received response: 
Closing connection
Stopping client

CPython versions tested on:

3.9, 3.10, 3.11, 3.12, 3.13

Operating systems tested on:

Linux

@tovrstra
Copy link
Author

Here is a workaround that mimics the implementation in Python 3.12.1 when using a Python version without the fix in #111424. One can add the following code right before print("Server stopped") in server.py:

    if sys.version_info < (3, 12, 1) and server._waiters is not None:
        # Workaround for server.wait_closed() issue fixed in Python 3.12.1
        waiter = server.get_loop().create_future()
        server._waiters.append(waiter)
        await waiter

This may be a useful snippet for implementing a server that should work across different Python versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-asyncio type-bug An unexpected behavior, bug, or error
Projects
Status: Todo
Development

No branches or pull requests

2 participants