Raising an error inside a task of a task group can produce: RuntimeError: called 'started' twice on the same task status

[dominik-schwabe]

Things to check first

• I have searched the existing issues and didn't find my bug already reported there

• I have checked that my bug is still present in the latest release

AnyIO version

4.3.0

Python version

3.12.2

What happened?

Calling task_status.started() can raise RuntimeError: called 'started' twice on the same task status under certain circumstances (see examples), even though it was not called twice.
The reason for this is that the underlying future gets awaited in the start() method. When the start() method gets canceled, the underlying future gets also canceled which makes calling set_result on the future later fail and raise the erroneous RuntimeError.
This can be fixed by using asyncio.shield when awaiting the future.

anyio/src/anyio/_backends/_asyncio.py

Line 782 in 05a3d02

return await future

How can we reproduce the bug?

from anyio import create_task_group, run
from anyio.abc import TaskStatus


async def taskfunc(*, task_status: TaskStatus):
    task_status.started()
    raise ValueError


async def main():
    async with create_task_group() as tg:
        await tg.start(taskfunc)
        await tg.start(taskfunc)


run(main)

or

from anyio import create_task_group, run
from anyio.abc import TaskStatus


async def taskfunc(*, task_status: TaskStatus):
    task_status.started()


async def main():
    async with create_task_group() as tg:
        coro = tg.start(taskfunc)
        tg.cancel_scope.cancel()
        await coro


run(main)

[agronholm]

asyncio.shield() is an anti-pattern I would very much like to stay away from. I'll get to the bottom of this over Easter.

[dominik-schwabe]

Understandable. Alternatively one could check if the future got cancled and only raise an error if this is not the case.

anyio/src/anyio/_backends/_asyncio.py

Line 621 in 05a3d02

self._future.set_result(value)

[agronholm]

Understandable. Alternatively one could check if the future got cancled and only raise an error if this is not the case.

anyio/src/anyio/_backends/_asyncio.py

Line 621 in 05a3d02

self._future.set_result(value)

Sounds much better already.

[dominik-schwabe]

I modified my pull request with the alternative solution, but I am not sure if the code on line 628 and 629 should also be executed when the future is cancelled.
https://github.com/dominik-schwabe/anyio/blob/d9179b4750b49ab593c6692e30c8c62f7e295dc4/src/anyio/_backends/_asyncio.py#L619-L629

[agronholm]

No worries, I'll have a look within a couple days.

[jonathanslenders]

Thanks! I just see this thread; having exactly the same issue. :-) Let me know if there's anything I can help with or test.

[davidbrochart]

I just got hit by this bug, and @dominik-schwabe's fix solved it.