Skip to content

Commit

Permalink
[BugFix] Prevent the task of _force_log from being garbage collected
Browse files Browse the repository at this point in the history
The task returned by `asyncio.create_task` should not be discarded. See python/cpython#88831
  • Loading branch information
Atry committed May 2, 2024
1 parent 0f8a914 commit 9043f78
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion vllm/entrypoints/openai/api_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
openai_serving_completion: OpenAIServingCompletion
logger = init_logger(__name__)

_running_tasks = set()

@asynccontextmanager
async def lifespan(app: fastapi.FastAPI):
Expand All @@ -43,7 +44,9 @@ async def _force_log():
await engine.do_log_stats()

if not engine_args.disable_log_stats:
asyncio.create_task(_force_log())
task = asyncio.create_task(_force_log())
_running_tasks.add(task)
task.add_done_callback(_running_tasks.remove)

yield

Expand Down

0 comments on commit 9043f78

Please sign in to comment.