Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor create_kernels RPC call to asynchronous event-driven pattern #2953

Open
fregataa opened this issue Oct 23, 2024 · 0 comments
Open

Comments

@fregataa
Copy link
Member

fregataa commented Oct 23, 2024

Current Behavior

The "manager" component makes a RPC call create_kernels to "agent" components. The manager blocks and waits for the RPC response from agents, which:
Creates tight coupling between managers and agents
Increases latency as managers must wait for kernel creation completion

Proposed Changes

Refactor the create_kernels RPC function to use a fire-and-forget pattern with event-based result handling:

  1. Manager component:
  • Implement event subscription to receive kernel creation failure (managers are already handling creation success events).
  1. Agent component:
  • Modify create_kernels RPC handler to spawn background coroutine tasks
  • Return response in create_kernels RPC function immediately
  • Add error handling and dispatch kernel creation failure events
@fregataa fregataa changed the title Let a scheduler create sessions by "fire and forget" Refactor create_kernels RPC call to asynchronous event-driven pattern Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant