-
-
Notifications
You must be signed in to change notification settings - Fork 731
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trio support. #169
Comments
For the version I did in #118, rather than implementing a new protocol, I instead built a bridge between the asyncio protocol interface and trio. Specifically, the reason trio (and curio) do not have a protocol class, as I understand, is that the asyncio protocol implementation is not fully async/await (https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#review-and-summing-up-what-is-async-await-native-anyway). That is, both So, no "asyncio protocols" in trio. But in asyncio proper, a server code that is inside asyncio is just reading from the socket and calling the This means that we lose the purported advantage of trio's "async/await all the way down" in some way, but I don't think this matters here - it might be a better developer experience in general, but it's not necessary to implement a correctly functioning server. I packaged up this bridge code in https://github.com/miracle2k/trio-protocol. So my long-winded point: What are your thoughts on this approach? Are you ok with an optional dependency on Finally, the trio code could forgo the existing code for h11 and httptools, but re-implement this stuff on top of trio without using the protocol interface, but... that seems like a lot of work with little benefit. |
I don't think it matters at all to the end user in this case. We're correctly handling back-pressure, we're correctly handling fully-drained transports on graceful shutdown, we've done the extra legwork that a trio implementation might have helped with anyway, so as far as the developer is concerned there's just an
I'm okay with it, yup - I can't see any better option. One way or another we want to have an implementation that we share, and trio's streams are different enough in flow from asyncio's protocols that I can't see a better approach. Since #118 we've now got some timeout behavior in there. I'd far prefer to be able to be using trio's more structured timeout "with" blocks. I can see how I'd do something like that for the request/response timeout easily enough, since it wraps the ASGI call, but I can't see how to make that work with the keep alive timeouts, which span between requests. I think the |
I can see how we could restructure things slightly to avoid using |
My plan is to work on this sometime in the next week or so; I think we can make |
I'm very keen on moving forward with this, here's my initial analysis looking at the current state of the code. :-) ApproachWe won't be able to take strictly the same approach as in HTTPX, i.e. just have an The reason is that server code uses more low-level APIs, in this case we use That said, for some components (esp those at the ASGI layer rather than the networking layer), it wil make sense to be implemented using an So we'll need both:
PlanFirst, we'll need to isolate any asyncio-specific code:
These are the places where I'm seeing we do Once those are isolated into |
I've done more investigation, and from what I found out it seems there may not be a safe, simple way to transition incrementally from the current state to an "async agnostic" state, at least without partial rewrites of lots of stuff. For example, the "protocols" code ( Additive rewrite?So I'm exploring the space of what's possible on a branch: https://github.com/encode/uvicorn/tree/fm/async-agnostic My approach there is to rewrite parts of Uvicorn (basically everything that touches networking, plus the lifespan code) independently of the existing code paths. The rewritten code uses an The new code path can be taken by passing an Current statusI'm quite happy to say that I've got a lot working on that branch yet:
File structure (approximately): _async_agnostic/
# Async backends, similar to approach in HTTPCore. Provides an `AsyncSocket` abstraction, among other things
backends/
# HTTP/1.1 implementation
http11/
parsers/ # Sans-I/O parsers
base.py # Interface (inspired by `h11.Connection`)
h11.py
httptools.py
handler.py # Handler interface implementation.
# Server implementation.
# Relies on a "handler" interface: `async handler(sock, state, config) -> None`
# This is similar to trio's and curio's server APIs, which expect
# a `async handler(stream) -> None` function to be passed.
# This allows preparing support for HTTP/2 (basically: add `http2/`, and a handler).
# The server is responsible for "things around actually serving connections", such as
# running the lifespan task, or listening for shutdown signals.
# (This means handlers are testable in isolation.)
server.py The I also have a Is this madness?Well, probably. :-) Obviously we can't ever downright replace what's in Uvicorn right now with what I'm working on in that branch. Also I'm aware the code I'm working on mixes two problems: async-agnosticity, and preparing for HTTP/2 support. It's very possible that preparing HTTP/2 support could be done separately. (And an internal "handler API" could be a sensible way to go. It should make it much easier to add HTTP/2 support incrementally, by simply adding an HTTP/2 handler, implemented with But as for the async-agnostic side of things, there may be a way forward in the form of a "new implementation" that exists alongside the existing one, and that users are able to opt into (using If we think this is an interesting path forward, we could go for something like this:
There are pain points, though:
But…
Happy to read thoughts about this! I keep working on my branch since I think it's a valuable bunch of work anyway (I've already cumulatively spent a few days straight on it!), at least for figuring out what can be done and how. (Also interesting to think about this wrt HTTPCore: I think we have a "someday" ambition to build a server-side minimal HTTP server implementation, compatible with sync/async, with support for various async libraries, various parsers, etc…) |
this is brillant and very elegantly written. |
Actually it works very well. Well, at least, the tests pass! https://github.com/encode/uvicorn/blob/fm/async-agnostic/tests/async_agnostic/test_main.py I did have to figure out some gnarly issues (esp binding to a raw |
well I looked very rapidly but :
which incidentally is funny because I've got kind of the same pickle issue but on windows only in my attempt to deal with multiprocessing better in #853 |
Okay, I revert: I meant to say "It works very well on my machine" (macOS). 😄 |
this is linux where you see this stacktrace, the windows issue I'm mentioning is another story from the PR I linked above. anyway, we can talk about it in another thread but the whole Multiprocess class is currently unable to handle properly the various spawned servers mostly because it uses a You can then add to that the fact that signals in the multiprocess case should be dealt with differently (ie ignore SIGINT first mostly then start the processes then registering signals) than in a the single runner case where you just have to put a signal on the loop, and that multiprocessing context (spawn vs fork) is different from os to os and you see the issue is complex. I'm mentioning the 2 points above because it has deep impacts on the way the run method of the Server (the one from the approach taken in #853 works fine on linux, hangs on windows because it can't pickle stuff (I think the ssl context but not sure) and I didnt test on macos. |
One of the issues here is that adding trio support gracefully would involve some changes that negatively impact the micro-benchmarking performance of uvicorn. The server handling isn't super reusable across differing frameworks, but that's partly because it's geared towards "performance". It's a bit of a nonsense, but at the same time it's important that we're adequately demonstrating that ASGI servers/frameworks are equally as fast as tightly-coupled alternatives. It might be that if we want to be able to preserve that, then we just need to suck up that there will be duplication, and have a fast path for asyncio-specifically, and a neat path for wider ecosystem support. |
hello, is AnyIO something that could help pushing this forward? |
@kontsaki We know about AnyIO yup, thanks for the suggestion. From my POV at this point this is more of a change management issue (how to get this in incrementally, in a way that's low risk for existing users, and easy enough to review by maintainers and contributors, ...), rather than a technical one. We've already achieved this async agnosticism in HTTPX, we know it's possible and that there are various technical approaches, but we "just" have to do put in the work. :) It's a tad harder than in HTTPX because of how Uvicorn already supports a wide amount of users; we're not starting from a clean slate. |
Is supporting structured concurrency (Trio) an ASGI issue? For instance, if I want to run a background task during the lifespan of my application, there's no way to do that (AFAICT) conforming to structured concurrency principles. A simple example using starlette: from starlette import Starlette
async def lifespan(app):
async with trio.open_nursery() as nursery:
nursery.start_soon(some_background_task)
yield
app = Starlette(lifespan=lifespan) Why this is bad is well explained here: https://anyio.readthedocs.io/en/stable/cancellation.html#avoiding-cancel-scope-stack-corruption. TL;DR:
|
Many web apps do probably want a way to start background work that outlives a single request handler. If that's what your design needs, then structured concurrency doesn't object, it just wants you to be explicit about lifetimes :-) Quart is an ASGI framework, and it handles this using standard ASGI features. Specifically, it uses a "lifespan" handler to start up a nursery at startup that stays open as long as the app is running, and make it available to user code as an attribute on their |
No.
Ah, that Starlette could perfectly well tweak things around there to ensure there's a nursery available for the duration of the app lifespan, in the same way that Quart already does, yup. (*) I needed to read up on this a bit more, and found the Trio docs helpful so I'm going to reference them here for other potential readers: https://trio.readthedocs.io/en/stable/reference-core.html#cancel-scopes-and-nurseries |
Thanks @tomchristie and @njsmith . I'm able to use class TaskGroupFastAPI(FastAPI):
async def __call__(self, scope, receive, send):
async with anyio.create_task_group() as task_group:
self.task_group = task_group
await super().__call__(scope, receive, send)
app = TaskGroupFastAPI()
async def manager_task(shutdown_event):
async with manager:
await shutdown_event.wait()
async def lifespan(app):
shutdown_event = anyio.create_event()
await app.task_group.spawn(manager_task, shutdown_event)
try:
yield
finally:
await shutdown_event.set()
# XXX: Better support lifespan context
# https://github.com/tiangolo/fastapi/issues/2943
app.router.lifespan_context = lifespan Seems like a reasonable way forward. |
Starlette currently does the right thing with |
Regarding trio support in uvicorn, currently |
@graingert Let's remember from past discussions that we want to keep a fast-track code path for asyncio, as Uvicorn is also meant to be a flagship for a fast and lightweight async server. This means keeping the existing asyncio-optimized code, and so I don't think we should make anyio required like we did in Starlette (where I think we could have done without anyio magic too for the sake of future maintenance, but I didn't participate in that decision). I had also laid out ways we could move forward without anyio for trio support. I'd understand the motivation for using that as a first step for trio support, but this is just a note that I hope it is clear that we really do want to keep the existing asyncio code in place. |
Closing this off for now with the same rationale as #47 For now I think the best option is us helping promote Hypercorn as a |
FYI I started anycorn which is a fork of Hypercorn that uses AnyIO instead of having a separate code base for asyncio and Trio. |
@davidbrochart Interesting, thanks. You've also prompted me to revisit this...
Somewhat. The WSGI-like messaging API isn't fundamentally incompatible, but doesn't neatly match up to how you'd design a request/response + lifespans SC API. |
Revisiting some thoughts on this following on from #118
I'm not against having trio-support as one of the built-in options. Presumably that'd mean using
--loop trio
, and having that switch out:--http auto
.The
asyncio
/trio
/curio
split is problematic, but it's an ecosystem issue, rather than an ASGI issue (ASGI is just an async/await interface, and is agnostic what event loop implementation is used to call into that interface.)The problem is that whatever event loop implementation you're running in must also match any async primitives used in the application codebase.
If you're just using
async
/await
andsend
/receive
, then it doesn't matter, but as soon as you start using things likeasyncio.sleep()/trio.sleep()
, or creating concurrent tasks, or accessing network or file I/O directly, then you're locked into whichever system you're in.I'd say that Trio is indisputably more refined and more properly constrained than
asyncio
, but the ecosystem split is problematic.The text was updated successfully, but these errors were encountered: