Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document how to create ASGI middlewares #1656

Merged
merged 34 commits into from
Jun 30, 2022
Merged
Changes from 13 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
09ce984
Document how to create ASGI middlewares
Kludex May 31, 2022
7c51da5
Fix typo
Kludex May 31, 2022
7975168
Merge branch 'master' into document-asgi-middleware
Kludex Jun 2, 2022
a51d5c1
Merge remote-tracking branch 'origin/master' into document-asgi-middl…
Kludex Jun 2, 2022
ea082ab
Add more documentation on how to create pure ASGI middlewares
Kludex Jun 3, 2022
9672c0d
Merge branch 'document-asgi-middleware' of github.com:encode/starlett…
Kludex Jun 3, 2022
961b98d
Apply suggestions from code review
Kludex Jun 3, 2022
7aa5575
Apply suggestions from code review
Kludex Jun 8, 2022
a4800fc
remove empty spaces
Kludex Jun 8, 2022
6f9eda4
Add asgiref hyperlink
Kludex Jun 8, 2022
3384cb1
Fix per-request section
Kludex Jun 8, 2022
c6568bd
Fix scope info
Kludex Jun 8, 2022
46662e9
Remove example section
Kludex Jun 8, 2022
6948b81
Merge branch 'master' into document-asgi-middleware
Kludex Jun 8, 2022
c737fb1
Merge branch 'master' into document-asgi-middleware
Kludex Jun 12, 2022
9a3e9e8
Update docs/middleware.md
Kludex Jun 16, 2022
b0f8029
Merge branch 'master' into document-asgi-middleware
Kludex Jun 16, 2022
3dd6b11
Update docs/middleware.md
adriangb Jun 16, 2022
514c818
Merge branch 'master' into document-asgi-middleware
Kludex Jun 19, 2022
5150507
Apply suggestions from code review
Kludex Jun 19, 2022
0408af2
Merge branch 'master' into document-asgi-middleware
Kludex Jun 27, 2022
5b54dd9
Update docs/middleware.md
Kludex Jun 28, 2022
aa6bbae
Merge branch 'master' into document-asgi-middleware
Kludex Jun 28, 2022
4f12d08
Apply suggestions from code review
Kludex Jun 28, 2022
efbf115
fix typo
adriangb Jun 28, 2022
d218689
Merge branch 'master' into document-asgi-middleware
Kludex Jun 28, 2022
a2c504c
Apply suggestions from code review
Kludex Jun 30, 2022
74664e1
Apply suggestions from code review
Kludex Jun 30, 2022
77e2c27
Apply suggestions from code review
Kludex Jun 30, 2022
b2af992
Update docs/middleware.md
Kludex Jun 30, 2022
046f4d4
Add reference to Starlette components
Kludex Jun 30, 2022
263de76
Add `MutableHeaders` import
Kludex Jun 30, 2022
8868e49
Fix typos from suggestions
Kludex Jun 30, 2022
756036b
Merge remote-tracking branch 'origin/master' into document-asgi-middl…
Kludex Jun 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
265 changes: 265 additions & 0 deletions docs/middleware.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,271 @@ around explicitly, rather than mutating the middleware instance.
- It's not possible to use `BackgroundTasks` with `BaseHTTPMiddleware`. Check [#1438](https://github.com/encode/starlette/issues/1438) for more details.
- Using `BaseHTTPMiddleware` will prevent changes to [`contextlib.ContextVar`](https://docs.python.org/3/library/contextvars.html#contextvars.ContextVar)s from propagating upwards. That is, if you set a value for a `ContextVar` in your endpoint and try to read it from a middleware you will find that the value is not the same value you set in your endpoint (see [this test](https://github.com/encode/starlette/blob/621abc747a6604825190b93467918a0ec6456a24/tests/middleware/test_base.py#L192-L223) for an example of this behavior).

## Pure ASGI Middleware

Due to how ASGI was designed, we are able to build a chain of ASGI applications, on which each application calls the next one.
Kludex marked this conversation as resolved.
Show resolved Hide resolved
Each element of the chain is an [`ASGI`](https://asgi.readthedocs.io/en/latest/) application by itself, which per definition, is also a middleware.

This is also an alternative approach in case the limitations of `BaseHTTPMiddleware` are a problem.
Kludex marked this conversation as resolved.
Show resolved Hide resolved

### Guiding principles

The most common way to create an ASGI middleware is with a class.

```python
class ASGIMiddleware:
def __init__(self, app):
self.app = app

async def __call__(self, scope, receive, send):
await self.app(scope, receive, send)
```

The middleware above is the most basic ASGI middleware. It receives an ASGI application as an argument for its constructor, and implements the `async __call__` method. This method should accept `scope`, which contains information about the current connection, and `receive` and `send` which allow to exchange ASGI event messages with the ASGI server (learn more in the [ASGI specification](https://asgi.readthedocs.io/en/latest/specs/index.html)).

As an alternative for the class approach, you can also use a function:

```python
import functools

def asgi_middleware():
Kludex marked this conversation as resolved.
Show resolved Hide resolved
def asgi_decorator(app):
@functools.wraps(app)
async def wrapped_app(scope, receive, send):
await app(scope, receive, send)
return wrapped_app
return asgi_decorator
```

!!! note
Kludex marked this conversation as resolved.
Show resolved Hide resolved
The function pattern is not commonly spread, but you can check a more advanced implementation of it on
[asgi-cors](https://github.com/simonw/asgi-cors/blob/10ef64bfcc6cd8d16f3014077f20a0fb8544ec39/asgi_cors.py).

#### `Scope` types

As we mentioned, the scope holds the information about the connection. There are three types of `scope`s:

- [`lifespan`](https://asgi.readthedocs.io/en/latest/specs/lifespan.html#scope) is a special type of scope that is used for the lifespan of the ASGI application.
Kludex marked this conversation as resolved.
Show resolved Hide resolved
- [`http`](https://asgi.readthedocs.io/en/latest/specs/www.html#http-connection-scope) is a type of scope that is used for HTTP requests.
- [`websocket`](https://asgi.readthedocs.io/en/latest/specs/www.html#websocket-connection-scope) is a type of scope that is used for WebSocket connections.

If you want to create a middleware that only runs on HTTP requests, you'd write something like:

```python
class ASGIMiddleware:
def __init__(self, app):
self.app = app

async def __call__(self, scope, receive, send):
if scope["type"] != "http":
return await self.app(scope, receive, send)

# Do something here!
await self.app(scope, receive, send)
```
In the example above, if the `scope` type is `lifespan` or `websocket`, we'll directly call the `self.app`.
Kludex marked this conversation as resolved.
Show resolved Hide resolved

The same applies for the other scopes.
adriangb marked this conversation as resolved.
Show resolved Hide resolved

!!! note
Kludex marked this conversation as resolved.
Show resolved Hide resolved
Middleware classes should be stateless -- see [Per-request state](#per-request-state) if you do need to store per-request state.

#### Wrapping `send` and `receive`

A common pattern, that you'll probably need to use is to wrap the `send` or `receive` callables.

For example, here's how we could write a middleware that logs the response status code, which we'd obtain
by wrapping the `send` with the `send_wrapper` callable:

```python
class LogStatusCodeMiddleware:
def __init__(self, app):
self.app = app

async def __call__(self, scope, receive, send):
if scope["type"] != "http":
return await self.app(scope, receive, send)

status_code = 500

async def send_wrapper(message):
if message["type"] == "http.response.start":
status_code = message["status"]
await send(message)

await self.app(scope, receive, send_wrapper)

print("This is a primitive access log")
print(f"status = {status_code}")
```

!!! info
You can check a more advanced implementation of the same rationale on [asgi-logger](https://github.com/Kludex/asgi-logger/blob/main/asgi_logger/middleware.py).

#### Type annotations
Kludex marked this conversation as resolved.
Show resolved Hide resolved

There are two ways of annotating a middleware: using Starlette itself or [`asgiref`](https://github.com/django/asgiref).

Using Starlette, you can do as:

```python
from starlette.types import Message, Scope, Receive, Send
from starlette.applications import Starlette


class ASGIMiddleware:
def __init__(self, app: Starlette) -> None:
self.app = app
Kludex marked this conversation as resolved.
Show resolved Hide resolved

async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] == "http":
async def send_wrapper(message: Message) -> None:
await send(message)
return await self.app(scope, receive, send_wrapper)
await self.app(scope, receive, send)
Kludex marked this conversation as resolved.
Show resolved Hide resolved
```

Although this is easy, you may prefer to be more strict. In which case, you'd need to use `asgiref`:

```python
from asgiref.typing import ASGI3Application, Scope, ASGISendCallable
from asgiref.typing import ASGIReceiveEvent, ASGISendEvent


class ASGIMiddleware:
def __init__(self, app: ASGI3Application) -> None:
self.app = app

async def __call__(self, scope: Scope, receive: ASGIReceiveCallable, send: ASGISendCallable) -> None:
if scope["type"] == "http":
async def send_wrapper(message: ASGISendCallable) -> None:
Kludex marked this conversation as resolved.
Show resolved Hide resolved
await send(message)
return await self.app(scope, receive, send_wrapper)
await self.app(scope, receive, send)
```

The `ASGI3Application` is meant to represent an ASGI application that follows the third version of the standard.
Starlette itself is an ASGI 3 application.
Kludex marked this conversation as resolved.
Show resolved Hide resolved
Kludex marked this conversation as resolved.
Show resolved Hide resolved

!!! note
Kludex marked this conversation as resolved.
Show resolved Hide resolved
You can read more about ASGI versions on the [Legacy Applications section on the ASGI documentation](https://asgi.readthedocs.io/en/latest/specs/main.html#legacy-applications).
Kludex marked this conversation as resolved.
Show resolved Hide resolved

### Reusing Starlette components

If you need to work with request or response data, you may find it more convenient to reuse Starlette data structures (`Request`, `Headers`, `QueryParams`, `URL`, etc) rather than work with raw ASGI data. All these components can be built from the ASGI `scope`, `receive` and `send`, allowing you to work on pure ASGI middleware at a higher level of abstraction.

For example, we can create a `Request` object, and work with it.
```python
from starlette.requests import Request

class ASGIMiddleware:
def __init__(self, app):
self.app = app

async def __call__(self, scope, receive, send):
if scope["type"] == "http":
request = Request(scope, receive, send)
# Do something here!
await self.app(scope, receive, send)
```

Or we might use `MutableHeaders` to change the response headers:

```python
class ExtraResponseHeadersMiddleware:
def __init__(self, app, headers):
self.app = app
self.headers = headers

async def __call__(self, scope, receive, send):
if scope["type"] != "http":
return await self.app(scope, receive, send)

async def wrapped_send(message):
if message["type"] == "http.response.start":
headers = MutableHeaders(scope=message)
for key, value for self.headers:
adriangb marked this conversation as resolved.
Show resolved Hide resolved
headers.append(key, value)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
headers = MutableHeaders(scope=message)
for key, value for self.headers:
headers.append(key, value)
headers = MutableHeaders(scope=message)
for key, value for self.headers:
headers.append(key, value)
message["headers"] = headers.raw # ASGI headers is a List[Tuple[bytes, bytes]]

I think this more explicit assignment is clear than relying on MutableHeaders to modify message in-place.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementations I've seen use it in the way I'm using it. I do agree with you on which one is clear, but I'd rather prefer the user to think a bit on a small documentation like this instead of another more complicated code source.

That being said, if you are strong about it, we can add a note about it below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't say I feel strongly about it, but it is 1 LOC

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also there’s a typo here: for key, value in self.headers

await send(message)

await self.app(scope, receive, wrapped_send)
```

### Per-request state

ASGI middleware classes should be stateless, as we typically don't want to leak state across requests.

The risk is low when defining wrappers inside `__call__`, as state would typically be defined as inline variables.

But if the middleware grows larger and more complex, you might be tempted to refactor wrappers as methods. Still, state should not be stored in the middleware instance. Instead, if you need to manipulate per-request state, you may write a separate `Responder` class:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option which I personally like is using a generator:

async def tweaked_send() -> AsyncGenerator[None, Message]:
    message = yield
    headers = Headers(raw=message["headers"])
    should_tweak = headers.get("X-Tweak") == "1"
    await send(message)

    while True:
        message = yield
        if not should_tweak:
            await send(message)
        else:
            raise NotImplementedError  # TODO: tweak

async with aclosing(tweaked_send()) as wrapped_send:
    await wrapped_send.__anext__()  # start the generator
    await self.app(scope, receive, wrapped_send.asend)

Maybe it's worth mentioning?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s wild!

Also, hmm… I’m less convinced about documenting the « responder class » now, although I’m aware I’m the one who brought it up initially. That’s because that’s a style I’ve been using. I think I was itches by the fact we’d be redefining a function every time with the closure style.

There’s a couple of styles that could be used, eg. responder class, or this generator style, etc. I think this section is complex and new enough for most readers already that we should stick to just one.

I’m curious what you would think about dropping this section. We’ve got a note somewhere on the fact that middleware classes should be stateless. The closure style actually doesn’t incite users to use class attributes, so it doesn’t pose a « per request state » problem, so we could drop this entire section.

Copy link
Member

@adriangb adriangb Jun 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s wild!

It is a neat pattern huh! Generators are great little state machines that you can ratchet forwards and pass messages to. And of course you can go full functional:

def functional_middleware(app: ASGIApp) -> ASGIApp:
    async def new_app(scope: Scope, receive: Receive, send: Send) -> None:
        async def tweaked_send() -> "AsyncGenerator[None, Message]":
            message = yield
            headers = Headers(raw=message["headers"])
            should_tweak = headers.get("X-Tweak") == "1"
            await send(message)

            while True:
                message = yield
                if not should_tweak:
                    await send(message)
                else:
                    await send(message)

        async with aclosing(tweaked_send()) as wrapped_send:
            await wrapped_send.__anext__()  # start the generator
            await app(scope, receive, wrapped_send.asend)

    return new_app

I’m curious what you would think about dropping this section.

I do think this section is getting a bit in depth, maybe at this point we should let users fly and use their imagination and dev skills to figure out which solution they want to use? We'll still need to explain why they should not use attributes on the middleware. And maybe the best way to do that is with a counterexample, and if I had to pick one style I'd pick the responder class because I think it'll be easier to understand.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to re read this section several times, I think it would be extremely clear to have a don't do this example and another one following, do this instead

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to simplify this section and apply @euri10 's comment. I may remove the snippets here, and point to external implementations...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can even point to our own internal implementations? GZipMiddleware would be a good example.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hehe, I always struggle with generators. 😅 What I don't like about yield for receiving stuff is that there's no way to have type inference, any values obtained at the left side would have to be manually annotated to have typing support (autocompletion, inline errors, mypy, etc). I'm also not sure what better approach is there. But anyway...

I would leave this PR without more examples, to try and limit the scope. Maybe a future PR could include suggesting more examples, but I think it would be good to have this close to what is already there, without adding too much.

I would just add another phrase clarifying the stateless stuff. I'm adding that as a separate suggestion below. 🤓

Kludex marked this conversation as resolved.
Show resolved Hide resolved

```python
from functools import partial

class TweakMiddleware:
"""
Make a change to the response body if 'X-Tweak' is
present in the reponse headers.
"""

async def __call_(self, scope, receive, send):
if scope["type"] != "http":
return await self.app(scope, receive, send)

responder = MaybeTweakResponder(self.app)
await responder(scope, receive, send)

class TweakResponder:
def __init__(self, app):
self.app = app
self.should_tweak = False

async def __call__(self, scope, receive, send):
send = partial(self.maybe_send_with_tweaks, send=send)
await self.app(scope, receive, send)

async def maybe_send_with_tweaks(self, message, send):
if message["type"] == "http.response.start":
headers = Headers(raw=message["headers"])
self.should_tweak = headers.get("X-Tweak") == "1"
await send(message)
return

if message["type"] == "http.response.body":
if not self.should_tweak:
await send(message)
return

# Actually tweak the response body...
```

See also [`GZipMiddleware`](https://github.com/encode/starlette/blob/9ef1b91c9c043197da6c3f38aa153fd874b95527/starlette/middleware/gzip.py) for a full example of this pattern.

### Storing context in `scope`

As we know by now, the `scope` holds the information about the connection.

As per the ASGI specifications, any application can store custom information on the `scope`.
To be precise, it should be stored under the `extensions` key.
Kludex marked this conversation as resolved.
Show resolved Hide resolved

```python
class ASGIMiddleware:
def __init__(self, app):
self.app = app

async def __call__(self, scope, receive, send):
scope["extensions"] = {"super.extension": True}
await self.app(scope, receive, send)
Kludex marked this conversation as resolved.
Show resolved Hide resolved
```
On the example above, we stored an extension called "super.extension". That can be used by the application itself, as the scope is forwarded to it.
Kludex marked this conversation as resolved.
Show resolved Hide resolved

!!! important
This documentation should be enough to have a good basis on how to create an ASGI middleware.
Nonetheless, there are great articles about the subject:

- [Introduction to ASGI: Emergence of an Async Python Web Ecosystem](https://florimond.dev/en/posts/2019/08/introduction-to-asgi-async-python-web/)
- [How to write ASGI middleware](https://pgjones.dev/blog/how-to-write-asgi-middleware-2021/)

## Using middleware in other frameworks

To wrap ASGI middleware around other ASGI applications, you should use the
Expand Down