Improve ProxyHeadersMiddleware (#2468)

* Allow ip ranges for FORWARDED_ALLOW_IPS

* Fix host for empty x-forwarded-for header

* Add multi-host ip-range testcase, run scripts/lint

* Re-run black with line-length 88

* Wrap comments to fit in line-length

* Another missed line-length violation

* Fix __contains__ annotations

* Add support for IPv6

* Fix linting

* Fix test cases

* Update docs, comments

* Update index's usage

* Fix more linting

* Better unix socket support

* Expand proxy documentation

* More docs and code comments

* Refactor common test cases, add cases for UDS

* Update settings docs

* Remove "better UDS handling"

* Remove code leading to coverage failures

* Apply suggestions from code review

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

* Fixes, and more tests

* Update uvicorn/config.py

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

* Run ruff formatter

* Replace cast with type: ignore

* remove make_x_headers

* nitpicks

* remove unneeded return

* fix docs

---------

Co-authored-by: Patrick Düggelin <dueggelin@textshuttle.ai>
Co-authored-by: Nicholas Hairs <info@nicholashairs.com>
Co-authored-by: Nicholas Hairs <nicholashairs@gmail.com>

docs/deployment.md

@@ -93,10 +93,12 @@ Options:
                                   Enable/Disable default Server header.
   --date-header / --no-date-header
                                   Enable/Disable default Date header.
-  --forwarded-allow-ips TEXT      Comma separated list of IPs to trust with
-                                  proxy headers. Defaults to the
-                                  $FORWARDED_ALLOW_IPS environment variable if
-                                  available, or '127.0.0.1'.
+  --forwarded-allow-ips TEXT      Comma separated list of IP Addresses, IP
+                                  Networks, or literals (e.g. UNIX Socket
+                                  path) to trust with proxy headers. Defaults
+                                  to the $FORWARDED_ALLOW_IPS environment
+                                  variable if available, or '127.0.0.1'. The
+                                  literal '*' means trust everything.
   --root-path TEXT                Set the ASGI 'root_path' for applications
                                   submounted below a given URL path.
   --limit-concurrency INTEGER     Maximum number of concurrent connections or
@@ -279,12 +281,9 @@ Using Nginx as a proxy in front of your Uvicorn processes may not be necessary,
 
 In managed environments such as `Heroku`, you won't typically need to configure Nginx, as your server processes will already be running behind load balancing proxies.
 
-The recommended configuration for proxying from Nginx is to use a UNIX domain socket between Nginx and whatever the process manager that is being used to run Uvicorn.
-Note that when doing this you will need to run Uvicorn with `--forwarded-allow-ips='*'` to ensure that the domain socket is trusted as a source from which to proxy headers.
+The recommended configuration for proxying from Nginx is to use a UNIX domain socket between Nginx and whatever the process manager that is being used to run Uvicorn. If using Uvicorn directly you can bind it to a UNIX domain socket using `uvicorn --uds /path/to/socket.sock <...>`.
 
-When fronting the application with a proxy server you want to make sure that the proxy sets headers to ensure that the application can properly determine the client address of the incoming connection, and if the connection was over `http` or `https`.
-
-You should ensure that the `X-Forwarded-For` and `X-Forwarded-Proto` headers are set by the proxy, and that Uvicorn is run using the `--proxy-headers` setting. This ensures that the ASGI scope includes correct `client` and `scheme` information.
+When running your application behind one or more proxies you will want to make sure that each proxy sets appropriate headers to ensure that your application can properly determine the client address of the incoming connection, and if the connection was over `http` or `https`. For more information see [Proxies and Forwarded Headers][#proxies-and-forwarded-headers] below.
 
 Here's how a simple Nginx configuration might look. This example includes setting proxy headers, and using a UNIX domain socket to communicate with the application server.
 
@@ -333,7 +332,7 @@ In those cases, you might want to use an ASGI middleware to set the `client` and
 
 ## Running behind a CDN
 
-Running behind a content delivery network, such as Cloudflare or Cloud Front, provides a serious layer of protection against DDOS attacks. Your service will be running behind huge clusters of proxies and load balancers that are designed for handling huge amounts of traffic, and have capabilities for detecting and closing off connections from DDOS attacks.
+Running behind a content delivery network, such as Cloudflare or Cloud Front, provides a serious layer of protection against DDoS attacks. Your service will be running behind huge clusters of proxies and load balancers that are designed for handling huge amounts of traffic, and have capabilities for detecting and closing off connections from DDoS attacks.
 
 Proper usage of cache control headers can mean that a CDN is able to serve large amounts of data without always having to forward the request on to your server.
 
@@ -362,3 +361,44 @@ $ gunicorn --keyfile=./key.pem --certfile=./cert.pem -k uvicorn.workers.UvicornW
 [nginx_websocket]: https://nginx.org/en/docs/http/websocket.html
 [letsencrypt]: https://letsencrypt.org/
 [mkcert]: https://github.com/FiloSottile/mkcert
+
+## Proxies and Forwarded Headers
+
+When running an application behind one or more proxies, certain information about the request is lost.
+To avoid this most proxies will add headers containing this information for downstream servers to read.
+
+Uvicorn currently supports the following headers:
+
+- `X-Forwarded-For` ([MDN Reference](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For))
+- `X-Forwarded-Proto`([MDN Reference](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Proto))
+
+Uvicorn can use these headers to correctly set the client and protocol in the request.
+However as anyone can set these headers you must configure which "clients" you will trust to have set them correctly.
+
+Uvicorn can be configured to trust IP Addresses (e.g. `127.0.0.1`), IP Networks (e.g. `10.100.0.0/16`),
+or Literals (e.g. `/path/to/socket.sock`). When running from CLI these are configured using `--forwarded-trust-ips`.
+
+!!! Warning "Only trust clients you can actually trust!"
+    Incorrectly trusting other clients can lead to malicious actors spoofing their apparent client address to your application.
+
+For more information, check [`ProxyHeadersMiddleware`](https://github.com/encode/uvicorn/blob/master/uvicorn/middleware/proxy_headers.py).
+
+### Client Port
+
+Currently if the `ProxyHeadersMiddleware` is able to retrieve a trusted client value then the client's port will be set to `0`. This is because port information is lost when using these headers.
+
+### UNIX Domain Sockets (UDS)
+
+Although it is common for UNIX Domain Sockets to be used for communicating between various HTTP servers, they can mess with some of the expected received values as they will be various non-address strings or missing values.
+
+For example:
+
+- when NGINX itself is running behind a UDS it will add the literal `unix:` as the client in the `X-Forwarded-For` header.
+- When Uvicorn is running behind a UDS the initial client will be `None`.
+
+
+### Trust Everything
+
+Rather than specifying what to trust, you can instruct Uvicorn to trust all clients using the literal `"*"`.
+You should only set this when you know you can trust all values within the forwarded headers (e.g. because
+your proxies remove the existing headers before setting their own).

docs/index.md

@@ -163,10 +163,12 @@ Options:
                                   Enable/Disable default Server header.
   --date-header / --no-date-header
                                   Enable/Disable default Date header.
-  --forwarded-allow-ips TEXT      Comma separated list of IPs to trust with
-                                  proxy headers. Defaults to the
-                                  $FORWARDED_ALLOW_IPS environment variable if
-                                  available, or '127.0.0.1'.
+  --forwarded-allow-ips TEXT      Comma separated list of IP Addresses, IP
+                                  Networks, or literals (e.g. UNIX Socket
+                                  path) to trust with proxy headers. Defaults
+                                  to the $FORWARDED_ALLOW_IPS environment
+                                  variable if available, or '127.0.0.1'. The
+                                  literal '*' means trust everything.
   --root-path TEXT                Set the ASGI 'root_path' for applications
                                   submounted below a given URL path.
   --limit-concurrency INTEGER     Maximum number of concurrent connections or

docs/settings.md

@@ -88,9 +88,9 @@ Note that WSGI mode always disables WebSocket support, as it is not supported by
 ## HTTP
 
 * `--root-path <str>` - Set the ASGI `root_path` for applications submounted below a given URL path.
-* `--proxy-headers` / `--no-proxy-headers` - Enable/Disable X-Forwarded-Proto, X-Forwarded-For, X-Forwarded-Port to populate remote address info. Defaults to enabled, but is restricted to only trusting
+* `--proxy-headers` / `--no-proxy-headers` - Enable/Disable X-Forwarded-Proto, X-Forwarded-For to populate remote address info. Defaults to enabled, but is restricted to only trusting
 connecting IPs in the `forwarded-allow-ips` configuration.
-* `--forwarded-allow-ips` <comma-separated-list> Comma separated list of IPs to trust with proxy headers. Defaults to the `$FORWARDED_ALLOW_IPS` environment variable if available, or '127.0.0.1'. A wildcard '*' means always trust.
+* `--forwarded-allow-ips` <comma-separated-list> Comma separated list of IP Addresses, IP Networks, or literals (e.g. UNIX Socket path) to trust with proxy headers. Defaults to the `$FORWARDED_ALLOW_IPS` environment variable if available, or '127.0.0.1'. The literal `'*'` means trust everything.
 * `--server-header` / `--no-server-header` - Enable/Disable default `Server` header.
 * `--date-header` / `--no-date-header` - Enable/Disable default `Date` header.