HTTPCore II - the director's cut

[tomchristie]

I've spent a significant amount of time lately working on a redesign of httpcore.
There's some key issues that this redesign resolves...

• The connection pooling design currently isn't quite right. There's a critical bug that can occur when maxing out the number of concurrent connections. PoolTimeout when num tasks in asyncio.gather() exceeds client max_connections httpx#1171 There's also some other minor niggles... for example if you concurrently send several HTTP/2 requests, and then inspect the state of the pool, you'll find that they end up on separate connections.
• The interaction between proxy connection pools and connections isn't quite right. See NewConnectionRequired exception raised when running behind a proxy #335

The redesign that I've been working on at https://github.com/tomchristie/httpcore-the-directors-cut is now ready to go. It addresses these issues with a reworking from scratch. It's well documented, and has a more directly-usable API than this current version. It's also got 100% test coverage.

There's some other neat bits in there too, such as the "trace" extension, which gives a hook into inspecting the internal events that occur during the request/response cycle. This extension is needed for the HTTPX cli, in order to...

• Log the point at which the connection is established, and the IP/port on which it is made.
• Determine if the outgoing request should log as HTTP/1.1 or HTTP/2, rather than having to assume it's HTTP/2 if the --http2 flag was passed. (Which may not actually be true.)
• Log SSL version info / certificate info.

Next question is how to best to migrate over to the new version. Most likely I'm going to need to make a major pull request into this repo, that pretty much updates everything. (Happy to take suggestions, tho)

I'd also like to push a 0.14 release (or pre-release) to PyPI, so that I can open a 0.21 httpx PR that integrates against the new backend.

Aside: I'd previously planned to release HTTP 1.0, and then follow up with 1.1 (adding the cli tool) and follow up with 1.2 (switching to this redesign), but we're actually going to instead have 1.0 include the cli and the httpcore redesign.

[florimondmanca]

Nice, very nice. 😄

I skimmed through the new code… I must say, the Trace callback implementation is very elegant. Very glad that the new implementation would also solve the outstanding painpoints, notably about connection pooling.

A pre-PR review comment - looks like we're missing a timeout arg here? https://github.com/tomchristie/httpcore-the-directors-cut/blob/1a36c6f6645ad4f80dc14ae71854875d17282007/httpcore/_async/http11.py#L133

I also think having the redesign into 1.0 is the most sensible option, rather than issuing 1.0 and then risking breaking some nitty gritty cases post-1.0.

[tomchristie]

Okay, here we go... #420

Great work on spotting the timeout arg - I've address that one in the pull request.

I think the only sensible plan here is going to be to get it merged in and released. At that point it'll be easier for y'all to take a look through the new docs, and start giving things a prod. I can then get an HTTPX 0.21 pull request up and rolling, and plan on releasing that early next week (15th Nov).

Once httpx 0.21 is out we can see if we're hitting any unexpected networking issues, add socks proxy support, and plan on releasing 1.0 shortly after.

[tomchristie]

(Oh and yeah, it's using MkDocs for the documentation. That wasn't particularly a one-way-or-the-other thing, it was just easier for me to get everything I needed sorted that way. Also the mkdocstrings plugin is a better option than mkautodoc. I'm not sure it existed / was mature last time we were looking at docs tooling options.)

[tomchristie]

Okay, now published on PyPI, and docs live.

---> https://www.encode.io/httpcore/

[florimondmanca]

Giving a friendly ping to some people in the HTTPX ecosystem that might be interested in this, because they may be relying on HTTPCore to this day, based on Third Party Packages and grep.app:

• @lundberg (RESPX)
• @johtso (httpx-caching)
• @jlaine (aioquic, aiortc)

Since the plan is to merge and release #420 then issue HTTPX 0.21, I guess there's no actual action item for folks for now other than watch out for the HTTPCore release and the corresponding API changes. Just pinging to prevent surprises coming up from user issues. :-)

[lundberg]

Thanks @florimondmanca ❤️

[tomchristie]

Also worth noting that this httpcore change doesn't affect the httpx Transport API.
From an httpx API perspective, the 0.20 -> 0.21 change is an implementation change, rather than an API one. 👍

[Tinche]

Interesting! I was digging around through the httpx codebase the other day to fish out some metrics to expose and I went through the Async Connection Pool class. I remember being confused by the connection handling logic when the pool was full, and the fact both a lock and a semaphore was being used.