Low-level HTTP APIs

[scalablecory]

We've got desires for a) code sharing between HttpClient and Kestrel, and b) very precise control for partners with advanced needs (e.g. reverse proxies).

What do we think of an XmlReader-like API for low level HTTP? Minimal allocation, no connection pooling, no proxy support, etc. -- something like:

HttpStreamReader reader = ...;
HttpStreamWriter writer = ...;
ReadOnlySpan<byte> uri = ...;

await writer.WriteRequestAsync(HttpMethod.Get, uri);
await writer.WriteHeaderAsync(..., ...);
await writer.FlushMessage();

HttpTokenType token = await reader.ReadNextAsync();
Debug.Assert(token == HttpTokenType.Response);

if(reader.MessageStatusCode != 200)
{
	throw ...;
}

while((token = await reader.ReadNextAsync()) == HttpTokenType.Header)
{
	Span<byte> name = reader.HeaderName;
	Span<byte> value = reader.HeaderValue;
	UseHeader(name, value);
}

while(token == HttpTokenType.Content)
{
	UseContent(reader.Content);
	token = await reader.ReadNextAsync();
}

while(token == HttpTokenType.TrailingHeader)
{
	UseHeader(reader.HeaderName, reader.HeaderValue);
	token = await reader.ReadNextAsync();
}

Debug.Assert(token == HttpTokenType.MessageComplete);

@geoffkizer @stephentoub @davidfowl @davidsh @anurse

[Joe4evr]

Isn't this the entire point of ASP.NET's Project Bedrock?

[scalablecory]

@Joe4evr Bedrock is a connection establishment abstraction, not a HTTP API.

[Joe4evr]

Yeah, but the HTTP APIs would built on top of Bedrock.

[davidfowl]

What do we think of an XmlReader-like API for low level HTTP? Minimal allocation, no connection pooling, no proxy support, etc. -- something like:

You'd need a reader and writer API for HTTP/1 and 2 (maybe 3 as well?) for server and client. I worry about making the writing and reader of each header asynchronous. I also worry about using this for reading the body. We have lots of optimizations to avoid copying and this would likely negate that (though we could just skip the body altogether) or have different types for those readers.

A callback API, though much less convenient, would likely be more efficient here.

[Rodrigo-Andrade]

OMG plz do! Several bottlenecks in many projects i work are on HttpClient.

[scalablecory]

@Rodrigo-Andrade do you have more info? We'd be interested in hearing...

[Rodrigo-Andrade]

@scalablecory sory, for some reason i didn't see this sooner.

We have a bunch of microservices and an API Gateway that runs on dotnet core. I am kind of the "performance guy" and every time there are issues of performance i get to profile a different system on a different team. After most obvious optimizations, most performance to win are from HttpClient.

Things got really better, but making http requests from the HttpClient model is still too costly.

I cobbled up a simple http1.1 client that departs from HttpRequestMessage and the win was obvious. I could buffer most stuff, bypass most parsing and checking (since most requests can be proven well formed), allocating way less. Many workloads have more headers and long uris then big bodies, so serialization is a lesser issue (specially with serializers like MessagePack)

The cobbled up client is already a lot of code (even when its only http1.1, no ssl suport etc), so it would be nice to have lower level apis for this stuff, so we could benefit from all the work on http2 and 3 you guys are doing.

Tell me if you need more details (ill be sure to be checking this)

[freddyrios]

We are interested in this for our OSS gateway/reverse proxy that forwards http requests that are sent to it as udp packets across a data diode.

The udp receiving side uses the allocation free Socket stack, but the sending side has the allocations one gets when using HttpClient to send to remote servers.

[karelz]

@freddyrios why is HttpClient not sufficient for you? Do you have hard requirement to have no allocations? Or limit on allocations?

[freddyrios]

@karelz we were aiming for little to no allocations from the beginning, but quickly realized there are some unavoidable allocations with HttpClient. We did not consider the trade off worth it to remove the allocations by avoiding the HttpClient, so we do use it and it does work for the current throughput levels.

Our context is we get data through a 1 way physical link, so no type of flow control is possible, either the data gets handled at the right steady speed or not. Of course buffers change how much slow downs can be tolerated, so its only an issue for very high throughput and for long enough pauses/slow downs.