Rework HttpClient content buffering

[MihaZupan]

Contributes to #81628
Fixes #62845
Fixes #75631

When buffering the HttpContent, we currently use either a MemoryStream or LimitArrayPoolWriteStream.
If the Content-Length is known upfront, we'll allocate the correct buffer size right away.
If not, we'll keep resizing the buffer as we read the content.
If the buffered content is exposed outside of our control (outside of GetByteArrayAsync/GetStringAsync), we'll curently avoid using the pooled variant, even when growing.

This change does a couple things:

• If we know the Content-Length upfront, we'll still allocate the exact buffer upfront.
  • To limit the memory consumption for slow downloads, we limit that upfront allocation (currently at 16 MB).
  • For contents larger than 16 MB, this means we'll incur slighlty more memory copying, but such scenarios are already sub-optimal if we're allocating such buffers instead of streaming the response. There are also other places where we currently introduce memory copies that could be avoided with some effort.
• We'll use the pooled buffering approach even if the buffer is exposed, in which case we'll allocate a single non-pooled buffer at the end.
  • This significantly reduces memory allocations when the content length isn't known upfront (Excessive allocations in HttpContent.ReadAsByteArrayAsync #81628).
• Instead of resizing a single buffer, we rent ever-larger buffers from the pool and maintain a list of previous ones. This avoids memory copies when resizing buffers, and we only pay for it once at the end if we need an exact buffer.
  • This is the reason why GetByteArrayAsync results below show a time improvement even though allocations are the same.
  • Some operations don't need an exact buffer (GetStringAsync), so we start with a larger buffer (currently 16 KB) to avoid another memory copy for smaller responses, and to lower the number of buffers we have to rent & track.
• Added a bunch of tests to check that all methods behave the same w.r.t. limit enforcement (Inconsistent large response content validation in HttpClient #75631)

I kept the behavior the same w.r.t. allocating a new byte[] every time the user calls ReadAsByteArrayAsync (as discussed in #81628) for now, but we can do so in a follow-up.

---

Benchmarks when the response doesn't have a Content-Length header, in-memory only (no I/O).

Method	Toolchain	Length	Mean	Ratio	Allocated	Alloc Ratio
GetByteArrayAsync	main	10000	1,114.6 ns	1.00	10.81 KB	1.00
GetByteArrayAsync	pr	10000	994.3 ns	0.89	10.84 KB	1.00
GetAsync	main	10000	1,284.7 ns	1.00	28.93 KB	1.00
GetAsync	pr	10000	1,018.1 ns	0.79	10.84 KB	0.37
GetByteArrayAsync	main	100000	28,454.2 ns	1.00	98.73 KB	1.00
GetByteArrayAsync	pr	100000	28,038.7 ns	0.99	98.76 KB	1.00
GetAsync	main	100000	31,313.5 ns	1.00	251.58 KB	1.00
GetAsync	pr	100000	28,186.0 ns	0.90	98.76 KB	0.39
GetByteArrayAsync	main	1000000	127,633.7 ns	1.00	978.16 KB	1.00
GetByteArrayAsync	pr	1000000	110,941.5 ns	0.87	978.11 KB	1.00
GetAsync	main	1000000	151,716.7 ns	1.00	2032.28 KB	1.00
GetAsync	pr	1000000	110,231.7 ns	0.73	978.1 KB	0.48
GetByteArrayAsync	main	10000000	1,946,940.2 ns	1.00	9768.06 KB	1.00
GetByteArrayAsync	pr	10000000	1,113,033.0 ns	0.57	9769.06 KB	1.00
GetAsync	main	10000000	2,720,321.2 ns	1.04	32553.84 KB	1.00
GetAsync	pr	10000000	1,137,477.7 ns	0.43	9768.88 KB	0.30
GetByteArrayAsync	main	100000000	21,216,153.0 ns	1.00	97657.44 KB	1.00
GetByteArrayAsync	pr	100000000	17,026,034.7 ns	0.80	97657.69 KB	1.00
GetAsync	main	100000000	25,177,538.4 ns	1.00	260446.05 KB	1.00
GetAsync	pr	100000000	16,487,114.6 ns	0.66	97657.69 KB	0.37

[stephentoub]

so we start with a larger buffer (currently 256 KB)

If we hit bursty scenarios where the pool is exhausted, we end up allocating buffers, and then there's not enough room in the pool so we end up dropping buffers, won't using such a large buffer be particularly problematic because it's above the LOH threshold and will always be LOH / gen2?

[CarnaViire]

Would it be possible to leverage MutliArrayBuffer, which already has a collection of pooled buffers -- or will that be terribly inefficient?

Or the other way around: could something from here improve the MutliArrayBuffer implementation?

It just feels a little bit like reimplementing a very similar idea, but maybe I'm missing something.

[MihaZupan]

It's a good question and to a degree we are doing very similar things.
Using MultiArrayBuffer as-is would indeed be inefficient when it comes to larger responses because it uses only one fixed size for buffers (16 KB). A large content could quickly negate any pooling benefits by starving out the array pool for that bucket size.

That shouldn't be as much of an issue with our current uses of MultiArrayBuffer because those are generally meant to be short-lived and relatively small (e.g. hand off point between the connection's read loop and the request's calls to ReadAsync).
Side note: if we wanted to get really fancy, we could try to avoid that extra memory copy and instead rent new buffers for connection's reads, but that's something for the (far) future.

I think it's more likely we would go the other way as you mention and tweak the MultiArrayBuffer implementation instead. Its use of a single buffer size seems quite ingrained in most of the implementation, but looking more at our current usages of it, it shouldn't be necessary (for example, we don't care about faster indexing into the buffer that that allows for). I'd be more inclined to replace MultiArrayBuffer entirely with something closer to the buffering logic here. It might be easier to reason about the change as a follow-up PR though (at risk of having two somewhat duplicatey concepts in the repo for now).
The implementation I'm adding in HttpContent does have some odd specifics around its buffer growth logic, and optional switching from pooled to non-pooled buffers. That part probably isn't really transferable to other uses, but I think we can make a shared implementation work without regressing those.

[liveans]

Couple of nits, feel free to ignore, LGTM.

[liveans]

nit: redundant else