UDP protocols need congestion control

[leoluk]

Problem

Solana's UDP protocols do not detect path congestion and keep sending traffic even on highly congested paths, exacerbating the problem. Well-behaved protocols use a congestion control algorithm like New Reno or BBR v2, which use a probing mechanism to detect loss and throttle sending: https://www.ietf.org/proceedings/104/slides/slides-104-iccrg-an-update-on-bbr-00

Path congestion is detrimental both to other flows on the network as well as Solana itself, since many networks have choke points that cannot do fair queuing on their own, and will cause buffer bloat when congested.

The internet is built on the implicit contract of "TCP-friendly flow control", and network operators routinely de-prioritize flows that do not adhere to that contract.

See https://tools.ietf.org/html/rfc2914 and http://www.icir.org/floyd/papers/collapse.may99.pdf on why flow control matters to prevent congestive collapse of a shared resource.

Proposed Solution

Use BBR v2 or a similar modern congestion control mechanism to detect loss and either stop sending traffic, or reduce the send rate wherever that makes sense.