From ae7e26a1dc6f16beb7191adfe57effa6ce1b39f9 Mon Sep 17 00:00:00 2001 From: achingbrain Date: Tue, 5 Mar 2024 15:38:16 +0100 Subject: [PATCH] webtransport: add message framing to allow graceful stream closing When closing streams some implementations [do not guarantee all data has been sent](https://issues.chromium.org/issues/326887753) before closing the stream. If this sounds familiar it's because we observed [exactly the same behaviour with WebRTC](https://issues.chromium.org/issues/40072842#comment5). This PR adds opt-in message framing to WebTransport streams that lets us introduce a similar `FIN`/`FIN_ACK` mechanism that guarentees the remote has received all data sent on a stream before we close it. It uses a Noise extension to signal to the remote that we will be framing all outgoing messages, with a recommendation that the framing is omitted if the remote does not signal the same in response. This is an attempt to make this a backwards-compatible change which will be a lot less disruptive, even if the status quo is a lot more unsafe. We're still waiting a definitive answer from the Chromium team as to whether the data loss on closing is a bug or a feature but I thought I'd open this early so we can move quickly if they confirm it's an oversight. --- noise/README.md | 10 +++++++++- webtransport/README.md | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/noise/README.md b/noise/README.md index 6277f97a0..da8ea47f4 100644 --- a/noise/README.md +++ b/noise/README.md @@ -39,7 +39,6 @@ and spec status. - - [Overview](#overview) - [Negotiation](#negotiation) - [The Noise Handshake](#the-noise-handshake) @@ -48,6 +47,7 @@ and spec status. - [The libp2p Handshake Payload](#the-libp2p-handshake-payload) - [Handshake Pattern](#handshake-pattern) - [XX](#xx) + - [Noise Extensions](#noise-extensions) - [Cryptographic Primitives](#cryptographic-primitives) - [Noise Protocol Name](#noise-protocol-name) - [Wire Format](#wire-format) @@ -61,6 +61,9 @@ and spec status. - [Changelog](#changelog) - [r1 - 2020-01-20](#r1---2020-01-20) - [r2 - 2020-03-30](#r2---2020-03-30) + - [r3 - 2022-09-20](#r3---2022-09-20) + - [r4 - 2022-09-22](#r4---2022-09-22) + - [r5 - 2024-03-05](#r5---2024-03-05) @@ -221,6 +224,7 @@ syntax = "proto2"; message NoiseExtensions { repeated bytes webtransport_certhashes = 1; repeated string stream_muxers = 2; + boolean webtransport_message_framing = 3; } message NoiseHandshakePayload { @@ -461,6 +465,10 @@ unsupported types like RSA. - Add Noise extension registry +### r5 - 2024-03-05 + +- Add WebTransport message framing to extension registry + [peer-id-spec]: ../peer-ids/peer-ids.md [peer-id-spec-signing-rules]: ../peer-ids/peer-ids.md#how-keys-are-encoded-and-messages-signed diff --git a/webtransport/README.md b/webtransport/README.md index d20916a63..a430b269b 100644 --- a/webtransport/README.md +++ b/webtransport/README.md @@ -69,3 +69,39 @@ In order to verify end-to-end encryption of the connection, the peers need to es On receipt of the `webtransport_certhashes` extension, the client MUST verify that the certificate hash of the certificate that was used on the connection is contained in the server's list. If the client was willing to accept multiple certificate hashes, but cannot determine which certificate was actually used to establish the connection (this will commonly be the case for browser clients), it MUST verify that all certificate hashes are contained in the server's list. If verification fails, it MUST abort the handshake. For the client, the libp2p connection is fully established once it has sent the last Noise handshake message. For the server, processing of that message completes the handshake. + +## Message framing + +Upon closing a WebTransport stream, some implementations do not wait for all outstanding stream data to be sent over the wire before freeing up stream resources and making queue memory available for other uses. + +To allow cleanly closing streams without the loss of data using these implementations it's necessary to adopt similar semantics to the WebRTC transport when [closing datachannels](../webrtc/README.md#closing-an-rtcdatachannel). + +During the [security handshake](#security-handshake), a `webtransport_message_framing` boolean value may be supplied as part of the Noise extension block. + +If present, all messages sent over this connection will framed within a protobuf wrapper with the following structure: + +```proto +syntax = "proto3"; + +message Message { + enum Flag { + // The sender will no longer send messages on the stream. + FIN = 0; + // The sender acknowledges receipt of a FIN + FIN_ACK = 1; + } + + optional Flag flag = 1; + optional bytes message = 2; +} +``` + +If a node sends the `webtransport_message_framing` flag but does not receive one from the remote, it MUST fall back to sending unframed messages. + +If both nodes send the `webtransport_message_framing` flag, when either node wishes to close a stream for writing, it MUST send a message with the `FIN` flag set. + +If a `FIN` flag is received the node SHOULD respond with a `FIN_ACK`. + +A node SHOULD only close it's writable end of the stream once it has received a `FIN_ACK`. + +The node MAY close the writable end of the stream without receiving a `FIN_ACK`, for example in the case of a timeout, but there will be no guarantee that all previously sent messages have been received by the remote.