From 2a2c2f3d5151406e25e978e90c530c7d957f3b4d Mon Sep 17 00:00:00 2001 From: Alex Potsides Date: Wed, 27 Sep 2023 09:32:16 -0400 Subject: [PATCH 1/7] webrtc: add FIN_ACK to spec to close datachannels without data loss Specify closing datachannels by mutual agreement to ensure all data has been received by the remote before closing. Refs: #575 --- webrtc/README.md | 52 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 50 insertions(+), 2 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index 50011237e..a2fea5743 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -67,6 +67,10 @@ message Message { // The sender abruptly terminates the sending part of the stream. The // receiver MAY discard any data that it already received on that stream. RESET_STREAM = 2; + // The receiver has previously sent a message with a FIN flag. On receiving + // this flag they may now close the stream as they know the remote has received + // all data that was sent. + FIN_ACK = 3; } optional Flag flag=1; @@ -132,8 +136,8 @@ real-world experiments. `RTCDataChannel`s are negotiated in-band by the WebRTC user agent (e.g. Firefox, Pion, ...). In other words libp2p WebRTC implementations MUST NOT change the -default value `negotiated: false` when creating a standard libp2p stream -of type `RTCDataChannel` via `RTCPeerConnection.createDataChannel`. +default value `negotiated: false` when creating a standard libp2p stream +of type `RTCDataChannel` via `RTCPeerConnection.createDataChannel`. Setting `negotiated: true` is reserved only for creating Noise handshake channels under certain protocol conditions. @@ -161,6 +165,50 @@ MUST pass an empty string. When receiving an `RTCDataChannel` via an empty string. This allows future versions of this specification to make use of the `RTCDataChannel` `label` property. +## Closing an `RTCDataChannel` + +Some WebRTC implementations do not guarantee that any queued messages will be +sent after a datachannel is closed. Other implementatations maintain separate +outgoing message and transport queues, the status of which may not be visible +to the user. Consequently we must add an additional layer of signaling to ensure +reliable data delivery. + +When a node wishes to close a stream for writing, it should send a message with +the `FIN` flag set, then proceed to wait for a `FIN_ACK` message from the remote +node. + +If a `FIN` flag is received and the node has finished writing data, it should +respond with a `FIN_ACK` immediately. + +If a `FIN` flag is recieved but the node has not finished writing data, it should +delay sending a `FIN_ACK` response until it has finished writing data, sent a +`FIN` flag, and finally received a `FIN_ACK` in response. + +When a `FIN_ACK` has been sent and received, a node may close the datachannel. + +The node may close the datachannel without receiving a `FIN_ACK`, for example in +the case of a timeout, but there will be no guarantee that all previously sent +messages have been received. + +If a node has previously sent a `STOP_SENDING` flag to the remote node, it must +continue to act on any flags present in received messages in order to +successfully process an incoming `FIN_ACK`. + +### Example of closing an `RTCDataChannel` + +NodeA closes for writing, NodeB delays allowing the channel to close until it also +finishes writing. + +```mermaid +sequenceDiagram + NodeA->>NodeB: DATA + NodeA->>NodeB: FIN + NodeB->>NodeA: DATA + NodeB->>NodeA: FIN + NodeA->>NodeB: FIN_ACK + NodeB->>NodeA: FIN_ACK +``` + ## Previous, ongoing and related work - Completed implementations of this specification: From 1cebbb689c9231ab4f779c923afc4c18e3e90cb3 Mon Sep 17 00:00:00 2001 From: Alex Potsides Date: Wed, 27 Sep 2023 15:09:27 -0400 Subject: [PATCH 2/7] chore: apply suggestions from code review Co-authored-by: Max Inden --- webrtc/README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index a2fea5743..319b67379 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -68,8 +68,8 @@ message Message { // receiver MAY discard any data that it already received on that stream. RESET_STREAM = 2; // The receiver has previously sent a message with a FIN flag. On receiving - // this flag they may now close the stream as they know the remote has received - // all data that was sent. + // the FIN_ACK flag they may now close the stream as they know the remote has received + // all data that was sent before the FIN. FIN_ACK = 3; } @@ -173,24 +173,24 @@ outgoing message and transport queues, the status of which may not be visible to the user. Consequently we must add an additional layer of signaling to ensure reliable data delivery. -When a node wishes to close a stream for writing, it should send a message with +When a node wishes to close a stream for writing, it SHOULD send a message with the `FIN` flag set, then proceed to wait for a `FIN_ACK` message from the remote node. -If a `FIN` flag is received and the node has finished writing data, it should +If a `FIN` flag is received and the node has finished writing data, it SHOULD respond with a `FIN_ACK` immediately. -If a `FIN` flag is recieved but the node has not finished writing data, it should +If a `FIN` flag is received but the node has not finished writing data, it SHOULD delay sending a `FIN_ACK` response until it has finished writing data, sent a `FIN` flag, and finally received a `FIN_ACK` in response. When a `FIN_ACK` has been sent and received, a node may close the datachannel. -The node may close the datachannel without receiving a `FIN_ACK`, for example in +The node MAY close the datachannel without receiving a `FIN_ACK`, for example in the case of a timeout, but there will be no guarantee that all previously sent messages have been received. -If a node has previously sent a `STOP_SENDING` flag to the remote node, it must +If a node has previously sent a `STOP_SENDING` flag to the remote node, it MUST continue to act on any flags present in received messages in order to successfully process an incoming `FIN_ACK`. From dec6a033c9490487ea52057e19950f7cb1e9da5e Mon Sep 17 00:00:00 2001 From: achingbrain Date: Tue, 3 Oct 2023 11:18:08 +0100 Subject: [PATCH 3/7] docs: remove delay --- webrtc/README.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index 319b67379..e9e3c3ae6 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -67,9 +67,9 @@ message Message { // The sender abruptly terminates the sending part of the stream. The // receiver MAY discard any data that it already received on that stream. RESET_STREAM = 2; - // The receiver has previously sent a message with a FIN flag. On receiving - // the FIN_ACK flag they may now close the stream as they know the remote has received - // all data that was sent before the FIN. + // Sending the FIN_ACK flag acknowledges the previous receipt of a message + // with the FIN flag set. Receiving a FIN_ACK flag gives the recipient + // confidence that the remote has received all sent messages. FIN_ACK = 3; } @@ -168,7 +168,7 @@ of the `RTCDataChannel` `label` property. ## Closing an `RTCDataChannel` Some WebRTC implementations do not guarantee that any queued messages will be -sent after a datachannel is closed. Other implementatations maintain separate +sent after a datachannel is closed. Other implementations maintain separate outgoing message and transport queues, the status of which may not be visible to the user. Consequently we must add an additional layer of signaling to ensure reliable data delivery. @@ -177,14 +177,9 @@ When a node wishes to close a stream for writing, it SHOULD send a message with the `FIN` flag set, then proceed to wait for a `FIN_ACK` message from the remote node. -If a `FIN` flag is received and the node has finished writing data, it SHOULD -respond with a `FIN_ACK` immediately. +If a `FIN` flag is received the node SHOULD respond with a `FIN_ACK`. -If a `FIN` flag is received but the node has not finished writing data, it SHOULD -delay sending a `FIN_ACK` response until it has finished writing data, sent a -`FIN` flag, and finally received a `FIN_ACK` in response. - -When a `FIN_ACK` has been sent and received, a node may close the datachannel. +When a `FIN_ACK` and a `FIN` have been received, a node may close the datachannel. The node MAY close the datachannel without receiving a `FIN_ACK`, for example in the case of a timeout, but there will be no guarantee that all previously sent @@ -203,12 +198,18 @@ finishes writing. sequenceDiagram NodeA->>NodeB: DATA NodeA->>NodeB: FIN + NodeB->>NodeA: FIN_ACK NodeB->>NodeA: DATA NodeB->>NodeA: FIN NodeA->>NodeB: FIN_ACK - NodeB->>NodeA: FIN_ACK ``` +After NodeA has received the `FIN` it is free to close the datachannel since it +has previously received a `FIN_ACK`. If NodeB receives the `FIN_ACK` before this +it may close the channel since it previously received a `FIN`. + +This way the channel can be closed from either end without data loss. + ## Previous, ongoing and related work - Completed implementations of this specification: From 0183659a426c41783472952b352c1e83b2cdf0e5 Mon Sep 17 00:00:00 2001 From: Alex Potsides Date: Wed, 4 Oct 2023 13:58:57 +0100 Subject: [PATCH 4/7] Apply suggestions from code review Co-authored-by: Thomas Eizinger --- webrtc/README.md | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index e9e3c3ae6..91ad98714 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -174,8 +174,7 @@ to the user. Consequently we must add an additional layer of signaling to ensure reliable data delivery. When a node wishes to close a stream for writing, it SHOULD send a message with -the `FIN` flag set, then proceed to wait for a `FIN_ACK` message from the remote -node. +the `FIN` flag set. A node SHOULD only consider its write-half closed once it received a `FIN_ACK`. If a `FIN` flag is received the node SHOULD respond with a `FIN_ACK`. @@ -183,7 +182,7 @@ When a `FIN_ACK` and a `FIN` have been received, a node may close the datachanne The node MAY close the datachannel without receiving a `FIN_ACK`, for example in the case of a timeout, but there will be no guarantee that all previously sent -messages have been received. +messages have been received by the remote. If a node has previously sent a `STOP_SENDING` flag to the remote node, it MUST continue to act on any flags present in received messages in order to @@ -196,16 +195,16 @@ finishes writing. ```mermaid sequenceDiagram - NodeA->>NodeB: DATA - NodeA->>NodeB: FIN - NodeB->>NodeA: FIN_ACK - NodeB->>NodeA: DATA - NodeB->>NodeA: FIN - NodeA->>NodeB: FIN_ACK + A->>B: DATA + A->>B: FIN + B->>A: FIN_ACK + B->>A: DATA + B->>A: FIN + A->>B: FIN_ACK ``` -After NodeA has received the `FIN` it is free to close the datachannel since it -has previously received a `FIN_ACK`. If NodeB receives the `FIN_ACK` before this +After _A_ has received the `FIN` it is free to close the datachannel since it +has previously received a `FIN_ACK`. If _B_ receives the `FIN_ACK` before this it may close the channel since it previously received a `FIN`. This way the channel can be closed from either end without data loss. From 6114dd7a2d7a76f2d91c8e2168e12875e6f13914 Mon Sep 17 00:00:00 2001 From: achingbrain Date: Wed, 4 Oct 2023 14:01:50 +0100 Subject: [PATCH 5/7] docs: update order --- webrtc/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/webrtc/README.md b/webrtc/README.md index 91ad98714..9b0e574c0 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -174,10 +174,12 @@ to the user. Consequently we must add an additional layer of signaling to ensure reliable data delivery. When a node wishes to close a stream for writing, it SHOULD send a message with -the `FIN` flag set. A node SHOULD only consider its write-half closed once it received a `FIN_ACK`. +the `FIN` flag set. If a `FIN` flag is received the node SHOULD respond with a `FIN_ACK`. +A node SHOULD only consider its write-half closed once it received a `FIN_ACK`. + When a `FIN_ACK` and a `FIN` have been received, a node may close the datachannel. The node MAY close the datachannel without receiving a `FIN_ACK`, for example in From 7da0fc8d8839b7ab8961c508c8f9886be2796406 Mon Sep 17 00:00:00 2001 From: achingbrain Date: Wed, 4 Oct 2023 14:03:45 +0100 Subject: [PATCH 6/7] docs: grammar edits --- webrtc/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index 9b0e574c0..43b564336 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -173,14 +173,14 @@ outgoing message and transport queues, the status of which may not be visible to the user. Consequently we must add an additional layer of signaling to ensure reliable data delivery. -When a node wishes to close a stream for writing, it SHOULD send a message with +When a node wishes to close a stream for writing, it MUST send a message with the `FIN` flag set. If a `FIN` flag is received the node SHOULD respond with a `FIN_ACK`. -A node SHOULD only consider its write-half closed once it received a `FIN_ACK`. +A node SHOULD only consider its write-half closed once it has received a `FIN_ACK`. -When a `FIN_ACK` and a `FIN` have been received, a node may close the datachannel. +When a `FIN_ACK` and a `FIN` have been received, the node may close the datachannel. The node MAY close the datachannel without receiving a `FIN_ACK`, for example in the case of a timeout, but there will be no guarantee that all previously sent From 0584c986fbb92cde1f6f37006a3f485655889ff3 Mon Sep 17 00:00:00 2001 From: achingbrain Date: Wed, 4 Oct 2023 14:06:16 +0100 Subject: [PATCH 7/7] chore: formatting --- webrtc/README.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index 43b564336..c482dfccc 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -170,17 +170,19 @@ of the `RTCDataChannel` `label` property. Some WebRTC implementations do not guarantee that any queued messages will be sent after a datachannel is closed. Other implementations maintain separate outgoing message and transport queues, the status of which may not be visible -to the user. Consequently we must add an additional layer of signaling to ensure -reliable data delivery. +to the user. Consequently we must add an additional layer of signaling to +ensure reliable data delivery. When a node wishes to close a stream for writing, it MUST send a message with the `FIN` flag set. If a `FIN` flag is received the node SHOULD respond with a `FIN_ACK`. -A node SHOULD only consider its write-half closed once it has received a `FIN_ACK`. +A node SHOULD only consider its write-half closed once it has received a +`FIN_ACK`. -When a `FIN_ACK` and a `FIN` have been received, the node may close the datachannel. +When a `FIN_ACK` and a `FIN` have been received, the node may close the +datachannel. The node MAY close the datachannel without receiving a `FIN_ACK`, for example in the case of a timeout, but there will be no guarantee that all previously sent @@ -192,8 +194,8 @@ successfully process an incoming `FIN_ACK`. ### Example of closing an `RTCDataChannel` -NodeA closes for writing, NodeB delays allowing the channel to close until it also -finishes writing. +NodeA closes for writing, NodeB delays allowing the channel to close until it +also finishes writing. ```mermaid sequenceDiagram