More Tests for WebRTC Data Channels #10468

lgrahl · 2018-04-13T23:54:19Z

Description

This PR updates all data channel related tests to reflect the current spec (at the time of writing, that was revision 1cc5bfc3ff18741033d804c4a71f7891242fb5b3).

Updated/Fixed Tests

RTCDataChannel-id
- DTLS client uses odd data channel IDs
- DTLS server uses even data channel IDs
RTCDataChannel-send-receive (renamed from RTCDataChannel-send)
- (<type> is of In-band negotiated channel, Negotiated channel or Asymmetric negotiated channel)
- <type>: Calling send() when not open should throw InvalidStateError (renamed from Calling send() when data channel is in connecting state should throw InvalidStateError)
- <type>: Calling send() with null should throw TypeError
- <type>: Calling send() with undefined should throw TypeError
- <type>: Should be able to send (local) simple string and receive (remote) as string (based on Data channel should be able to send simple string and receive as string)
- <type>: Should be able to send (remote) simple string and receive (local) as string (based on Data channel should be able to send simple string and receive as string)
- <type>: Should be able to send unicode string and receive as unicode string (renamed from Data channel should be able to send unicode string and receive as unicode string)
- <type>: Should ignore binaryType and always receive string message as string (renamed from Data channel should ignore binaryType and always receive string message as string)
- <type>: Should be able to send Uint8Array message and receive as ArrayBuffer (renamed from Data channel should be able to send Uint8Array message and receive as ArrayBuffer)
- <type>: Should be able to send ArrayBuffer message and receive as ArrayBuffer (renamed from Data channel should be able to send ArrayBuffer message and receive as ArrayBuffer)
- <type>: Should be able to send Blob message and receive as ArrayBuffer (renamed from Data channel should be able to send Blob message and receive as ArrayBuffer)
- <type>: Should be able to send ArrayBuffer message and receive as Blob (renamed from Data channel should be able to send ArrayBuffer message and receive as Blob)
- <type>: Should be able to transmit an empty string (moved from and based on datachannel-emptystring Can send empty strings across a WebRTC data channel.)
- <type>: Should receive messages as Blob by default (renamed from Data channel binaryType should receive message as Blob by default)
- <type>: Sending multiple messages with different types should succeed (based on Sending multiple messages with different types should succeed and be received)
- <type>: Receiving multiple messages with different types should succeed (based on Sending multiple messages with different types should succeed and be received)
RTCPeerConnection-createDataChannel
- createDataChannel attribute default values
- createDataChannel with provided parameters should initialize attributes to provided values
- createDataChannel with both maxPacketLifeTime and maxRetransmits should throw TypeError (renamed from createDataChannel with both maxPacketLifeTime and maxRetransmits should throw SyntaxError)
- createDataChannel with too long label should throw TypeError (renamed from createDataChannel with negotiated false and long label should throw TypeError)
- createDataChannel with too long protocol should throw TypeError (renamed from createDataChannel with negotiated false and long protocol should throw TypeError)
- Channels created (after setRemoteDescription) should have id assigned (renamed from Channels created after SCTP transport is established should have id assigned)
RTCPeerConnection-ondatachannel
- Data channel event should fire when new data channel is announced to the remote peer (renamed from datachannel event should fire when new data channel is announced to the remote peer)
- In-band negotiated channel created on remote peer should match the same configuration as local peer (renamed from Data channel created on remote peer should match the same configuration as local peer)
RTCSctpTransport-constructor
- setRemoteDescription() with answer containing data media should initialize pc.sctp
- setLocalDescription() with answer containing data media should initialize pc.sctp
RTCSctpTransport-maxMessageSize
- Remote offer SDP missing max-message-size attribute
- max-message-size with a (non-zero) value provided by the remote peer
- Renegotiate max-message-size with various values provided by the remote peer (based on Renegotiate max-message-size with a (non-zero) value provided by the remote peer)
- max-message-size with a (non-zero) value larger than canSendSize provided by the remote peer

New Tests

RTCDataChannel-binaryType
- Setting binaryType to 'blob' should succeed
- Setting binaryType to 'arraybuffer' should succeed
- Setting invalid binaryType 'jellyfish' should throw SyntaxError
- Setting invalid binaryType 'arraybuffer ' should throw SyntaxError
RTCDataChannel-bufferedAmount
- bufferedAmount initial value should be 0 for both peers
- bufferedAmount should not decrease immediately after initiating closure
- bufferedAmount should not decrease after closing the peer connection
RTCDataChannel-bufferedAmountLowThreshold
- (<type> is of DOMString or Uint8Array)
- <type> of 4 bytes: bufferedamountlow event should fire once bufferedAmount <= 0 (default)
- <type> of 4 bytes: bufferedamountlow event should fire once bufferedAmount <= 2 (custom)
- <type> of 4 bytes: bufferedamountlow event should fire once bufferedAmount <= 4 (custom)
- <type> of 4 bytes: bufferedamountlow event should fire once bufferedAmount <= 8 (custom)
- <type> of 4 bytes x 10: bufferedamountlow event should fire once bufferedAmount <= 2 (custom)
- <type> of 4 bytes x 10: bufferedamountlow event should fire once bufferedAmount <= 4 (custom)
- <type> of 4 bytes x 10: bufferedamountlow event should fire once bufferedAmount <= 8 (custom)
- <type> of 4 bytes: bufferedamountlow event should not fire on other channels
RTCDataChannel-close
- In-band data channel closing before connection establishment
- In-band data channel closing after connection establishment
- In-band data channels x10 closing on local side should be synchronized across peers
- In-band data channels x10 closing on remote side should be synchronized across peers
- Negotiated data channels x10 closing should be synchronized across peers
- Closing peer connection locally should close all 10 channels for the remote peer
RTCDataChannel-dcep
- (<value> is of undefined, null, 0, 65535 or 4294967295)
- Ordered and reliable channel should be created via DCEP
- Unordered and reliable channel should be created via DCEP
- Ordered and unreliable (maxRetransmits=<value>) channel should be created via DCEP
- Unordered and unreliable (maxRetransmits=<value>) channel should be created via DCEP
- Ordered and unreliable (maxPacketLifeTime=<value>) channel should be created via DCEP
- Unordered and unreliable (maxPacketLifeTime=<value>) channel should be created via DCEP
- Channel with priority set to high should be created via DCEP
- Channel IDs should be synchronized when created via DCEP
- Zero length label and protocol option should be transmitted via DCEP
- Maximum length label and protocol option should be transmitted via DCEP
- Maximum length label and protocol option (3 byte unicode) should be transmitted via DCEP
RTCDataChannel-id
- In-band negotiation with a specific ID should not be allowed
- Odd/even role should not be violated when mixing with negotiated channels
- Creating a channel after exhausting the maximum number of channels should throw OperationError (after connection establishment)
- ID reuse should not be possible before the former channel with the same ID closed
- ID reuse should be possible once the former channel with the same ID closed
- Channel IDs should be reassigned after exhaustion in case a channel has been closed
- Exhausting channels with one peer should not violate the odd/even rule
- Channel ID exhaustion handling (before and after connection establishment)
RTCDataChannel-onopen
- In-band channel: Open event should be fired (local) when the data channel opens
- In-band channel: Open event should be fired (remote) when the data channel opens
- Negotiated channel: Open event should be fired when the channels open
RTCDataChannel-send-receive
- (<type> is of In-band negotiated channel, Negotiated channel or Asymmetric negotiated channel)
- (<size> is of 1024, 16384, 65536, 131072, 262144, 524288, 1048576, 16777216 or 33554432)
- <type>: Calling send() with 42 should throw TypeError
- <type>: Should be able to send Int8Array message and receive as ArrayBuffer
- <type>: Should be able to send Uint8ClampedArray message and receive as ArrayBuffer
- <type>: Should be able to send Int16Array message and receive as ArrayBuffer
- <type>: Should be able to send Uint16Array message and receive as ArrayBuffer
- <type>: Should be able to send Int32Array message and receive as ArrayBuffer
- <type>: Should be able to send Uint32Array message and receive as ArrayBuffer
- <type>: Should be able to send Float32Array message and receive as ArrayBuffer
- <type>: Should be able to send Float64Array message and receive as ArrayBuffer
- <type>: Should be able to send DataView message and receive as ArrayBuffer
- <type>: Should be able to send Uint8Array (with offset) message and receive as ArrayBuffer
- <type>: Should be able to send DataView (with offset) message and receive as ArrayBuffer
- <type>: Should be able to send Blob (with offset) message and receive as ArrayBuffer
- <type>: Should be able to transmit an empty Uint8Array
- <type>: Should be able to transmit an empty Blob
- <type>: Sending and receiving <size> bytes should succeed or raise TypeError
- <type>: Sending and receiving <size> +1 bytes should raise TypeError
- <type>: Sending and receiving 16 KiB x64 should succeed
- <type>: Sending and receiving 16 KiB x256 on both peer simultaneously should succeed
- <type>: Sending and receiving 2048 messages should succeed and keep order
- <type>: Closing a channel (local) with pending data should transfer that data before becoming closed
- <type>: Sending after the channel has been closed (local) should raise InvalidStateError
- <type>: Sending after the channel has been closed (remote) should raise InvalidStateError
- <type>: Sending after the peer connection has been closed (local) should raise InvalidStateError
- <type>: Sending after the peer connection has been closed (remote) should raise InvalidStateError
RTCDataChannel-send-receive-stress
- Stress-test with multiple channels sending and receiving using various data sources
RTCPeerConnection-close
- Closing the peer connection should close all channels (before connection establishment)
- Closing the local peer connection should close all channels (after connection establishment)
- Closing the remote peer connection should close all channels (after connection establishment)
- Closing the peer connection should set the SCTP transport to closed
RTCPeerConnection-createDataChannel
- createDataChannel with both maxPacketLifeTime and maxRetransmits null should succeed
- createDataChannel with too long label (2 byte unicode) should throw TypeError
- createDataChannel with same label used twice should not throw
- createDataChannel with too long protocol (2 byte unicode) should throw TypeError
- createDataChannel with maximum length label and protocol should succeed
- createDataChannel with negotiated false should succeed
- createDataChannel with negotiated false and id 42 should ignore the id
- createDataChannel with negotiated true and id not defined should throw TypeError
- createDataChannel with negotiated true and id null should throw TypeError
- Reusing a data channel id that is in use should throw OperationError
- Reusing a data channel id that is in use (after setRemoteDescription) should throw OperationError
- Reusing a data channel id that is in use (after setRemoteDescription, negotiated via DCEP) should throw OperationError
- New data channel should be in the connecting state after creation (after connection establishment)
RTCPeerConnection-ondatachannel
- Should be able to send data in a datachannel event handler
- Open event should not be raised when closing the channel in the datachannel event
- Open event should be raised when closing the channel in the datachannel event after enqueuing a task
- Open event should not be raised when sending and immediately closing the channel in the datachannel event
- In-band negotiated channel created on remote peer should match the same (default) configuration as local peer
RTCSctpTransport-constructor
- setRemoteDescription() with answer not containing data media should not initialize pc.sctp
- setLocalDescription() with answer not containing data media should not initialize pc.sct
RTCSctpTransport-onstatechange
- RTCSctpTransport.onstatechange should fire when the state changes
historical
- RTCDataChannel member maxRetransmitTime should not exist

Comments

Only few helper functions have been changed but everything is either backwards compatible or the affected tests have been updated. FWIW, I have carefully tested new test cases (if that was possible) on Firefox and Chrome by temporarily commenting out asserts and in some cases even by throwing in adapter (RTCSctpTransport.maxMessageSize for example).

Towards WebRTC implementations: A bunch of new issues are revealed by these tests. I'm at a tight time schedule, so I can't file them for you at the moment. This listing here should help you in finding new or updated test cases, so you can track and file new issues in your bug trackers. /cc @nils-ohlmeier @taylor-b

And I apologise for creating this rather large PR.

Update to current spec version Fix missing import to RTCPeerConnection-helper.js Test .bufferedAmount and .bufferedAmountLowThreshold Fix invalid use of 'label' in RTCDataChannelInit Add negotiated true and id null test Add reusing an id that is in use tests

description has been updated)

options are synchronized correctly across peers. Add optional 'channelLabel' and 'channelOptions' to `createDataChannelPair`

Once canSendSize has been determined, require it to be available in further tests that depend on it Fix canSendSize checks (explicitly check for the 0 case) Fix remoteMMS > canSendSize test for the 0 case Update descriptions to make them more precise

Add a test that tries to create the maximum number of data channels until no ids are free anymore after the connection has been established. Add a test that checks if channel IDs are being reassigned after exhaustion in case a channel has been closed. Add a test that tries to create the theoretical maximum number of data channels and ensure that violation of the odd/even rule does not occur. Add a test that tries to create the theoretical maximum number of data channels before the connection has been established (and check if some of them are properly teared down).

Add test for violation of odd/even rule with mixed channels (negotiated & DCEP-negotiated)

… is being closed

… have a strong reference to the RTCPeerConnection instance anyway

Move RTCPeerConnection.close tests into their own file

Update to current spec version Add assert for 'state' Fix the assert for 'maxMessageSize' A bit of cleanup

…fires with the correct state Add an `IceCandidateQueue` helper class which allows you hold back ICE candidates conveniently until `exchangeIceCandidates` is called

lgrahl · 2018-05-19T00:27:12Z

Since I've started adding tests based on new PRs in https://github.com/w3c/webrtc-pc, I'll need to bump the spec revision. Will do that once all of them have been merged.

Add explicit *negotiated false* and *negotiated true and id not defined* test cases

Move an existing test case from RTCDataChannel-send-receive

…een merged

…annel message

…afari

…rk in Safari

…e it work in Safari

…nnel

alvestrand

This is a pretty massive CL, but seems straightforward (once you accept the "await test.done" paradigm).
I'd like to have it merged, but would like to see approval from someone who feels comfortable approving changes to testharness.js.

alvestrand · 2018-06-25T07:07:05Z

webrtc/RTCDataChannel-bufferedAmountLowThreshold.html

+
+// Disable global timeout
+// IMPORTANT: You need to add a timeout to *every test* of this file!
+setup({ explicit_timeout: true });


Why is the default timeout inappropriate here?
I see that a 5 second timeout is used, while the default timeout is 10 seconds. Saving 5 seconds in the failure case seems hardly worth doing.

The problem is that there's a varying amount of test cases onto which one constant global timeout is applied, so it's 10 or 60 seconds globally for all test cases no matter how many you have. This workaround allows us to apply a local timeout to each test case.

alvestrand · 2018-06-25T07:10:44Z

webrtc/RTCDataChannel-send-receive-stress.html

+
+  await t.done_promise;
+}, 'Stress-test with multiple channels sending and receiving using various data sources', {
+  timeout: 420000 // yep, 7m to be safe


I would suggest leaving the long-duration stress test in its own file, to minimize the use of explicit timeouts. It's the only place in the CL with a timeout that is longer than the default.

lgrahl · 2018-06-28T08:41:54Z

Thanks Harald. The await test.done_promise (#10933) is not being cheered on, so I'll likely replace it with the suggestion made in #10933 (comment). But I'll ping them in the IRC again regarding the issue before doing so.

…ent test

lgrahl · 2018-10-13T09:22:57Z

Closing because of messy flags.

lgrahl · 2018-10-13T11:31:50Z

-> #13499

Mar181-arc

frontend | git fast-import []

wpt-pr-bot added webrtc wg-webrtc labels Apr 13, 2018

wpt-pr-bot requested review from agouaillard, alvestrand, dontcallmedom, guidou, henbos, phoglund, rwaldron, snuggs and youennf April 13, 2018 23:54

lgrahl mentioned this pull request Apr 14, 2018

Conformance Test Mapping to WPT nplab/WebRTC-Data-Channel-Playground#23

Open

lgrahl force-pushed the webrtc-data-channel branch from f34c94b to c0217c6 Compare April 14, 2018 00:11

lgrahl mentioned this pull request May 3, 2018

In-band negotiated channel should be open when dispatching w3c/webrtc-pc#1851

Merged

lgrahl added 12 commits May 8, 2018 17:00

Update RTCPeerConnection-id to current spec version (only the

016ab0d

description has been updated)

Add RTCPeerConnection-dcep tests which test whether data channel

9fc3ada

options are synchronized correctly across peers. Add optional 'channelLabel' and 'channelOptions' to `createDataChannelPair`

Update the renegotiation test in RTCSctpTransport-maxMessageSize

a0c5ad9

Fix RTCSctpTransport-id tests to always check for reused IDs

d70a2e3

Add test for violation of odd/even rule with mixed channels (negotiated & DCEP-negotiated)

Add tests for data channel closing procedure when the peer connection…

8993fa1

… is being closed

Fix do not attempt to keep data channels referenced since they should…

63ecbc0

… have a strong reference to the RTCPeerConnection instance anyway

Fix title of DCEP tests

3318326

Move RTCPeerConnection.close tests into their own file

Update RTCSctpTransport-constructor tests

0874c02

Update to current spec version Add assert for 'state' Fix the assert for 'maxMessageSize' A bit of cleanup

Add RTCSctpTransport-onstatechange test which ensures that the event …

5da6bb8

…fires with the correct state Add an `IceCandidateQueue` helper class which allows you hold back ICE candidates conveniently until `exchangeIceCandidates` is called

wpt-pr-bot added the docs label May 9, 2018

wpt-pr-bot requested a review from sideshowbarker May 9, 2018 16:01

lgrahl mentioned this pull request May 9, 2018

Expose done promise on Test #10933

Closed

Add a test based on discussion in w3c/webrtc-pc#1851 with @jan-ivar

0322f4a

lgrahl added 14 commits May 19, 2018 20:04

Remove invalid *negotiated true and no id* test case

bfaebae

Add explicit *negotiated false* and *negotiated true and id not defined* test cases

Add RTCDataChannel-binaryType test file

f661db6

Move an existing test case from RTCDataChannel-send-receive

Remove a test case that is invalid now since w3c/webrtc-pc#1849 has b…

be2c249

…een merged

Add the number 42 to the list of invalid types when sending a data ch…

9424f2c

…annel message

Fix missing local timeout

ce857ba

fixup! Fix missing local timeout

d596bb8

Add an ugly workaround for createDataChannelPair to make it work in S…

cc9318b

…afari

Increase timeout for channel exhaustion tests

a33fdef

fixup! Add an ugly workaround for createDataChannelPair to make it wo…

efc98cc

…rk in Safari

fixup! fixup! Add an ugly workaround for createDataChannelPair to mak…

a99112e

…e it work in Safari

Fix assert_unreached description

9d9a057

Fix missing attribute check and a typo in RTCPeerConnection-ondatacha…

31fa169

…nnel

Fix a typo

6b64db5

Fix test description

6334a5c

alvestrand reviewed Jun 25, 2018

View reviewed changes

Make sure to skip at least a task in the datachannel then open ev…

f6b2be7

…ent test

wpt-pr-bot added the testharness.js label Aug 4, 2018

Fix bufferedamountlow event test

1614d70

lgrahl mentioned this pull request Aug 27, 2018

Added error detail "datachannel-id-generation-error", using in error … w3c/webrtc-pc#1971

Merged

lgrahl mentioned this pull request Oct 4, 2018

Add [EnforceRange]] to RTCDataChannel writable attributes. w3c/webrtc-pc#1999

Merged

lgrahl closed this Oct 13, 2018

Mar181-arc reviewed Dec 5, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

More Tests for WebRTC Data Channels #10468

More Tests for WebRTC Data Channels #10468

lgrahl commented Apr 13, 2018 •

edited

Loading

lgrahl commented May 19, 2018

alvestrand left a comment

alvestrand Jun 25, 2018

lgrahl Jun 28, 2018

alvestrand Jun 25, 2018

lgrahl commented Jun 28, 2018

lgrahl commented Oct 13, 2018

lgrahl commented Oct 13, 2018

Mar181-arc left a comment

More Tests for WebRTC Data Channels #10468

More Tests for WebRTC Data Channels #10468

Conversation

lgrahl commented Apr 13, 2018 • edited Loading

Description

Updated/Fixed Tests

New Tests

Comments

lgrahl commented May 19, 2018

alvestrand left a comment

Choose a reason for hiding this comment

alvestrand Jun 25, 2018

Choose a reason for hiding this comment

lgrahl Jun 28, 2018

Choose a reason for hiding this comment

alvestrand Jun 25, 2018

Choose a reason for hiding this comment

lgrahl commented Jun 28, 2018

lgrahl commented Oct 13, 2018

lgrahl commented Oct 13, 2018

Mar181-arc left a comment

Choose a reason for hiding this comment

lgrahl commented Apr 13, 2018 •

edited

Loading