More Tests for WebRTC Data Channels

[lgrahl]

Description

This PR updates all data channel related tests to reflect the current spec (at the time of writing, that was revision 1cc5bfc3ff18741033d804c4a71f7891242fb5b3).

Updated/Fixed Tests

• RTCDataChannel-id
  • DTLS client uses odd data channel IDs
  • DTLS server uses even data channel IDs
• RTCDataChannel-send-receive (renamed from RTCDataChannel-send)
  • (<type> is of In-band negotiated channel, Negotiated channel or Asymmetric negotiated channel)
  • <type>: Calling send() when not open should throw InvalidStateError (renamed from Calling send() when data channel is in connecting state should throw InvalidStateError)
  • <type>: Calling send() with null should throw TypeError
  • <type>: Calling send() with undefined should throw TypeError
  • <type>: Should be able to send (local) simple string and receive (remote) as string (based on Data channel should be able to send simple string and receive as string)
  • <type>: Should be able to send (remote) simple string and receive (local) as string (based on Data channel should be able to send simple string and receive as string)
  • <type>: Should be able to send unicode string and receive as unicode string (renamed from Data channel should be able to send unicode string and receive as unicode string)
  • <type>: Should ignore binaryType and always receive string message as string (renamed from Data channel should ignore binaryType and always receive string message as string)
  • <type>: Should be able to send Uint8Array message and receive as ArrayBuffer (renamed from Data channel should be able to send Uint8Array message and receive as ArrayBuffer)
  • <type>: Should be able to send ArrayBuffer message and receive as ArrayBuffer (renamed from Data channel should be able to send ArrayBuffer message and receive as ArrayBuffer)
  • <type>: Should be able to send Blob message and receive as ArrayBuffer (renamed from Data channel should be able to send Blob message and receive as ArrayBuffer)
  • <type>: Should be able to send ArrayBuffer message and receive as Blob (renamed from Data channel should be able to send ArrayBuffer message and receive as Blob)
  • <type>: Should be able to transmit an empty string (moved from and based on datachannel-emptystring Can send empty strings across a WebRTC data channel.)
  • <type>: Should receive messages as Blob by default (renamed from Data channel binaryType should receive message as Blob by default)
  • <type>: Sending multiple messages with different types should succeed (based on Sending multiple messages with different types should succeed and be received)
  • <type>: Receiving multiple messages with different types should succeed (based on Sending multiple messages with different types should succeed and be received)
• RTCPeerConnection-createDataChannel
  • createDataChannel attribute default values
  • createDataChannel with provided parameters should initialize attributes to provided values
  • createDataChannel with both maxPacketLifeTime and maxRetransmits should throw TypeError (renamed from createDataChannel with both maxPacketLifeTime and maxRetransmits should throw SyntaxError)
  • createDataChannel with too long label should throw TypeError (renamed from createDataChannel with negotiated false and long label should throw TypeError)
  • createDataChannel with too long protocol should throw TypeError (renamed from createDataChannel with negotiated false and long protocol should throw TypeError)
  • Channels created (after setRemoteDescription) should have id assigned (renamed from Channels created after SCTP transport is established should have id assigned)
• RTCPeerConnection-ondatachannel
  • Data channel event should fire when new data channel is announced to the remote peer (renamed from datachannel event should fire when new data channel is announced to the remote peer)
  • In-band negotiated channel created on remote peer should match the same configuration as local peer (renamed from Data channel created on remote peer should match the same configuration as local peer)
• RTCSctpTransport-constructor
  • setRemoteDescription() with answer containing data media should initialize pc.sctp
  • setLocalDescription() with answer containing data media should initialize pc.sctp
• RTCSctpTransport-maxMessageSize
  • Remote offer SDP missing max-message-size attribute
  • max-message-size with a (non-zero) value provided by the remote peer
  • Renegotiate max-message-size with various values provided by the remote peer (based on Renegotiate max-message-size with a (non-zero) value provided by the remote peer)
  • max-message-size with a (non-zero) value larger than canSendSize provided by the remote peer

New Tests

• RTCDataChannel-binaryType
  • Setting binaryType to 'blob' should succeed
  • Setting binaryType to 'arraybuffer' should succeed
  • Setting invalid binaryType 'jellyfish' should throw SyntaxError
  • Setting invalid binaryType 'arraybuffer ' should throw SyntaxError
• RTCDataChannel-bufferedAmount
  • bufferedAmount initial value should be 0 for both peers
  • bufferedAmount should not decrease immediately after initiating closure
  • bufferedAmount should not decrease after closing the peer connection
• RTCDataChannel-bufferedAmountLowThreshold
  • (<type> is of DOMString or Uint8Array)
  • <type> of 4 bytes: bufferedamountlow event should fire once bufferedAmount <= 0 (default)
  • <type> of 4 bytes: bufferedamountlow event should fire once bufferedAmount <= 2 (custom)
  • <type> of 4 bytes: bufferedamountlow event should fire once bufferedAmount <= 4 (custom)
  • <type> of 4 bytes: bufferedamountlow event should fire once bufferedAmount <= 8 (custom)
  • <type> of 4 bytes x 10: bufferedamountlow event should fire once bufferedAmount <= 2 (custom)
  • <type> of 4 bytes x 10: bufferedamountlow event should fire once bufferedAmount <= 4 (custom)
  • <type> of 4 bytes x 10: bufferedamountlow event should fire once bufferedAmount <= 8 (custom)
  • <type> of 4 bytes: bufferedamountlow event should not fire on other channels
• RTCDataChannel-close
  • In-band data channel closing before connection establishment
  • In-band data channel closing after connection establishment
  • In-band data channels x10 closing on local side should be synchronized across peers
  • In-band data channels x10 closing on remote side should be synchronized across peers
  • Negotiated data channels x10 closing should be synchronized across peers
  • Closing peer connection locally should close all 10 channels for the remote peer
• RTCDataChannel-dcep
  • (<value> is of undefined, null, 0, 65535 or 4294967295)
  • Ordered and reliable channel should be created via DCEP
  • Unordered and reliable channel should be created via DCEP
  • Ordered and unreliable (maxRetransmits=<value>) channel should be created via DCEP
  • Unordered and unreliable (maxRetransmits=<value>) channel should be created via DCEP
  • Ordered and unreliable (maxPacketLifeTime=<value>) channel should be created via DCEP
  • Unordered and unreliable (maxPacketLifeTime=<value>) channel should be created via DCEP
  • Channel with priority set to high should be created via DCEP
  • Channel IDs should be synchronized when created via DCEP
  • Zero length label and protocol option should be transmitted via DCEP
  • Maximum length label and protocol option should be transmitted via DCEP
  • Maximum length label and protocol option (3 byte unicode) should be transmitted via DCEP
• RTCDataChannel-id
  • In-band negotiation with a specific ID should not be allowed
  • Odd/even role should not be violated when mixing with negotiated channels
  • Creating a channel after exhausting the maximum number of channels should throw OperationError (after connection establishment)
  • ID reuse should not be possible before the former channel with the same ID closed
  • ID reuse should be possible once the former channel with the same ID closed
  • Channel IDs should be reassigned after exhaustion in case a channel has been closed
  • Exhausting channels with one peer should not violate the odd/even rule
  • Channel ID exhaustion handling (before and after connection establishment)
• RTCDataChannel-onopen
  • In-band channel: Open event should be fired (local) when the data channel opens
  • In-band channel: Open event should be fired (remote) when the data channel opens
  • Negotiated channel: Open event should be fired when the channels open
• RTCDataChannel-send-receive
  • (<type> is of In-band negotiated channel, Negotiated channel or Asymmetric negotiated channel)
  • (<size> is of 1024, 16384, 65536, 131072, 262144, 524288, 1048576, 16777216 or 33554432)
  • <type>: Calling send() with 42 should throw TypeError
  • <type>: Should be able to send Int8Array message and receive as ArrayBuffer
  • <type>: Should be able to send Uint8ClampedArray message and receive as ArrayBuffer
  • <type>: Should be able to send Int16Array message and receive as ArrayBuffer
  • <type>: Should be able to send Uint16Array message and receive as ArrayBuffer
  • <type>: Should be able to send Int32Array message and receive as ArrayBuffer
  • <type>: Should be able to send Uint32Array message and receive as ArrayBuffer
  • <type>: Should be able to send Float32Array message and receive as ArrayBuffer
  • <type>: Should be able to send Float64Array message and receive as ArrayBuffer
  • <type>: Should be able to send DataView message and receive as ArrayBuffer
  • <type>: Should be able to send Uint8Array (with offset) message and receive as ArrayBuffer
  • <type>: Should be able to send DataView (with offset) message and receive as ArrayBuffer
  • <type>: Should be able to send Blob (with offset) message and receive as ArrayBuffer
  • <type>: Should be able to transmit an empty Uint8Array
  • <type>: Should be able to transmit an empty Blob
  • <type>: Sending and receiving <size> bytes should succeed or raise TypeError
  • <type>: Sending and receiving <size> +1 bytes should raise TypeError
  • <type>: Sending and receiving 16 KiB x64 should succeed
  • <type>: Sending and receiving 16 KiB x256 on both peer simultaneously should succeed
  • <type>: Sending and receiving 2048 messages should succeed and keep order
  • <type>: Closing a channel (local) with pending data should transfer that data before becoming closed
  • <type>: Sending after the channel has been closed (local) should raise InvalidStateError
  • <type>: Sending after the channel has been closed (remote) should raise InvalidStateError
  • <type>: Sending after the peer connection has been closed (local) should raise InvalidStateError
  • <type>: Sending after the peer connection has been closed (remote) should raise InvalidStateError
• RTCDataChannel-send-receive-stress
  • Stress-test with multiple channels sending and receiving using various data sources
• RTCPeerConnection-close
  • Closing the peer connection should close all channels (before connection establishment)
  • Closing the local peer connection should close all channels (after connection establishment)
  • Closing the remote peer connection should close all channels (after connection establishment)
  • Closing the peer connection should set the SCTP transport to closed
• RTCPeerConnection-createDataChannel
  • createDataChannel with both maxPacketLifeTime and maxRetransmits null should succeed
  • createDataChannel with too long label (2 byte unicode) should throw TypeError
  • createDataChannel with same label used twice should not throw
  • createDataChannel with too long protocol (2 byte unicode) should throw TypeError
  • createDataChannel with maximum length label and protocol should succeed
  • createDataChannel with negotiated false should succeed
  • createDataChannel with negotiated false and id 42 should ignore the id
  • createDataChannel with negotiated true and id not defined should throw TypeError
  • createDataChannel with negotiated true and id null should throw TypeError
  • Reusing a data channel id that is in use should throw OperationError
  • Reusing a data channel id that is in use (after setRemoteDescription) should throw OperationError
  • Reusing a data channel id that is in use (after setRemoteDescription, negotiated via DCEP) should throw OperationError
  • New data channel should be in the connecting state after creation (after connection establishment)
• RTCPeerConnection-ondatachannel
  • Should be able to send data in a datachannel event handler
  • Open event should not be raised when closing the channel in the datachannel event
  • Open event should be raised when closing the channel in the datachannel event after enqueuing a task
  • Open event should not be raised when sending and immediately closing the channel in the datachannel event
  • In-band negotiated channel created on remote peer should match the same (default) configuration as local peer
• RTCSctpTransport-constructor
  • setRemoteDescription() with answer not containing data media should not initialize pc.sctp
  • setLocalDescription() with answer not containing data media should not initialize pc.sct
• RTCSctpTransport-onstatechange
  • RTCSctpTransport.onstatechange should fire when the state changes
• historical
  • RTCDataChannel member maxRetransmitTime should not exist

Comments

Only few helper functions have been changed but everything is either backwards compatible or the affected tests have been updated. FWIW, I have carefully tested new test cases (if that was possible) on Firefox and Chrome by temporarily commenting out asserts and in some cases even by throwing in adapter (RTCSctpTransport.maxMessageSize for example).

Towards WebRTC implementations: A bunch of new issues are revealed by these tests. I'm at a tight time schedule, so I can't file them for you at the moment. This listing here should help you in finding new or updated test cases, so you can track and file new issues in your bug trackers. /cc @nils-ohlmeier @jan-ivar. Please also forward this to whoever is responsible for Google's, Safari's and Edge's (:joy_cat:) data channel implementation.

And I apologise for creating this rather large PR.

[youennf]

Looking at #11029 (comment) and #11120, WPT seems to go away from setting timeout in individual tests.

[lgrahl]

Magnificent...

This will require a lot of tedious refactoring and I don't have time for that before next year.

[lgrahl]

Note to self: Needs updating to some tests due to w3c/webrtc-pc#1999

[alvestrand]

@youennf will try to get the impractical (multi-minute) tests disabled or removed; we should merge-and-iterate on the rest.

[lgrahl]

@alvestrand The main problem seems to be the design choice now (local timeouts for each test, disabled global timeout). Since I have found a little bit of time, a suggestion: I'll refactor one of the larger test groups and you folks tell me if it looks good to you. Then I'll proceed with refactoring the rest. Sounds good?

[youennf]

Current plan is to split this PR as smaller pieces:

• 1 PR for RTCPeerConnection-helper.js changes
• 1 or more PRs for updated test files
• 1 or more PRs for new test files
  Some of the new test files may take a very long time.
  I am not sure we will be able to land all of them if it cannot be easily run in browsers CI.

[lgrahl]

To do notes (mostly for myself):

• PR for RTCSctpTransport changes
• Fix usages of t.step_func_done() (should use resolver.resolve() instead)
• Use @jan-ivar's suggestion for the "exchange ICE candidates later" use case

[jan-ivar]

This being the lone use of IceCandidateQueue, I propose using a promise as a simple queue here instead:

promise_test(async (t) => {
  const pc1 = new RTCPeerConnection();
  const pc2 = new RTCPeerConnection();
  t.add_cleanup(() => pc1.close());
  t.add_cleanup(() => pc2.close());
  // We use a queue to hold back candidates, so no connection can be established
  let flush;
  const queue = new Promise(r => flush = r);
  pc1.onicecandidate = e => queue.then(() => pc2.addIceCandidate(e.candidate));
  pc2.onicecandidate = e => queue.then(() => pc1.addIceCandidate(e.candidate));

  // Negotiate SCTP transport
  pc1.createDataChannel('test');
  await doSignalingHandshake(pc1, pc2);

  for (const pc of [pc1, pc2]) {
    assert_equals(pc.sctp.state, 'connecting',
      'RTCSctpTransport should be in the connecting state');
  }
  flush(); // Let the peers connect
  await Promise.all([pc1, pc2].map(async pc => {
    await new Promise(r => pc.sctp.onstatechange = r);
    assert_equals(pc.sctp.state, 'connected');
  }));
}

This should also fix that the test doesn't bail before both pcs have had their statechange event tested.

[jan-ivar]

@lgrahl Odd, looks like github had some old unsubmitted review comments cached here. The last comment is the one I wanted you to look at.

[jan-ivar]

@youennf ^

[lgrahl]

I'm fine with implementing that for this specific test case.

Should I resolve the rest of the comments or are these still valid?

[jan-ivar]

The first unresolved comment thread is the one about splitting this up. Presumably that's still happening? I just commented here because I couldn't find this test in the newer PRs. If we split it up and avoid modifying exchangeIceCandidates to take a queue, then I'm happy.

[alvestrand]

Status Dec 18: Waiting for split-up PRs to land, then we'll close this one.

[alvestrand]

Closing this PR on the assumption that split-up PRs have landed.