Time out when negotiating connection

[ppv-milestone]

Hi,
I have an implementation using WebRTC part of SIPSorcery in place which 2 out of 3 times works fine, but approximately every third attempt the flow seems to stop midway and then eventually times out. The logs will show the below (I have replaced TURN server address with 1.2.3.4, TURN server uri with TURNURI and certificate with XYZ), but then next line will be a timeout:

2024-12-10 09:54:56.528+01:00 INFO - sipsorcery: No DTLS certificate is provided in the configuration
2024-12-10 09:54:56.553+01:00 INFO - sipsorcery: RTCPeerConnection created with DTLS certificate with fingerprint sha-256 AC:F4:AC:A2:D6:EA:71:27:70:EC:BB:5E:82:DF:DE:92:B0:C9:47:E8:93:B4:63:D8:8A:17:E5:D3:6E:95:29:F3 and signature algorithm SHA256withECDSA.
2024-12-10 09:54:56.570+01:00 INFO - sipsorcery: -----BEGIN CERTIFICATE-----
XYZ
-----END CERTIFICATE-----
2024-12-10 09:54:56.587+01:00 INFO - sipsorcery: CreateRtpSocket attempting to create and bind RTP socket(s) on [::]:0.
2024-12-10 09:54:56.611+01:00 INFO - sipsorcery: CreateBoundSocket attempting to create and bind socket(s) on [::]:0 using protocol Udp.
2024-12-10 09:54:56.633+01:00 INFO - sipsorcery: CreateBoundSocket successfully bound on [::]:57918, dual mode True.
2024-12-10 09:54:56.649+01:00 INFO - sipsorcery: Successfully bound RTP socket [::]:57918 (dual mode True).
2024-12-10 09:54:56.671+01:00 INFO - sipsorcery: CreateRtpSocket attempting to create and bind RTP socket(s) on [::]:0.
2024-12-10 09:54:56.689+01:00 INFO - sipsorcery: CreateBoundSocket attempting to create and bind socket(s) on [::]:0 using protocol Tcp.
2024-12-10 09:54:56.713+01:00 INFO - sipsorcery: CreateBoundSocket even port required, closing socket on [::]:34347 and retrying on 34348.
2024-12-10 09:54:56.737+01:00 INFO - sipsorcery: Successfully bound RTP socket [::]:34348 (dual mode True).
2024-12-10 09:54:56.759+01:00 INFO - sipsorcery: RTPChannel for [::]:57918 started.
2024-12-10 09:54:56.778+01:00 INFO - SIPSorcery.Net.SctpDataReceiver: SCTP windows size for data receiver set at 218.
2024-12-10 09:54:56.793+01:00 INFO - sipsorcery: SCTP creating DTLS based association, is DTLS client False, ID 5000:5000:57918.
2024-12-10 09:54:56.814+01:00 INFO - sipsorcery: Adding ICE server for turn:TURNURI.
2024-12-10 09:54:56.837+01:00 INFO - sipsorcery: RTPIceChannel TCP for [::]:57918 started.
2024-12-10 09:54:56.874+01:00 INFO - sipsorcery: RTP ICE Channel discovered 0 local candidates.
2024-12-10 09:54:56.896+01:00 INFO - sipsorcery: Adding ICE server for turn:TURNURI.
2024-12-10 09:54:56.915+01:00 INFO - sipsorcery: Attempting to resolve STUN server URI turn:TURNURI.
2024-12-10 09:54:56.932+01:00 INFO - sipsorcery: ICE server turn:TURNURI successfully resolved to 1.2.3.4:443.
2024-12-10 09:54:56.973+01:00 INFO - sipsorcery: Sending TURN allocate request to ICE server turn:TURNURI with address 1.2.3.4:443.
2024-12-10 09:54:57.116+01:00 INFO - sipsorcery: SendOverTCP status: True endpoint: [::ffff:1.2.3.4]:443
2024-12-10 09:54:57.162+01:00 INFO - sipsorcery: Sending TURN allocate request to ICE server turn:TURNURI with address 1.2.3.4:443.
2024-12-10 09:54:57.225+01:00 INFO - sipsorcery: Sending TURN allocate request to ICE server turn:TURNURI with address 1.2.3.4:443.
2024-12-10 09:54:57.289+01:00 INFO - sipsorcery: Sending TURN allocate request to ICE server turn:TURNURI with address 1.2.3.4:443.
2024-12-10 09:54:57.351+01:00 INFO - sipsorcery: Sending TURN allocate request to ICE server turn:TURNURI with address 1.2.3.4:443.
2024-12-10 09:54:57.413+01:00 INFO - sipsorcery: Sending TURN allocate request to ICE server turn:TURNURI with address 1.2.3.4:443.
2024-12-10 09:54:57.477+01:00 INFO - sipsorcery: Sending TURN allocate request to ICE server turn:TURNURI with address 1.2.3.4:443.
2024-12-10 09:54:57.490+01:00 INFO - sipsorcery: TURN allocate success response received for ICE server check to turn:TURNURI.
2024-12-10 09:54:57.501+01:00 INFO - sipsorcery: Adding relay ICE candidate for ICE server turn:TURNURI and 1.2.3.4:27356.
2024-12-10 09:54:57.539+01:00 INFO - sipsorcery: ICE RTP channel stopping ICE server checks in gathering state complete and connection state new.
2024-12-10 09:54:58.552+01:00 INFO - sipsorcery: RTP ICE Channel remote credentials set.
2024-12-10 09:54:58.576+01:00 INFO - sipsorcery: RTP ICE Channel received remote candidate: 2558687369 1 udp 16777215 1.2.3.4 31186 typ relay raddr 10.24.31.13 rport 34750 generation 0
2024-12-10 09:54:58.596+01:00 INFO - sipsorcery: Remote ICE candidate not added as no available ICE session for component rtcp.
2024-12-10 09:54:58.609+01:00 INFO - sipsorcery: Adding new candidate pair to checklist for: udp:1.2.3.4:27356 (relay)->udp:1.2.3.4:31186 (relay)
2024-12-10 09:54:58.611+01:00 INFO - sipsorcery: LogRemoteSDPSsrcAttributes: Audio:[ ] 
 Video: [  ]
2024-12-10 09:54:58.622+01:00 INFO - sipsorcery: ICE RTP channel sending TURN permissions request 1 to server turn:TURNURI for peer 1.2.3.4:31186 (TxID: LWEQUAFGHWIN).
2024-12-10 09:54:58.773+01:00 INFO - sipsorcery: A TURN Create Permission success response was received from [::]:0 (TxID: LWEQUAFGHWIN).
2024-12-10 09:54:58.800+01:00 INFO - sipsorcery: ICE RTP channel sending connectivity check for udp:1.2.3.4:27356 (relay)->udp:1.2.3.4:31186 (relay) from [::]:57918 to relay at 1.2.3.4:443 (use candidate False).
2024-12-10 09:54:58.945+01:00 INFO - sipsorcery: ICE RTP channel binding response state Succeeded as Controller for udp:1.2.3.4:31186 (relay)
2024-12-10 09:54:58.955+01:00 INFO - sipsorcery: ICE RTP channel sending connectivity check for udp:1.2.3.4:27356 (relay)->udp:1.2.3.4:31186 (relay) from [::]:57918 to relay at 1.2.3.4:443 (use candidate True).

The times where it work I get the exact same, but then it will continue with this:

2024-12-10 09:55:43.264+01:00 INFO - sipsorcery: ICE RTP channel remote peer nominated entry from binding response udp:1.2.3.4:33271 (relay)
2024-12-10 09:55:43.275+01:00 INFO - sipsorcery: ICE RTP channel connected after 2459ms udp:1.2.3.4:33270 (relay)->udp:1.2.3.4:33271 (relay).
2024-12-10 09:55:43.315+01:00 INFO - sipsorcery: ICE connected to remote end point 1.2.3.4:33271.
2024-12-10 09:55:43.337+01:00 INFO - sipsorcery: Starting DLS handshake with role passive.
2024-12-10 09:55:43.356+01:00 INFO - sipsorcery: RTCPeerConnection DoDtlsHandshake started.
2024-12-10 09:55:43.374+01:00 INFO - sipsorcery: DTLS commencing handshake as server.
2024-12-10 09:55:43.398+01:00 INFO - sipsorcery: Server offered cipher suites:
 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
 TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
 DRAFT_TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
2024-12-10 09:55:43.408+01:00 INFO - sipsorcery: Client offered cipher suites:
 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
2024-12-10 09:55:43.428+01:00 INFO - sipsorcery: Selected cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256. Using SHA256withECDSA certificate with fingerprint sha-256 EA:B1:56:02:CD:F0:54:73:7F:73:5A:3E:27:31:42:0D:2D:35:CA:64:E5:00:30:3B:60:3F:9A:2D:4C:4B:67:23.
2024-12-10 09:55:43.722+01:00 INFO - sipsorcery: RTCPeerConnection DTLS handshake result True, is handshake complete True.
2024-12-10 09:55:43.747+01:00 INFO - sipsorcery: Remote peer DTLS certificate, signature algorithm SHA256withECDSA.
2024-12-10 09:55:43.766+01:00 INFO - sipsorcery: -----BEGIN CERTIFICATE-----
XYZ
-----END CERTIFICATE-----
2024-12-10 09:55:43.780+01:00 INFO - sipsorcery: RTCPeerConnection remote certificate fingerprint matched expected value of FE:50:1F:8B:5F:A8:AD:7C:57:A1:6A:D9:FF:23:12:ED:BF:5B:A3:49:44:8C:90:44:AB:0D:1E:DE:18:E0:75:C4 for sha-256.

We have another web based client towards the same servers and do not see the same issue there. Does anyone have an idea what could be the reason why "ICE RTP channel sending connectivity check for" often times out?

[sipsorcery]

I can replicate the same behaviour and log messages if I re-use an RTCPeerConnection instance. Each instance is only intended to be used once. For each new WebRTC peer connection you should be creating a brand new instance.

If it helps I use the webrtccmdline tool to test scenarios like this.

Signalling server peer:

sipsorcery\examples\webrtccmdline>dotnet run --ws --stun turn:azamicrok8s?transport=tcp --relayonly --accepticetypes=relay

Signalling client peer:

sipsorcery\examples\webrtccmdline>dotnet run --wsclient ws://127.0.0.1:8081 --stun turn:azamicrok8s?transport=tcp --relayonly --accepticetypes=relay

[ppv-milestone]

Thanks a lot for your response! I am, however, not reusing the peer connection. I create one RTCPeerConnection for each session and it is closed and discarded after use.

[ppv-milestone]

@sipsorcery One thing I have noticed is that the problem is much less likely to happen when I run with the debugger attached (actually, I have not managed to reproduce at all with debugger on), so it seems like this could be a timing issue. Does that make any sense?

[sipsorcery]

@ppv-milestone I do plan on doing some more digging into this issue, hopefully should get some time over the next few days/week. A timing issue could make sense. Setting up an ICE channel via TURN is the most complicated and, in the sipsorcery library's case, also the least tested path.

[ppv-milestone]

I would greatly appreciate that. I am also trying to figure out what is going on, but it is complicated, and not being able to reproduce with debugger attached is not exactly helping either.