Add CoAP over TCP transport layer

[sbernard31]

This issue aims to centralize discussion about adding CoAP over TCP transport layer.

[boaks]

Californium - Complete the experimental CoAP over TCP implementation

Beside of the integration of BERT (TCP-blockwise) in 3.0, no further progress.

From my side, still no plans to work on the TCP stuff. Help and contributions will be welcome.

[sbernard31]

Some very very experimental implementation at #1312

[sbernard31]

Currently, I think there is some not so clear point in LWM2M over CoAP over TCP.
Here a summarize of my current understanding.

[sbernard31]

The current idea would be to implement this based on Californium. (another idea would be to try another library)
The current coap+tcp experimental implementation of Californium is far to be "complete" or even in "minimal viable feature" state.

See : https://github.com/eclipse-californium/californium/wiki/CoAP-over-TCP

So if we want to go further on this feature, the next step would probably be to implement missing feature in Californium. (at least what seems to be mandatory for our LWM2M use case)

Thinking a bit about that I bet that this will bring lot of API break in Californium.
E.g we probably need to add more "connection oriented event" Californium "Layer"
So this could only be doable for Californium v4.0.0.

[boaks]

Maybe we ask some developers of a C-library, which features that C-library supports/requires?
Beside of a compatible implementation, a subset may also help to see, if the benefits of TCP justify the implementation invest.

[sbernard31]

Clearly, a first step would be to just target a Minimal Viable Feature. (not a full implementation of the RFC)
I tried to identify what could be the scope of this Minimal Viable Feature at https://github.com/eclipse-californium/californium/wiki/CoAP-over-TCP.

I guess LWM2M use case will also bring some more new Californium API need (especially all about controlling/monitoring TCP connection lifetime) but it's not so clear to me for now.

Maybe we ask some developers of a C-library, which features that C-library supports/requires?

Yep It could help to define the scope of this Minimal Viable Feature.

[boaks]

Yep It could help to define the scope of this Minimal Viable Feature.

Done via e-mail. I added you to cc.

[sbernard31]

The answer's we get by mail from @mrdeep1

Work was in progress for CoAP over TCP when I started to use libcoap - which was driven by the DOTS requirements (initially RFC8782, superseded by RFC9132) which required support for CoAP over both UDP and TCP. That did cause some fixes to the libcoap TCP logic - in particular for the TLS support.

As you are aware, libcoap does not support the Websockets component of RFC8323. It also does not take action (other than checking syntax is correct) on CSM Release or CSM Abort (no one has raised questions about them, but should be easy to fix). Otherwise, I am pretty sure that libcoap is fully compliant and gets upset if the remote end is violating a MUST. Testing against the demo coap-server will highlight any implementation issues, as well as the demo coap-client will indicate if the remote end is not playing ball.

Libcoap fully supports the other CoAP RFCs - i.e. Observe/Block over TCP.

I think that you would need to implement all the MUST - especially where the remote end requires to make a decision.

I have no experience of interoperating against java implementations for TCP. I have fixed libcoap bugs against other CoAP implementations (as well as caused other implementations to get fixed!).

[sbernard31]

I just tested californiuum with libcoap (coap-server/coap-client) :

✔️ Tested with libcoap at client and server :

2022-11-28 16:14:01,483269768	127.0.0.1	127.0.0.1	38111	5683	CoAP	73	7.01 CSM
2022-11-28 16:14:01,483278739	127.0.0.1	127.0.0.1	5683	38111	CoAP	73	7.01 CSM
2022-11-28 16:14:01,483338328	127.0.0.1	127.0.0.1	38111	5683	CoAP	81	GET, TKN:01, /test
2022-11-28 16:14:01,483383129	127.0.0.1	127.0.0.1	5683	38111	CoAP	79	4.04 Not Found (text/plain), TKN:01, /test

✔️ Tested with java-coap at client side and libcoap at server side :

2022-11-28 16:33:19,780459494	127.0.0.1	127.0.0.1	5683	32802	CoAP	73	7.01 CSM
2022-11-28 16:33:19,800034426	127.0.0.1	127.0.0.1	32802	5683	CoAP	71	7.01 CSM
2022-11-28 16:33:19,814653271	127.0.0.1	127.0.0.1	32802	5683	CoAP	74	GET, /hello
2022-11-28 16:33:19,814798963	127.0.0.1	127.0.0.1	5683	32802	CoAP	78	4.04 Not Found (text/plain)

❌ Tested with Californium at client side and libcoap at server side :

2022-11-28 16:31:45,672056141	127.0.0.1	127.0.0.1	5683	59806	CoAP	73	7.01 CSM
2022-11-28 16:31:45,757129610	127.0.0.1	127.0.0.1	59806	5683	CoAP	92	GET, TKN:1c 4e f8 cd a4 89 f9 15, coap://localhost/test

// with log : 
Nov 28 18:03:24.386 DEBG ***[::ffff:127.0.0.1]:5683 <-> [::ffff:127.0.0.1]:44116 TCP : session 0x561e1f87f860: new incoming session
Nov 28 18:03:24.386 DEBG ***EVENT: 0x1001
Nov 28 18:03:24.386 DEBG ***[::ffff:127.0.0.1]:5683 <-> [::ffff:127.0.0.1]:44116 TCP : sending CSM
Nov 28 18:03:24.387 DEBG *  [::ffff:127.0.0.1]:5683 <-> [::ffff:127.0.0.1]:44116 TCP : sent 7 bytes
v:1 t:CON c:CSM i:0000 {} [ Max-Message-Size:8388864, Block-Wise-Transfer: ]
Nov 28 18:03:24.387 DEBG ***EVENT: 0x4001
Nov 28 18:03:24.455 DEBG *  [::ffff:127.0.0.1]:5683 <-> [::ffff:127.0.0.1]:44116 TCP : received 26 bytes
v:1 t:CON c:GET i:0000 {f495af4dad0ba315} [ Uri-Host:localhost, Uri-Path:test ]
Nov 28 18:03:24.455 DEBG request for unknown resource 'test', return 4.04
Nov 28 18:03:24.455 DEBG ** [::ffff:127.0.0.1]:5683 <-> [::ffff:127.0.0.1]:44116 TCP : mid=0x0: delayed
Nov 28 18:03:34.471 DEBG ***[::ffff:127.0.0.1]:5683 <-> [::ffff:127.0.0.1]:44116 TCP : session disconnected (reason 1)
Nov 28 18:03:34.472 DEBG ** [::ffff:127.0.0.1]:5683 <-> [::ffff:127.0.0.1]:44116 TCP : mid=0x0: not transmitted after disconnect
Nov 28 18:03:34.472 DEBG ***EVENT: 0x1002
Nov 28 18:03:34.472 DEBG ***EVENT: 0x2003
Nov 28 18:03:34.472 DEBG ***EVENT: 0x4002
Nov 28 18:03:34.472 DEBG ***[::ffff:127.0.0.1]:5683 <-> [::ffff:127.0.0.1]:44116 TCP : session 0x561e1f87f860: closed

☝️ I guess server never send any answer because response is delayed waiting client send CSM message.

❌ Tested with libcoap at client side and Californium at server side :

2022-11-28 17:06:46,741370664	127.0.0.1	127.0.0.1	40643	5683	CoAP	73	7.01 CSM
 

// with log : 
Nov 28 18:06:46.741 DEBG ***0.0.0.0:0 <-> 127.0.0.1:5683 TCP : session 0x55b4e99dba00: created outgoing session
Nov 28 18:06:46.741 DEBG ***EVENT: 0x1001
Nov 28 18:06:46.741 DEBG ***127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : sending CSM
Nov 28 18:06:46.741 DEBG *  127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : sent 7 bytes
v:1 t:CON c:CSM i:0000 {} [ Max-Message-Size:8388864, Block-Wise-Transfer: ]
Nov 28 18:06:51.746 DEBG ** 127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : timeout waiting for first response
Nov 28 18:06:51.746 DEBG timeout is set to 90 seconds
Nov 28 18:06:51.746 DEBG sending CoAP request:
Nov 28 18:06:51.746 DEBG PDU presented by app
v:1 t:CON c:GET i:ce12 {01} [ Uri-Path:test, Request-Tag:0x5cb1a6b4 ]
Nov 28 18:06:51.746 DEBG ** 127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : lg_crcv 0x55b4e99dc560 initialized - stateless token xxxx000000000002
Nov 28 18:06:51.746 DEBG ** 127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : mid=0xce12: delayed
Nov 28 18:06:56.897 DEBG ***127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : session disconnected (reason 1)
Nov 28 18:06:56.899 DEBG ** 127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : mid=0xce12: not transmitted after disconnect
Nov 28 18:06:56.900 ERR  cannot send CoAP pdu
Nov 28 18:06:56.900 DEBG ** 127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : lg_crcv 0x55b4e99dc560 released
Nov 28 18:06:56.900 DEBG ***EVENT: 0x1002
Nov 28 18:06:56.900 DEBG ***EVENT: 0x2003
Nov 28 18:06:56.900 DEBG ***127.0.0.1:40643 <-> 127.0.0.1:5683 TCP : session 0x55b4e99dba00: closed

☝️ I guess client never send the request because, it doesn't receive the CSM message ? 🤔

🎯 So without surprises, we should probably focus on Capabilities and Settings Messages first.

[mrdeep1]

Agreed. libcoap is following (and hence expecting the first thing to be received is a CSM) as per RFC8323 Section 5.3

   One CSM MUST be sent by each endpoint at the start of the Transport
   Connection.  Additional CSMs MAY be sent at any other time by either
   endpoint over the lifetime of the connection.

[sbernard31]

Using Californium to implement coap+tcp sounds not to be the right way. (see eclipse-californium/californium#2092 (comment))

So next ideas to try, in order will be :

1. try with java-coap (fork).
2. try to implement a new coap+tcp lib.

But before to work on coap+tcp with java-coap we need to work on : #1373

[sbernard31]

• Experiment Transport layer with java-coap (fork) #1373 is done

So we can try to use this lib now to support coap+tcp.

[sbernard31]

Draft about coap+tcp based on java-coap : #1528