Plan for 2.0.0-M11 🎯

[sbernard31]

A 2.0.0-M11 was requested by Orange, so let's use this issue to organize this new release.

One BIG change :

This milestones release will include massive changes about transport layer abstraction (#1025 (comment))
This will come will lot of API changes and also some break on Redis stores data format.

About stability of the Transport layer abstraction, we have some good indicators :
1.) Those changes are include in master since begin of January and so running on leshan sandbox without reported issue.
2.) Some contributors tested it without issues (#1398 (comment), #1222 (comment))
3.) Orange also tested it and reported some bugs which are fixed in master. (#1384, #1385, #1387)

On my side, even with that I feel this is maybe a bit too soon but I'm probably too cautious.

Current modifications in master since M10 are :

d7ed1cfe6: Remove usage of deprecated API of Californium 3.8.0
3c4668b39: Integration of Californium 3.8.0
bee515350: Migration from Junit4 to Junit5.
dbe5aa218: Add some test about LwM2mObservationStore.remove()
233c209a9: GH-1384: Fix issue in LwM2mObservationStore.remove(Token)
346223b13: Add method RegistrationStore.getObservation(ObservationIdentifier)
96fcbded4: Remove redundant validation in Registration constructor
0f0464a39: Add lastEndpointUsed URI validation in Registration Builder.
c31ec7800: lastEndpointUsed in Registration should me mandatory
bdc1d7242: Fix NPE in RedisRegistrationStoreTest
250c8e2e2: GH-1387: Fix BootstrapDeleteRequest always delete all instances
a6e55d737: Add some Object Enabler validation at client creation.
73fd995b3: Fix Registration Update triggered by SUPPORTED_BINDING changes.
6a1ff2dad: GH-1376: Better to raise CodecException in LwM2mNodeSenMLDecoder.
a963d778b: Remove no more used classes (since transport layer refactoring is done)
6091fc3ba: Add Transport Layer abstraction at Bootstrap server side.
9859a330e: Add Transport Layer abstraction at Client side.
fa38111cf: Add Transport Layer abstraction at Server side.
2a2a8a7ec: GH-1374: fix NPE on server.destroy() when queue mode is disabled
5b0ccd16c: Fix Typo in RegistrationHandlerTest (givenIdenity => givenIdentity)
a851b81ba: Add MdcConnectionListener to all Demo.

More to include in M11 :

Ideally I would like to add :

• Some Idea to try to improve API to create CaliforniumServerEndpointsProvider #1395
  (if you have opinions about it let me know commenting at this issue ☝️ )

Maybe it makes sense to add ongoing work on Redis Store too :

• Allow to set custom REDIS keys for security endpoint and PSK ID #1398
• Configurable prefix for Redis stores #1249
  (@aliakseiz, @jvermillard let us know about that)

Maybe also :

• Invalid datatype for DTLS/TLS Ciphersuite resource (/0/?/16) in BootstrapConfig. #1402

I'm currently refactoring integrations tests (#1399), this will maybe help to find more issues about transport layer abstraction but clearly this is probably not a big deal if this is not included in M11.

If you have more ideas, please comments 👇

Targeted release date :

@JaroslawLegierski, @Warmek could you give us approximate date which could be OK for you ?

Updated : Ideally before March 13th

(Any other comments about that feel free to comments 😉)

[JaroslawLegierski]

@JaroslawLegierski, @Warmek could you give us approximate date which could be OK for you ?

It would be great if M11 could be available before March 13th

When it comes to issue 1402 - if it can be fixed before that's fine, (but we can live without it in M11).

[sbernard31]

That means we have 2 weeks.
But I guess you want kind of code freeze several days before the release to test it ? That means probably more 1 week or 1,25 week to work on feature just above. We will try but no guarantee.

On my side, Monday, I will pause my work on integrations tests and I will start to work on : #1395. That lets you today, if you eventually want to provide feedback.

[aliakseiz]

From my side, I will definitely finish #1398 and will do my best to complete #1249 as well.

[sbernard31]

I implemented #1395 with #1407 : if wanted, feedback are welcomed 🙂

[sbernard31]

#1402 is now fixed.

I go back to work on "refactoring of integrations tests" (#1399) maybe I will new find bugs. 🎯 !

[sbernard31]

(I will be out of office tomorrow)

[sbernard31]

All planned tasks are now integrated into master. 🎉

Current change logs looks like :

cc9f41dfc: Use new endpoints provider API in BS server and server demos/tests
95b344d16: GH-1395: Add a way to customize DTLS Config at ProtocolProvider level
0d0b57c7d: GH-1395: Add Builder to Server/BootstrapServer EndpointFactory.
f89cd16cb: GH-1395: Add a way set Configuration with lambda in Endpoints Provider
3e6b0a594: Fix getLoggingTag in EndpointFactory
5dec807b7: Enhance error message form EndpointUriUtil.validateURI()
a7e4a5e5a: GH-1249: Allow to use custom KEYS in RedisRegistrationStore
b8d817b29: GH-1410: Fix  github action issue with impsort-maven-plugin.
8f0db2449: GH-1402:Fix Invalid datatype of Ciphersuite resource in BootstrapConfig
fd5c9c0b4: Allow to use custom KEYS in RedisSecurityStore
b7dc3cf3f: Fix typo in CONTRIBUTING.md guide
d7ed1cfe6: Remove usage of deprecated API of Californium 3.8.0
3c4668b39: Integration of Californium 3.8.0
bee515350: Migration from Junit4 to Junit5.
dbe5aa218: Add some test about LwM2mObservationStore.remove()
233c209a9: GH-1384: Fix issue in LwM2mObservationStore.remove(Token)
346223b13: Add method RegistrationStore.getObservation(ObservationIdentifier)
96fcbded4: Remove redundant validation in Registration constructor
0f0464a39: Add lastEndpointUsed URI validation in Registration Builder.
c31ec7800: lastEndpointUsed in Registration should me mandatory
bdc1d7242: Fix NPE in RedisRegistrationStoreTest
250c8e2e2: GH-1387: Fix BootstrapDeleteRequest always delete all instances
a6e55d737: Add some Object Enabler validation at client creation.
73fd995b3: Fix Registration Update triggered by SUPPORTED_BINDING changes.
6a1ff2dad: GH-1376: Better to raise CodecException in LwM2mNodeSenMLDecoder.
a963d778b: Remove no more used classes (since transport layer refactoring is done)
6091fc3ba: Add Transport Layer abstraction at Bootstrap server side.
9859a330e: Add Transport Layer abstraction at Client side.
fa38111cf: Add Transport Layer abstraction at Server side.
2a2a8a7ec: GH-1374: fix NPE on server.destroy() when queue mode is disabled
5b0ccd16c: Fix Typo in RegistrationHandlerTest (givenIdenity => givenIdentity)
a851b81ba: Add MdcConnectionListener to all Demo.

📢 : So we will enter in small ❄️ code freeze 🥶 until the end of the week. During this period only small bug fix will be allowed.

@JaroslawLegierski, @Warmek, @aliakseiz, @jvermillard, @cyril2maq. if you can, please 🙏 use those few days to test current master to give us more chance to have a stable 2.0.0-M11 release.

(And let me know, if you find anything wrong ❌ or all is OK ✔️)

[jvermillard]

I'm trying to rebase my branch; I see some things to improve:
https://github.com/eclipse/leshan/blob/master/leshan-server-core/src/main/java/org/eclipse/leshan/server/registration/Registration.java#L350

The "last used endpoint" concept is not explained; it would be nice to add some Javadoc.

[sbernard31]

The "last used endpoint" concept is not explained; it would be nice to add some Javadoc.

Yes, there is currently a lot of javadoc missing.
I don't know if this is a good reason (probably not), but I didn't write too much javadoc because I wasn't sure about the API and didn't plan to release a M11 so soon.

We can add some javadoc before M11 as there is no risk of adding regressions with this kind of changes.

Let me know if there are any other important javadoc missing.

[jvermillard]

After rebasing, my code works on master 👍

[cyril2maq]

On my side, I did rebase; use new mechanisms for customized DTLS config; and re-run tests that revealed defects I had found on the first M11 draft (and you fixed) and all is ✔️ on my side.

Good job 👍

[sbernard31]

@jvermillard, about "last used endpoint", I think a bit about it and I feel this is not a so good name ... see #1415, but I don't know if this is good idea to change it now just before the release. (I guess it's not but that means maybe new compatibility break on next release 😬)

@cyril2maq, I understood that you are using Redis stores, correct ? How did you deal with data format compatibility break ? I'm curious.

@jvermillard, @cyril2maq, @aliakseiz I don't know if you are aware but there nightly build for master and 1.x branch. (I just fixed an issue about it #1408)
Maybe this repository could be useful to you to test master continiously.
Nightly build (snapshot) are available on eclipse repo. (#885)

[cyril2maq]

Yes we do use a RedisRegistrationStore.

Regarding the compatibility break, we have used the redis database mechanism.
We use a specific database number for this registrationStore, so we just increment the redis database number for this M11 version (so new instances will start with an empty store 😞).
We also have created a migration job which will clear previously used redis database once our M11 based application will be deployed.

So it would be great not to break this model too often !

[sbernard31]

So it would be great not to break this model too often !

We will try 😅

[jvermillard]

We need to force Simon to write a migration script every time he decides to break redis compatibility 🤭

[sbernard31]

We need to force Simon to write a migration script every time he decides to break redis compatibility 🤭

I didn't even want official store implementation based on Redis in Leshan 😁

More seriously, I guess for Registration Store the simple solution could be to just drop all data ias this is just a kind of "session" store.

If migration is really needed, I don't know what is the best move :

• writing a migration script
• or writing a redis implementation store which is able read previous format.

(If this is needed, help from community is welcomed 🙏)

[jvermillard]

maybe we should just introduce a "version" so we can either discard or introduce some migration script?

[JaroslawLegierski]

I used the M11 release during lb retests and it seems to be working fine. I have info from my French colleagues that on their side is also ✔️

[sbernard31]

@aliakseiz let me know if you plan to test this before the release ?

I plan to begin to prepare the release around : 15h UTC

[sbernard31]

About Redis store compatibility break :

• How to handle data format Compatibility break in Redis Store ? #1417
• Redis Store Migration from 2.0.0-M10 to 2.0.0-M11 #1418

[aliakseiz]

@sbernard31 I rebased our gateways in the test environment to master and all client devices are operating normally 👍

[sbernard31]

All lights are green 🟢, I start the release process... 🚧 !

(:building_construction: : https://ci.eclipse.org/leshan/job/leshan-release-java8/)

[sbernard31]

Leshan v2.0.0-M11 should now be available on maven central ! 🎉

Thanks all for your helps 🙏, that's really appreciated !

[sbernard31]

I created the release in github : https://github.com/eclipse/leshan/releases/tag/leshan-2.0.0-M11.

I think we can close this issue.