End-to-End, Multi-Tier architecture

[azizalaliq8]

Hello, Leshan Development Team.

Hope you are well.

Thank you for creating Leshan. Great work 👍. We are looking for some help regarding deploying Leshan in an End-to-End, Multi-Tier architecture fashion like the diagram provided below.

Scenario :

• End-user (citizen) has multiple end devices (IoT devices such as Indoor Air Quality Sensors/ Smart Plugs etc.) running LwM2M on Contiki-NG OS in a house.

• The end devices in the house send data to Leshan Server running at the Edge (e-Leshan Server) (running on Raspberry Pi).

• Let us assume that above setup is deployed in multiple homes (10 homes).

Further, we want to provide few capabilities such that:

• the end-devices (sensor nodes) in the house can be managed centrally by the administrator (via cloud) as well as the end-user (living in the house).

• the end-user can manage the device as per their privacy preferences. For Instance, they may want to turn off the sensor for an hour; send only specific environmental parameters to the central server; only select few devices to send the data. For instance, the end-user lives in 4 room house and has air quality sensor in all room but only want to send the values to cloud of 2 rooms.

Problem:

• Currently, we can deploy the Leshan server either on the cloud or on edge (user homes).

• When the Leshan server is deployed on edge, managing all the deployed end-devices in the different homes becomes challenging for the IT administrator as the devices are behind the home-router firewall. Further, if the number of end-devices is large, management of devices in terms where they are located is challenging.

• When the Leshan server is deployed on the cloud, the end-user loses the capability to configure the privacy settings.

Possible Solution:

• A possible solution could be running a Leshan server on the cloud as well as the edge.

• The edge Leshan-server (e-Leshan) would communicate with the cloud-Leshan-server (c-Leshan).

• The c-Leshan will be able to manage the eLeshan and the end-devices connected to the e-Leshan.

Possible Requirements:

• R1: Provide information to e-Leshan about the c-Leshan (username/password/token, cloud IP Address) to connect.

• R2: Provide encrypted communication between e-Leshan and c-Leshan such that e-Leshan can send and receive information.

• R3: e-Leshan should be able to send the information about end-nodes and c-Leshan should be able admin the e-Leshan end-devices.

Queries:

Q1: What is the best way to implement R1, R2, R3.

Q2: Regarding R1, on c-Leshan maybe we can keep a static token and run a service on a port. On e-Leshan, we can provide that static token with IP:Port. Also, how do we store the credentials at the edge? (Should we have mySQL database or something).

Q3: Regarding R2, is there a way or library which can be used to have e-Leshan to c-Leshan communication?

Q4: Regarding R3, is there a way to select all or some of the end-device and send those to e-Leshan or allow central administrator to admin those?

Any guidance much appreciated, Thank you in advance :)

[sbernard31]

This is an interesting use case 🤔
Thx to share it 🙏.

My understanding is that the communication between IoT devices and Edge Server is perfectly in the scope of Leshan library. But considering your design, I guess the communication and logic between Edge Server and Cloud Server is out of scope. I mean this is no more linked to LWM2M protocol and I guess this is up to you to design it as you want.
So for me :

• Edge Server should use Leshan to manage communication with IoT devices
• It's unclear why Cloud Server should use Leshan to communicate with Edge server as it will probably not use LWM2M directly.

About your questions :
Q1/Q2: I will probably disappoint you but to me this is classic cloud/distributed application issue. I'm not sure I can offer you better advice than what you could find on the web.
Q3; You will easily find the library you need in java once you decide which protocol to use. Java ecosystem is huge.
Q4: I'm not sure I get the point but I guess this is up to you to design the API between Edge and Cloud server. Once you did it you should be able to manage which devices are visible for the central administrator.

[sbernard31]

Some thoughts about this regarding your use case and the LWM2M specification
I will share some ideas which comes to my mind reading you but your solution could be totally valid and maybe even better.

1. I ask myself if there is bootstrap server involved in all of this ?
2. I can imagine a solution with multi server at client side, each device will have 2 server the cloud and the edge server and they will talk in LWM2M directly to both server. Edge and cloud server does not talk to each other directly (or at least this is not needed to interact with the device) End user will be able to select what admin can do using ACL.
3. I see in LWM2M 1.2, there is a kind if LWM2M gateway concept but this is not clear at all : (see More detailed explanation about LWM2M Gateway ? OpenMobileAlliance/OMA_LwM2M_for_Developers#518)

About 2) this is a solution which uses more LWM2M but I don't know if this is a better solution in your case as this bring several issue :

• multi-server and ACL must be supported at device side. (so bring more complexity at device side)
• to contact cloud server device should probably use Queue mode (because of NAT/firewall issue) and this will maybe prevent to not use queue mode for edge server. (see Disable queue mode by sending Update Operation OpenMobileAlliance/OMA_LwM2M_for_Developers#490 (comment))
• I don't know if this is possible to easily disable the cloud server, I guess if it can it could at least be deleted. (see Disable Server indefinitely OpenMobileAlliance/OMA_LwM2M_for_Developers#519)

So I just share you this idea but at first sight I guess you way sounds better.

[bitvijays]

Thank you @sbernard31 Hope you are doing well and had a good weekend. Thank you for your support and creating issues to track on OMA_LwM2M_for_Developrs. We really do appreciate it 👍

• Edge Server should use Leshan to manage communication with IoT devices
• It's unclear why Cloud Server should use Leshan to communicate with Edge server as it will probably not use LWM2M directly.

Agreed. Edge-Leshan to Cloud-Leshan need not be LWM2M. It might be just a wrapper which allows edge-Leshan server to connect to cloud-Leshan server.

About your questions :
Q1/Q2: I will probably disappoint you but to me this is classic cloud/distributed application issue. I'm not sure I can offer you better advice than what you could find on the web.

• Q1a: For the cloud/edge communication, We were thinking something similar to Kubernetes or K3S (Master and Worker node). For instance, we can develop a servlet on the cloud-leshan, which provides a random join token which could be provided to the client to join the cloud-leshan. Concept-wise this probably sounds good. However, we are not aware how to do it/ which library can be utilised. Few pointers in this direction would be appreciated. (Unfortunately, we are beginners in the Java ecosystem.)

Q3; You will easily find the library you need in java once you decide which protocol to use. Java ecosystem is huge.

• Q3a: In terms of protocols, Did you meant TCP/UDP? Reading Leshan Spec 1.2, it seems we perform lwm2m messaging over TCP/TLS, HTTP, MQTT. We would be happy to use anything which is secure, so probably TCP/TLS, HTTPS?

Q4: I'm not sure I get the point but I guess this is up to you to design the API between Edge and Cloud server. Once you did it you should be able to manage which devices are visible for the central administrator.

• Q4a: We spend sometime, understanding the LeshanServerDemo. In the ClientServer.java, There is this code, which collects all the endpoint registration.

            Collection<Registration> registrations = new ArrayList<>();
            for (Iterator<Registration> iterator = server.getRegistrationService().getAllRegistrations(); iterator
                    .hasNext();) {
                registrations.add(iterator.next());
            }

We were thinking,

• After getting all the endpoints, If we send the registrations array list to the cloud-server with the additional unique_edge_id and edge_ip_address. On the cloud side, display these endpoints with the edge_id and edge_ip_address.

• On the cloud, when the administrator clicks the Read/Discover, we can just pass the request to particular edge_ip. Similar to client read request

                ReadRequest request = new ReadRequest(contentFormat, target);
                ReadResponse cResponse = server.send(registration, request, extractTimeout(req));
                processDeviceResponse(req, resp, cResponse);

Do you think, this makes senses and might be a good direction to follow?

Thank you :)

Yours Sincerely,
bitvijays

[sbernard31]

@bitvijays,

Q1a: I guess there is several way to manage authentication to your cloud API (see https://zapier.com/engineering/apikey-oauth-jwt/) but it's up to you to decide which one is the more adapted.

Q3a: I was not talking about LWM2M communication but about your edge/cloud communication API. I guess some kind of REST API could be used but you need to be sure it is adapted for bi-directionnal communication and your "home-router firewall" constraint.

Q4a: I'm not sure I get your point. My understanding was that cloud server does not talk directly to the device but ask to edge server to do it :

• cloud asks edge to send a request S1 to the device with the endpoint name E1. (Your custom protocol / cloud --> edge) (cloud does not need to know the IP of the device it is probably not accessible anyway as it is behind the home router)
• edge server search the registration R1 for the endpoint name E1. (he also check if server is allowed to do that)
• edge send the request to R1 and wait the response. (LWM2M protocol edge <--> device)
• edge send-back the response to cloud. (Your custom protocol / edge --> cloud)

Anyway, keep in mind this is just hint from someone who have a slight idea of your needs. I can not take too much time to dig this subject as this is clearly out of the scope of Leshan and LWM2M.

[bitvijays]

@sbernard31 Agreed Cloud-Leshan server should never talk to the device. It asks Edge Leshan server to do it. We should have made it more clear: unique_edge_id and edge_ip_address refers to the Edge-Leshan-Server ID and IP address. The workflow you have defined is good.

Our assumption is if we are able to transfer registration array list containing the output of server.getRegistrationService().getAllRegistrations(); to the cloud-leshan server and import it on the cloud-server.

On the cloud-leshan server, we should see something like (after making changes to the template)

Client Endpoint	Registration ID	Edge Leshan Server	Registration Date	Last Update
Test	ABCDEF	ELS-1 (192.168.1.51)	Date	Date

Right?

[sbernard31]

Our assumption is if we are able to transfer registration array list containing the output of server.getRegistrationService().getAllRegistrations(); to the cloud-leshan server and import it on the cloud-server.

Ok get it now.

On the cloud-leshan server, we should see something like (after making changes to the template)

What template are you talking about ?

[bitvijays]

@sbernard31 We meant, making minor changes to the ClientServlet.java in Leshan Demo

Also, wanted to ask few things

• Do you think, this would be useful to the community?
• Do you other small projects kind of which can be done for the community.

Probably, in the coming year, we would be able to work more on Leshan.

[sbernard31]

Do you think, this would be useful to the community?

Which part exactly 😅 ?

Do you other small projects kind of which can be done for the community.

No sure to get you 🤔

Probably, in the coming year, we would be able to work more on Leshan.

Contribution of any kind (feedback about code/feature/API/documentation, tests, bug report, code contribution) is very welcome. 🙏 !

[bitvijays]

@sbernard31 This project, if we are able to develop Cloud-Leshan and communication from Edge-Leshan to Cloud. Also, Is there a way to chat with you apart from Github? Slack/ Gitter for Leshan?

Also, We were thinking maybe a good idea to enable GitHub Discussions

[sbernard31]

This project, if we are able to develop Cloud-Leshan and communication from Edge-Leshan to Cloud.

At first sight, I see it more like a new project based on Leshan than a part of Leshan. 🤔
Leshan scope is about implement LWM2M protocol + some simple demo (not intended to be reused), but it would be great to have a more complete ecosystem with new tools based on Leshan. Currently I don't know if Leshan project itself is the right place but I guess this is a not so important question for now :-)

Also, Is there a way to chat with you apart from Github? Slack/ Gitter for Leshan?

No generally, I prefer to use github because this is easy to refer previous conversation (e.g. to answer question or to give feedback to OMA, this kind of thing ...)

Also, We were thinking maybe a good idea to enable GitHub Discussions

This could be an idea do you know what is benefits comparing to github issue ?

(Just to let you know I'm currently experimenting vuejs to maybe rewrite totally the demo: #906)

[sbernard31]

@bitvijays in this topic I was talking to you about the LWM2M Gateway feature of LWM2M 1.2.

The specification of this was release recently : http://www.openmobilealliance.org/release/LwM2M_Gateway/ (see OpenMobileAlliance/OMA_LwM2M_for_Developers#518)

[sbernard31]

@bitvijays, @azizalaliq8 I'm not sure, should we close this issue ? 🤔

[bitvijays]

Thanks @sbernard31 :) Yes. this could be closed.

[sbernard31]

You're welcome :)