Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

examples/gnrc_border_router: add example for WiFi ⇆ esp_now border router #13545

Merged
merged 2 commits into from
Mar 19, 2020

Conversation

benpicco
Copy link
Contributor

@benpicco benpicco commented Mar 3, 2020

Contribution description

This adds an example configuration to use the gnrc_border_router to route
between a regular WiFi network and an esp_now network.

All you need is an esp based board (esp8266, esp32), an IPv6 uplink and a router that supports DHCPv6. Any OpenWRT router or Fritz!Box will do, make sure you have the relavant config opion enabled:
Fritz!Box

Testing procedure

Configure the access data for your WiFi network in Makefile.esp.conf, then run

make -C examples/gnrc_border_router BOARD=esp32-wroom-32 UPLINK=wifi PORT=/dev/ttyUSB0 flash term 

to turn a e.g. esp32-wroom-32 into a boarder router.

You can use other esp based boards as nodes in the network by flashing
e.g. the gnrc_networking example.
Be sure to add CFLAGS += -DGNRC_IPV6_NIB_CONF_SLAAC=1 as otherwise
your esp_now node will not receive a link-local IP address.

CFLAGS="-DGNRC_IPV6_NIB_CONF_SLAAC=1" make -C examples/gnrc_networking BOARD=esp8266-esp-12x PORT=/dev/ttyUSB1 flash term

It may take a couple of minutes before the border router has a global address on both interfaces, but then you should be able to ping global addresses from the esp_now nodes.

esp32-wroom-32

2020-03-03 23:55:59,520 # ifconfig
2020-03-03 23:55:59,526 # Iface  9  HWaddr: 3C:71:BF:9E:13:FC  Link: up 
2020-03-03 23:55:59,530 #           L2-PDU:1500 MTU:1492  HL:255  RTR  
2020-03-03 23:55:59,533 #           RTR_ADV  Source address length: 6
2020-03-03 23:55:59,536 #           Link type: wireless
2020-03-03 23:55:59,542 #           inet6 addr: fe80::3e71:bfff:fe9e:13fc  scope: link  VAL
2020-03-03 23:55:59,549 #           inet6 addr: 2001:16b8:45c7:d600:3e71:bfff:fe9e:13fc  scope: global  VAL
2020-03-03 23:55:59,552 #           inet6 group: ff02::2
2020-03-03 23:55:59,554 #           inet6 group: ff02::1
2020-03-03 23:55:59,558 #           inet6 group: ff02::1:ff9e:13fc
2020-03-03 23:55:59,559 #           
2020-03-03 23:55:59,562 # Iface  8  HWaddr: 3C:71:BF:9E:13:FD 
2020-03-03 23:55:59,566 #           L2-PDU:249 MTU:1280  HL:64  RTR  
2020-03-03 23:55:59,570 #           RTR_ADV  6LO  Source address length: 6
2020-03-03 23:55:59,573 #           Link type: wireless
2020-03-03 23:55:59,579 #           inet6 addr: fe80::3e71:bfff:fe9e:13fd  scope: link  VAL
2020-03-03 23:55:59,586 #           inet6 addr: 2001:16b8:45c7:d6f4:3e71:bfff:fe9e:13fd  scope: global  VAL
2020-03-03 23:55:59,588 #           inet6 group: ff02::2
2020-03-03 23:55:59,591 #           inet6 group: ff02::1
2020-03-03 23:55:59,595 #           inet6 group: ff02::1:ff9e:13fd

esp8266-esp-12x

2020-03-03 23:56:13,765 # Iface  11  HWaddr: 82:7D:3A:7F:E0:1D 
2020-03-03 23:56:13,768 #           L2-PDU:249 MTU:1280  HL:64  RTR  
2020-03-03 23:56:13,773 #           6LO  Source address length: 6
2020-03-03 23:56:13,776 #           Link type: wireless
2020-03-03 23:56:13,782 #           inet6 addr: fe80::807d:3aff:fe7f:e01d  scope: link  VAL
2020-03-03 23:56:13,788 #           inet6 addr: 2001:16b8:45c7:d6f4:807d:3aff:fe7f:e01d  scope: global  VAL
2020-03-03 23:56:13,790 #           inet6 group: ff02::2
2020-03-03 23:56:13,793 #           inet6 group: ff02::1
2020-03-03 23:56:13,796 #           inet6 group: ff02::1:ff7f:e01d
2020-03-03 23:56:13,799 #           inet6 group: ff02::1a
2020-03-03 23:56:13,801 #           
2020-03-03 23:56:13,804 #           Statistics for Layer 2
2020-03-03 23:56:13,807 #             RX packets 0  bytes 0
2020-03-03 23:56:13,810 #             TX packets 0 (Multicast: 0)  bytes 66
2020-03-03 23:56:13,815 #             TX succeeded 1 errors 0
2020-03-03 23:56:13,816 #           Statistics for IPv6
2020-03-03 23:56:13,820 #             RX packets 1  bytes 120
2020-03-03 23:56:13,824 #             TX packets 5 (Multicast: 5)  bytes 288
2020-03-03 23:56:13,828 #             TX succeeded 5 errors 0
2020-03-03 23:56:13,829 # 
> ping6 2600::
2020-03-03 23:56:17,666 #  ping6 2600::
2020-03-03 23:56:17,721 # scandone
2020-03-03 23:56:18,886 # 12 bytes from 2600::: icmp_seq=1 ttl=49 time=213.690 ms
2020-03-03 23:56:19,913 # 12 bytes from 2600::: icmp_seq=2 ttl=49 time=240.123 ms
2020-03-03 23:56:20,667 # 
2020-03-03 23:56:20,668 # --- 2600:: PING statistics ---
2020-03-03 23:56:20,673 # 3 packets transmitted, 2 packets received, 33% packet loss
2020-03-03 23:56:20,679 # round-trip min/avg/max = 213.690/226.906/240.123 ms

Issues/PRs references

The border router is not very reliable. After a short while, routing may stop working, but it may also start working agaim.

2020-03-03 23:58:27,417 # ping6 2600::
2020-03-03 23:58:27,744 # scandone
2020-03-03 23:58:30,418 # 
2020-03-03 23:58:30,419 # --- 2600:: PING statistics ---
2020-03-03 23:58:30,426 # 3 packets transmitted, 0 packets received, 100% packet loss
> 2020-03-03 23:58:37,746 #  scandone
2020-03-03 23:58:59,657 # ping6 2600::
2020-03-03 23:58:59,862 # 12 bytes from 2600::: icmp_seq=0 ttl=49 time=199.205 ms
2020-03-03 23:59:00,884 # 12 bytes from 2600::: icmp_seq=1 ttl=49 time=220.856 ms
2020-03-03 23:59:01,907 # 12 bytes from 2600::: icmp_seq=2 ttl=49 time=244.704 ms
2020-03-03 23:59:01,907 # 
2020-03-03 23:59:01,910 # --- 2600:: PING statistics ---
2020-03-03 23:59:01,915 # 3 packets transmitted, 3 packets received, 0% packet loss
2020-03-03 23:59:01,920 # round-trip min/avg/max = 199.205/221.588/244.704 m

There is also the occasional

2020-03-04 00:01:25,265 # gnrc_netif: possibly lost interrupt.
2020-03-04 00:01:25,268 # gnrc_netif: possibly lost interrupt.
2020-03-04 00:01:25,272 # gnrc_netif: possibly lost interrupt.
2020-03-04 00:01:25,275 # gnrc_netif: possibly lost interrupt.

on the border router or

2020-03-04 00:05:35,861 # LmacRxBlk:0
2020-03-04 00:05:36,862 # LmacRxBlk:0
2020-03-04 00:05:37,863 # LmacRxBlk:0

on the node.

@benpicco benpicco added Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation Area: examples Area: Example Applications labels Mar 3, 2020
@benpicco benpicco requested review from miri64 and gschorcht March 3, 2020 22:58
@benpicco benpicco changed the title examples/gnrc_border_router: add example for WiFi <-> esp_now border router examples/gnrc_border_router: add example for WiFi ⇆ esp_now border router Mar 3, 2020
examples/gnrc_border_router/Makefile Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile.board.dep Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile.esp.conf Outdated Show resolved Hide resolved
cpu/esp_common/Makefile.features Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile.board.dep Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile.esp.conf Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile.esp.conf Outdated Show resolved Hide resolved
@gschorcht
Copy link
Contributor

@benpicco I'm pretty busy today, I will take a look on it tomorrow. First, I have to understand exactly what the intention of the example is and need some time to remember again on the disussions with @miri64 about ESP-NOW and the 6Lo node handling.

So far my understanding of a border router (@miri64 please correct me) was that the inner network behind the border router is not part of the prefix of the outer network. Rather, the border router provides the inner network with its own prefix according to the address configuration at its inner netif (ESP-NOW in our case). In our examples we therefore use 2001:db8::/64 for the inner network. In a network with 6Lo nodes according to RFC 6775, the multihop prefix is disseminated using the Neighbour Discovery. In networks where only 6Lo protocol is used such as ESP-NOW, RPL/DODAG is used for prefix dissemination.

If you are using DHCPv6, the nodes are given the outer network prefix and are therefore part of the outer network. In this case, you probably are not using RPL. If so, the nodes that are out of range of the border router will not get a prefix.

@miri64
Copy link
Member

miri64 commented Mar 4, 2020

In general, not quite sure what you mean with outer or inner network here.

So far my understanding of a border router (@miri64 please correct me) was that the inner network behind the border router is not part of the prefix of the outer network. Rather, the border router provides the inner network with its own prefix according to the address configuration at its inner netif (ESP-NOW in our case). In our examples we therefore use 2001:db8::/64 for the inner network. In a network with 6Lo nodes according to RFC 6775, the multihop prefix is disseminated using the Neighbour Discovery. In networks where only 6Lo protocol is used such as ESP-NOW, RPL/DODAG is used for prefix dissemination.

Mainly, the border router is conceptionally just a router between one link-layer technology and another. With RFC 6775 it has a special position in the network (it is basically administering the network). As it is a router to a new subnet it has to has a (subnet-)prefix to route traffic to. That prefix is the prefix we configure either with DHCPv6 or UHCP (which is on top of the border router functionality, but not part of it).

If you are using DHCPv6, the nodes are given the outer network prefix and are therefore part of the outer network. In this case, you probably are not using RPL. If so, the nodes that are out of range of the border router will not get a prefix.

DHCPv6 provides the subnet-prefix for the new subnet constructed by the LoWPAN (i.e. the prefix that RPL uses). It uses prefix delegation for that (which is a distinct operation than from the well-known operation of DHCP(v6) of address assignment).

@miri64
Copy link
Member

miri64 commented Mar 4, 2020

DHCPv6 is not part of this PRs discussion, but just to clarify in graphic. From the perspective of DHCPv6 client the border router is a box with two interfaces:

     | [upstream]
+----o----+
|         |
+----o----+
     | [LoWPAN]

For the LoWPAN interface (identified by a client-unique identifier i) it requests a prefix delegation from a DHCPv6 servre beyond the upstream interface.

     | ▲ "I can_ haz prefix for i?"
     | [upstream]
+----o----+
|         |
+----o----+
     | [LoWPAN]

The server replies with a tuple (i, prefix)

     | ▼ "You can use 2001:db8:1:1::/64" for i"
     | [upstream]
+----o----+
|         |
+----o----+
     | [LoWPAN]

That allows the client to assign an address for that prefix to the LoWPAN interface

     | [upstream]
+----o----+
|         |
+----o----+
     | [LoWPAN, inet6 addr: 2001:db8:1:1::affe]

and the border router (and the downstream routers) to dessiminate this prefix via its router advertisements through-out the LoWPAN.

@gschorcht
Copy link
Contributor

@miri64 Thanks for the detailed explaination.

DHCPv6 provides the subnet-prefix for the new subnet constructed by the LoWPAN (i.e. the prefix that RPL uses). It uses prefix delegation for that (which is a distinct operation than from the well-known operation of DHCP(v6) of address assignment).

Just to be sure, although we use DHCPv6, RPL is used for the inner network, and even nodes that are out of range of the border router get the prefix ?

@miri64
Copy link
Member

miri64 commented Mar 4, 2020

Just to be sure, although we use DHCPv6, RPL is used for the inner network, and even nodes that are out of range of the border router get the prefix ?

RPL and (or only if RPL is not used) the multihop prefix dissemination of RFC 6775, yes. At least with our current feature-set DHCPv6 is only used to provide the border router (= RPL root) with a prefix it can use to bootstrap the rest of the network.

Iff we ever implement clients that support address assignment via DHCPv6 (this is totally possible under the given specification, even recommended when IEEE 802.15.4 short addresses are used), we also would need DHCPv6 relay agent (or server) support for the border router and routers.

@gschorcht
Copy link
Contributor

gschorcht commented Mar 5, 2020

Be sure to add CFLAGS += -DGNRC_IPV6_NIB_CONF_SLAAC=1 as otherwise
your esp_now node will not receive a link-local IP address.

It's already for a long time on my ToDo list to activate this flag automatically for esp_now netdev 😟

@gschorcht
Copy link
Contributor

Command to compile the border router has to be

- make -C examples/gnrc_border_router BOARD=esp32-wroom-32 USE_ETHOS=0 USE_ESP=1
+ make -C examples/gnrc_border_router BOARD=esp32-wroom-32 USE_ETHOS=0 USE_WIFI=1

Changed 😉

examples/gnrc_border_router/Makefile.wifi.conf Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile.wifi.conf Outdated Show resolved Hide resolved
@gschorcht
Copy link
Contributor

At least with our current feature-set DHCPv6 is only used to provide the border router (= RPL root) with a prefix

@miri64 OK, so one last question remains: Do we need the module gnrc_rpl in our ESP-BR to make the ESP-BR an RPL root? Or is it already sufficient that the ESP-NOW nodes are router nodes with RPL enabled for the prefix dessimination over the ESP-NOW network with RPL?

@benpicco
Copy link
Contributor Author

benpicco commented Mar 5, 2020

@miri64 GNRC_IPV6_NIB_CONF_SLAAC gets disabled for esp_wifi esp_now because the gnrc_ipv6_nib_6lr module is used, which is selected by gnrc_sixlowpan_router_default.
Looks like it only gets enabled if the gnrc_ipv6_nib_6lbr (via gnrc_sixlowpan_border_router_default) is used - so how does it get enabled for 802.15.4 radios by default?

@gschorcht
Copy link
Contributor

@benpicco I can't test it with DHCPv6 yet, my Telekom Speedport Pro does not provide any configuration options for IPv6 for the LAN.
I have another rooted and already modified Telekom Speedport Hybrid router, but it is not yet hacked for DHCPv6 and prefix delegation. Since I have to update it anyway, I will try to do that in the next days.

@gschorcht
Copy link
Contributor

@miri64 GNRC_IPV6_NIB_CONF_SLAAC gets disabled for esp_wifi because the gnrc_ipv6_nib_6lr module is used, which is selected by gnrc_sixlowpan_router_default.

@benpicco I guess you meant esp_now instead of esp_wifi, right?

@gschorcht
Copy link
Contributor

gschorcht commented Mar 5, 2020

As I remember, before PR #10499 it worked out of the box for ESP-NOW nodes. There was no distinction between nodes that are "real" 6Lo nodes which support 6Lo-ND according to RFC 6775 and nodes that only use 6Lo for IPv6 communication, such as ESP-NOW.

Nodes that only use the 6Lo but do not support 6Lo-ND have to use SLAAC to get a valid link local address. That's why GNRC_IPV6_NIB_CONF_SLAAC has to be enabled for ESP-NOW nodes. They use 6Lo but do not support 6Lo-ND according to RFC 6775.

I suppose that there shouldn't be a problem to enable GNRC_IPV6_NIB_CONF_SLAAC in case of esp_now by default as long as it is not used in a border router.

@benpicco
Copy link
Contributor Author

benpicco commented Mar 5, 2020

Looks like the fix might be as simple as #12683

@miri64
Copy link
Member

miri64 commented Mar 5, 2020

@miri64 OK, so one last question remains: Do we need the module gnrc_rpl in our ESP-BR to make the ESP-BR an RPL root? Or is it already sufficient that the ESP-NOW nodes are router nodes with RPL enabled for the prefix dessimination over the ESP-NOW network with RPL?

You need gnrc_rpl to provide a RPL root.

@miri64 GNRC_IPV6_NIB_CONF_SLAAC gets disabled for esp_wifi esp_now because the gnrc_ipv6_nib_6lr module is used, which is selected by gnrc_sixlowpan_router_default.
Looks like it only gets enabled if the gnrc_ipv6_nib_6lbr (via gnrc_sixlowpan_border_router_default) is used - so how does it get enabled for 802.15.4 radios by default?

It is not, if I understood your question correctly... If SLAAC is enabled or not is decided in the NIB's config header for the most part.

@gschorcht
Copy link
Contributor

You need gnrc_rpl to provide a RPL root.

@miri64 Thanks. Now my picture of the world is all right again.

@gschorcht
Copy link
Contributor

gschorcht commented Mar 5, 2020

You need gnrc_rpl to provide a RPL root.

@benpicco We should think about enabling gnrc_rpl by default if esp_now is used as the inner interface. Otherwise, unlike in a 6LoWPAN, the prefix will not be propagated to nodes that are out of range of the border router.

@miri64
Copy link
Member

miri64 commented Mar 5, 2020

[…] Otherwise, unlike in a 6LoWPAN, the prefix will not be propagated to nodes that are out of range of the border router.

If they are configured as advertising routers, prefixes should be advertised in the Router Advertisement's PIO.

@gschorcht
Copy link
Contributor

[…] Otherwise, unlike in a 6LoWPAN, the prefix will not be propagated to nodes that are out of range of the border router.

If they are configured as advertising routers, prefixes should be advertised in the Router Advertisement's PIO.

No, they aren't:

> ifconfig
Iface  11  HWaddr: 5E:CF:7F:80:3F:08 
          L2-PDU:249 MTU:1280  HL:64  RTR  
          6LO  Source address length: 6
          Link type: wireless

@benpicco
Copy link
Contributor Author

benpicco commented Mar 7, 2020

That Readme hasn't been properly updated in a while…
wrt the Speedport, I just experienced that myself too and it's so frustrating. IA_PD would be such a good feature just to cleanly set up multiple APs in an IPv6 world, it's sad this can not be expected in such a widely deployed (and brand new!) device.

examples/gnrc_border_router/README.md Outdated Show resolved Hide resolved
@gschorcht
Copy link
Contributor

@benpicco Finally, I could test it. However, I could not get prefix delegation with my Speedport Hybrid. Therefore, I configured a DHCPv6 server on a separate Linux machine which delegates a sub-prefix of the Speedport Hybrid. After setting the route on the Speedport Hybrid by hand, the ESP-NOW node could ping GUAs.

So the PR is good from my point of view.

@gschorcht gschorcht added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Mar 19, 2020
Copy link
Contributor

@gschorcht gschorcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@gschorcht
Copy link
Contributor

@miri64 Is the PR OK from your side?

Copy link
Member

@miri64 miri64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor implementation proposal (I see no reason to enforce DHCPv6 via WiFi). Other than that, the doc update needs some fixing

examples/gnrc_border_router/Makefile Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile Outdated Show resolved Hide resolved
examples/gnrc_border_router/Makefile Show resolved Hide resolved
examples/gnrc_border_router/README.md Outdated Show resolved Hide resolved
examples/gnrc_border_router/README.md Outdated Show resolved Hide resolved
examples/gnrc_border_router/README.md Outdated Show resolved Hide resolved
examples/gnrc_border_router/README.md Outdated Show resolved Hide resolved
examples/gnrc_border_router/README.md Outdated Show resolved Hide resolved
examples/gnrc_border_router/README.md Outdated Show resolved Hide resolved
examples/gnrc_border_router/README.md Outdated Show resolved Hide resolved
@miri64
Copy link
Member

miri64 commented Mar 19, 2020

Can't reply to the comment directly, so answering here

Is there even a way to use UHCP with WiFi?

Nothing prevents you from running UHCP on the upstream router / WiFi AP.

@miri64
Copy link
Member

miri64 commented Mar 19, 2020

Well esp_now is always available without the need to connect anything, so I think it's worth mentioning.
I've reworded it a bit though.

I still think mentioning esp_now here is worth it. You are introducing a very special case into a doc of an application that is supposed to be a very broad example. Yes, esp_wifi is the only WiFi we support at the moment, but that can change. So why add a bunch of ESP-specific doc there [edit]that has little relevance for the border router example[/edit]?

@benpicco
Copy link
Contributor Author

If we have other WiFi options we can come back and update the documentation.
Right now it's the default and I think many users are not even aware of esp_now even though it's probably the easiest & cheapest option to build a 6LoWPAN network. So I think it's worth adding a sentence about it.

@miri64
Copy link
Member

miri64 commented Mar 19, 2020

That said, it is good to mention that only esp32 and esp8466 platforms are currently supported for UPLINK=wifi. The rest is just stating something, that is as true for the gnrc_networking example or any other networking application on Esperessif-based platforms and becomes irrelevant as soon as there is another WiFi driver.

@miri64
Copy link
Member

miri64 commented Mar 19, 2020

and I think many users are not even aware of esp_now even though it's probably the easiest & cheapest option to build a 6LoWPAN network. So I think it's worth adding a sentence about it.

I just don't think the border router example is the right place for that.

Copy link
Member

@miri64 miri64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not thinking mentioning esp_now specifically in the doc is a good idea, but I don't keep grudges...

credentials. You can alternatively edit the `Makefile`.

Currently, `wifi` requires an esp8266 or esp32 for the border router and will default
to using `esp_now` for the downstream interface.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You seem insistent to keep esp_now in... Fine, I'm not fighting about minor details such as this...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean it is the default, so that should be documented.

This adds an example configuration to use the gnrc_border_router to route
between a regular WiFi network and an esp_now network.

All you need is an esp based board (esp8266, esp32).

Configure the access data for your WiFi network in `Makefile.esp.conf`,
then run

    make BOARD=esp32-wroom-32 USE_ETHOS=0 USE_ESP=1 flash term

to turn a e.g. `esp32-wroom-32` into a boarder router.

You can use other esp based boards as nodes in the network by flashing
e.g. the `gnrc_networking` example.
Be sure to add `CFLAGS += -DGNRC_IPV6_NIB_CONF_SLAAC=1` as otherwise
your esp_now node will not receive a link-local IP address.
@miri64
Copy link
Member

miri64 commented Mar 19, 2020

Please squash.

@miri64
Copy link
Member

miri64 commented Mar 19, 2020

(and fix the errors in the static tests, please)

@miri64
Copy link
Member

miri64 commented Mar 19, 2020

For those who squash too fast ;-) https://travis-ci.org/github/RIOT-OS/RIOT/builds/664597333#L163-L165

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: examples Area: Example Applications CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants