Support wired network interfaces (W5500, W5100, ENC28J60) #1703

earlephilhower · 2023-09-09T19:40:55Z

Allows wired networking support for all the WiFi-related classes (WebServer, WiFiClient, WiFiServer, etc.)

Supports WizNet W5500, WizNet 5100, and ENC28J60 modules.

Fixes #775

WIP - Builds and can get a DHCP address using W5500. Lots of infrastructure work still required: [ ] Separate CYW43 stuff and LWIP stuff (mutex, etc.) [ ] Add async worker for pumping frames [ ] Real LWIP-only mutex [ ] Should WiFiClient be available as TCPClient? etc. Fixes #775

earlephilhower · 2023-09-10T20:52:59Z

AdvancedWebServer example now running on the W5500!

earlephilhower · 2023-09-10T21:36:56Z

@khoih-prog if you could give the 5100 and/or the ENC adapters a test I would appreciate it. Simply replace the header of the example in the lwip_w5500 example with below (and any SPI pinout adjustment):

#include <ENC28J60lwIP.h>

ENC28J60lwIP eth(/*SS*/ 1);  // <== adapt to your hardware

I'll add in some #ifs to the WebServer examples to allow swapping between WiFi and W5500 (since that's all I have on hand).

Removes WiFiClass code from pure LWIPEthernet sketches.

Make sure to remove any handlers we used to have from our ethernet processing loops.

ndavi · 2023-09-13T05:34:55Z

Hello ! Thanks you for doing this feature ! I'm trying to implements the Lwip stack with this library, https://github.com/hideakitai/ArtNet, by using the WifiUDP implementation, but I have a strange error compilation.
I'm using a W5500

/Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: /Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: DWARF error: can't find .debug_ranges section.

Full error here :

Linking .pio/build/pico/firmware.elf
/Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: /Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: DWARF error: can't find .debug_ranges section.
/Users/nico/.platformio/packages/framework-arduinopico/lib/libpicow-noipv6-nobtc-noble.a(cyw43_ctrl.c.obj): in function cyw43_ensure_up': cyw43_ctrl.c:(.text.cyw43_ensure_up+0x18): undefined reference to __wrap_cyw43_cb_tcpip_deinit'
/Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: cyw43_ctrl.c:(.text.cyw43_ensure_up+0x20): undefined reference to __wrap_cyw43_cb_tcpip_deinit' /Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: /Users/nico/.platformio/packages/framework-arduinopico/lib/libpicow-noipv6-nobtc-noble.a(cyw43_ctrl.c.obj): in function cyw43_cb_process_async_event':
cyw43_ctrl.c:(.text.cyw43_cb_process_async_event+0x8a): undefined reference to __wrap_cyw43_cb_tcpip_set_link_up' /Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: cyw43_ctrl.c:(.text.cyw43_cb_process_async_event+0xe6): undefined reference to __wrap_cyw43_cb_tcpip_set_link_down'
/Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: cyw43_ctrl.c:(.text.cyw43_cb_process_async_event+0x166): undefined reference to __wrap_cyw43_cb_tcpip_set_link_up' /Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: cyw43_ctrl.c:(.text.cyw43_cb_process_async_event+0x16e): undefined reference to __wrap_cyw43_cb_tcpip_set_link_down'
/Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: /Users/nico/.platformio/packages/framework-arduinopico/lib/libpicow-noipv6-nobtc-noble.a(cyw43_ctrl.c.obj): in function cyw43_wifi_set_up': cyw43_ctrl.c:(.text.cyw43_wifi_set_up+0x84): undefined reference to __wrap_cyw43_cb_tcpip_deinit'
/Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: cyw43_ctrl.c:(.text.cyw43_wifi_set_up+0x8c): undefined reference to __wrap_cyw43_cb_tcpip_init' /Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: /Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: DWARF error: can't find .debug_ranges section. /Users/nico/.platformio/packages/framework-arduinopico/lib/libpicow-noipv6-nobtc-noble.a(cyw43_ll.c.obj): in function cyw43_ll_process_packets':
cyw43_ll.c:(.text.cyw43_ll_process_packets+0x3c): undefined reference to __wrap_cyw43_cb_process_ethernet' /Users/nico/.platformio/packages/toolchain-rp2040-earlephilhower/bin/../lib/gcc/arm-none-eabi/10.3.0/../../../../arm-none-eabi/bin/ld: /Users/nico/.platformio/packages/framework-arduinopico/lib/libpicow-noipv6-nobtc-noble.a(cyw43_ll.c.obj): in function cyw43_do_ioctl':
cyw43_ll.c:(.text.cyw43_do_ioctl+0xba): undefined reference to `__wrap_cyw43_cb_process_ethernet'
collect2: error: ld returned 1 exit status
*** [.pio/build/pico/firmware.elf] Error 1

Any idea ? Thanks !

earlephilhower · 2023-09-13T07:06:19Z

I suggest trying the w5500 example first. That is built by the CI and is tested to work by me. The UDP example was also run by me today (after adjusting to use ETH and not WiFi).

The logs look like youree trying to build for the PicoW or calling WiFi.xxx but without an MCVE all that can be done is guess. You should be building for the stock Pico or your specific board.

ndavi · 2023-09-13T12:26:53Z

You are right, it's seems the library do some strange imports for the Pico W, so it's not working. Also, for now, the lwip stack seems to not be the best solution for my problem, because I don't see where I can change some settings that I can change in the Ethernet library (Like the ETHERNET_LARGE_BUFFERS).
I make my example work with the lwip stack, but it's too slow for my usage

I will try to keep to the Ethernet library for now. thanks for your answer.

earlephilhower · 2023-09-13T15:59:26Z

Your call. We're using the Ethernet adapters as dumb NICs just like the Wi-Fi chip, so things like ETHERNET LARGE BUFFERS don't make sense. The main Pico memory is being used, not the onboard adapter sram.

To make the library work proper you'd probably just need to tweak the Pico W port not to use Wifi. and instead use Eth.

The DNS lookup in LWIP doesn't specify a netif, so a single call is all that's needed. No need to keep a list of DNS lookup callbacks.

ndavi · 2023-09-13T23:04:03Z

Your call. We're using the Ethernet adapters as dumb NICs just like the Wi-Fi chip, so things like ETHERNET LARGE BUFFERS don't make sense. The main Pico memory is being used, not the onboard adapter sram.

To make the library work proper you'd probably just need to tweak the Pico W port not to use Wifi. and instead use Eth.

Unfortunately, it's half working for me. I've done like you said, tweaking the library to use "WifiUdp" class instead of "EthernetUdp" class, and everything is fine, I can receive the data and send data using the lwip stack, like I want, but it's too slow for me. I send really a lot of packets in a short amount of time, and I need no latency. And everything is really slow in comparaison to the Ethernet library, like just the ping function.

Example : this is the result of a ping with the lwip stack. No data is send to the pico, I'm just binding my udp port with the WifiUdp class

64 bytes from 2.0.0.1: icmp_seq=84 ttl=255 time=23.971 ms
64 bytes from 2.0.0.1: icmp_seq=85 ttl=255 time=22.180 ms
64 bytes from 2.0.0.1: icmp_seq=86 ttl=255 time=22.943 ms
64 bytes from 2.0.0.1: icmp_seq=87 ttl=255 time=22.887 ms

This is the result of a ping with the Ethernet stack, same case, binding the udp port using the EthernetUdp library, same setup

64 bytes from 2.0.0.1: icmp_seq=13 ttl=128 time=0.792 ms
64 bytes from 2.0.0.1: icmp_seq=14 ttl=128 time=0.322 ms
64 bytes from 2.0.0.1: icmp_seq=15 ttl=128 time=0.599 ms
64 bytes from 2.0.0.1: icmp_seq=16 ttl=128 time=0.821 ms

To setup the stack, I'm using the code you provided in the WiFiClient-W5500 example

Do you have the same ping result as me using the lwip stack ?

Thank you

earlephilhower · 2023-09-13T23:58:44Z

As Knuth said, "Premature optimization is the root of all evil" 😆 I've been focused on functionality and stability and not performance so much. That and not breaking WiFi (since all share the same backbone now).

I have a feeling increasing the SPI speed would improve things a lot. I just used whatever the default is for the examples and did not try and find a datasheet to figure out the W5500's max speed.

I'm also doing a check of the W5500 at 50ms intervals which could easily be made to 10ms or 5ms in LWIPEthernet:138. That also pumps LWIP, meaning you might have an average of 20-25ms between receiving something in the ethernet module and seeing it in the Pico. I suggest changing that ..., 50 to ..., 10 if you have a chance and seeing if it helps things.

Also, just generally WRT to pings, is that handled inside the W5500 itself and not the Ethernet library? That might be the case...we use the dumb NIC model since we already run LWIP here so a ping needs to get read in from the outside world (at 50ms heartbeat) and then responded to (which would happen almost immediately once it was read it, but you'd still wait that ~50% of a period on average).

earlephilhower · 2023-09-14T00:07:32Z

Yup, we're running default Arduino SPI clk of 4MHZ in this driver. The W5500 datasheet says this about SCLK (33MHz):

Even though theoretical design speed is 80MHz, the signal in the high speed may be distorted because
of the circuit crosstalk and the length of the signal line. The minimum guaranteed speed of the SCLK
is 33.3 MHz which was tested and measured with the stable waveform

ndavi · 2023-09-14T01:03:33Z

Ok I understand ! I will take a try. I think you have the point, the SPI speed is too slow for my usage at this time.

For the ping function, I don't really understand how it's works in the W5500, so I can't tell.

Do you know where I can change it ? it's seems that in the SPIClassRP2040 the clock divider function is deprecated, I really have no idea how in this driver or in arduino in general we can set the SPI clock, so maybe I don't have the point

Thanks !

earlephilhower · 2023-09-14T01:45:16Z

Changing to a 5ms heartbeat for LWIP:

earle@amd:~/Arduino/hardware/pico/rp2040/libraries/lwIP_Ethernet/src$ ping 192.168.1.245
PING 192.168.1.245 (192.168.1.245) 56(84) bytes of data.
64 bytes from 192.168.1.245: icmp_seq=1 ttl=255 time=1.91 ms
64 bytes from 192.168.1.245: icmp_seq=2 ttl=255 time=4.24 ms
64 bytes from 192.168.1.245: icmp_seq=3 ttl=255 time=1.81 ms
64 bytes from 192.168.1.245: icmp_seq=4 ttl=255 time=3.47 ms
64 bytes from 192.168.1.245: icmp_seq=5 ttl=255 time=3.89 ms
64 bytes from 192.168.1.245: icmp_seq=6 ttl=255 time=3.59 ms

Increasing SPI speed to 30MHZ

voids setup() {
 SPI.setRX(0);
  SPI.setCS(1);
  SPI.setSCK(2);
  SPI.setTX(3);
  SPI.begin();
  SPI.beginTransaction(SPISettings(30000000, MSBFIRST, SPI_MODE0));
  SPI.endTransaction();
...

not much of a change

earle@amd:~/Arduino/hardware/pico/rp2040/libraries/lwIP_Ethernet/src$ ping 192.168.1.245
PING 192.168.1.245 (192.168.1.245) 56(84) bytes of data.
64 bytes from 192.168.1.245: icmp_seq=1 ttl=255 time=3.86 ms
64 bytes from 192.168.1.245: icmp_seq=2 ttl=255 time=2.80 ms
64 bytes from 192.168.1.245: icmp_seq=3 ttl=255 time=4.15 ms
64 bytes from 192.168.1.245: icmp_seq=4 ttl=255 time=1.36 ms
64 bytes from 192.168.1.245: icmp_seq=5 ttl=255 time=4.66 ms
64 bytes from 192.168.1.245: icmp_seq=6 ttl=255 time=5.79 ms
64 bytes from 192.168.1.245: icmp_seq=7 ttl=255 time=4.21 ms
64 bytes from 192.168.1.245: icmp_seq=8 ttl=255 time=5.64 ms
64 bytes from 192.168.1.245: icmp_seq=9 ttl=255 time=4.65 ms
64 bytes from 192.168.1.245: icmp_seq=10 ttl=255 time=5.83 ms

Making LWIP run 1K/sec (1ms):

PING 192.168.1.245 (192.168.1.245) 56(84) bytes of data.
64 bytes from 192.168.1.245: icmp_seq=1 ttl=255 time=2.59 ms
64 bytes from 192.168.1.245: icmp_seq=2 ttl=255 time=1.41 ms
64 bytes from 192.168.1.245: icmp_seq=3 ttl=255 time=1.64 ms
64 bytes from 192.168.1.245: icmp_seq=4 ttl=255 time=1.21 ms
64 bytes from 192.168.1.245: icmp_seq=5 ttl=255 time=1.16 ms
64 bytes from 192.168.1.245: icmp_seq=6 ttl=255 time=1.64 ms
64 bytes from 192.168.1.245: icmp_seq=7 ttl=255 time=1.66 ms
64 bytes from 192.168.1.245: icmp_seq=8 ttl=255 time=1.64 ms
64 bytes from 192.168.1.245: icmp_seq=9 ttl=255 time=1.88 ms

Probably can make the SPI speed a per-driver option, and the LWIP cycle a per-system one.

earlephilhower · 2023-09-14T01:47:01Z

And for crazy fast, 1ms delays, 30MHZ SPI, and overclocked Pico:

PING 192.168.1.245 (192.168.1.245) 56(84) bytes of data.
64 bytes from 192.168.1.245: icmp_seq=1 ttl=255 time=1.78 ms
64 bytes from 192.168.1.245: icmp_seq=2 ttl=255 time=0.791 ms
64 bytes from 192.168.1.245: icmp_seq=3 ttl=255 time=0.853 ms
64 bytes from 192.168.1.245: icmp_seq=4 ttl=255 time=1.22 ms
64 bytes from 192.168.1.245: icmp_seq=5 ttl=255 time=0.764 ms
64 bytes from 192.168.1.245: icmp_seq=6 ttl=255 time=0.991 ms
64 bytes from 192.168.1.245: icmp_seq=7 ttl=255 time=1.56 ms
64 bytes from 192.168.1.245: icmp_seq=8 ttl=255 time=0.747 ms

earlephilhower · 2023-09-14T01:48:32Z

Given those #s, I'd say ping responses are handled directly on the w5500 chip in "intelligent mode immediately on receipt of a ICMP PING packet. So for actual BW/latency for real data I'd suggest using something other than ping...so a bulk transfer of 16K or something and time reception on the PC...

earlephilhower · 2023-09-14T01:53:35Z

Also, the currentW5500 driver does not use the IRQ line. Using the IRQ line would allow for a lot of the lower LWIP latency w/o needing to run a polling operation every 5ms or so. But, again, that just wasn't implemented in the ESP8266 core I'm taking this from AFAIK.

ndavi · 2023-09-14T02:18:34Z

Just tested it, it's better with these settings !

But also, no enough fast for my usage (I'm sending a lot of UDP packets to test it), and the Ethernet library is doing a better job

I control a led panel using these udp packets, so I can really see where it's not working. Like in this video, we can see that some packets are dropped

panel.mp4

I think, maybe it's related to the IRQ line, or maybe, it's because we don't use the onboard adapter sram to do some fancy stuff, so some buffer is overloaded and packets are loose. I don't really understand what's going on here, but, with these settings that we experiments, I'm near the performance that i've got using esp32 and LAN8720.

Supports higher SPI SCK by an app

earlephilhower · 2023-09-14T02:43:29Z

Ah well, too bad. The Ethernet stuff should still work, but you can't do all the neat stuff like OTA/etc. as the native LWIP stack can do.

In any case, much thanks for reporting this. It was trivial to add user configurable options (and give better default timers). Doing so improved the responsiveness of the web server (I can get >30 page refreshes/sec in the AdvancedWebServer at 133MHZ CPU), so 50ms was really a bottleneck I think.

Ing-Dom · 2023-09-15T05:19:23Z

@earlephilhower you are awesome !

earlephilhower · 2023-09-15T23:26:14Z

The WizNet 5100S EVB Pico also tested and working with the following setup

Wiznet5100lwIP eth(17 /* chip select */);
...
  SPI.setRX(16);
  SPI.setCS(17);
  SPI.setSCK(18);
  SPI.setTX(19);

earlephilhower marked this pull request as draft September 9, 2023 19:41

earlephilhower force-pushed the lwipconn branch from 0a3b969 to e1274dc Compare September 9, 2023 19:47

Async context to pump LWIP and non-CYW43 ethernet

29c60ce

earlephilhower force-pushed the lwipconn branch from 7620dee to 29c60ce Compare September 10, 2023 00:30

earlephilhower added 5 commits September 9, 2023 18:36

Add in LWIP locking

7f63214

Migrate to LwipEthernet, support multiple interfaces

c8b9f65

Clean up and refactor

156aba6

Add'l cleanup for CYW43, runs AdvancedWebServer!

d004ce9

CYW43 build fixes

d0a8925

earlephilhower added 5 commits September 10, 2023 13:58

Fix P.IO build

cc8955a

Fix Schroedinger's struct init w/PIO and normal IDE

8fa2dc7

Add (untested) W5100 support

4a68730

Add (untested) ENC28J60 support

174cfec

WiFi library dependency breaking P.IO

49a791e

earlephilhower force-pushed the lwipconn branch from 1aca44c to 49a791e Compare September 10, 2023 21:28

earlephilhower marked this pull request as ready for review September 10, 2023 21:32

earlephilhower changed the title ~~Allow non-CYW43 network interfaces (W5500, etc.)~~ Support wired network interfaces (W5500, W5100, ENC28J60) Sep 10, 2023

earlephilhower added 5 commits September 10, 2023 14:36

Merge branch 'master' into lwipconn

459c412

Add documentation

c5cc8de

Add some examples

51edf72

Clean up headers

4636b50

Use async context lock support

8d98a9d

earlephilhower force-pushed the lwipconn branch from acc3a19 to 8d98a9d Compare September 11, 2023 17:09

earlephilhower added 4 commits September 11, 2023 10:19

Remove unneeded LWIP copied example

099c08a

Style updates, remove dead functions

83dcf6b

Readme and cleanup

1936a48

Avoid race condition while adding interfaces

612c755

earlephilhower added 5 commits September 12, 2023 16:53

Have single, combined DNS lookup process

0210547

Removes WiFiClass code from pure LWIPEthernet sketches.

Clean up, make CYW43 use more generic code

71f2298

One WiFi.hostByName straggler

d93f2c5

Unbreak ClientContext for LWIPEthernet

ddbc2bc

Allow LWIP device removal (::end)

cd02ae5

Make sure to remove any handlers we used to have from our ethernet processing loops.

earlephilhower added 3 commits September 13, 2023 09:53

DNS doesn't need to iterate over interfaces, d'oh

6c78ab6

The DNS lookup in LWIP doesn't specify a netif, so a single call is all that's needed. No need to keep a list of DNS lookup callbacks.

Use the CYW43 lock for Ethernet on the PicoW

f44c813

Merge branch 'master' into lwipconn

dfaa0d4

earlephilhower added 2 commits September 13, 2023 19:21

Add lwipPollingPeriod adjustment and docs

2e47a7e

Add eth.setSPISpeed() and eth.setSPISettings()

21eee36

Supports higher SPI SCK by an app

earlephilhower added 2 commits September 14, 2023 14:59

Use PicoW CYW43 context when present

235351c

Protect Ethernet drivers from re-entrancy

8e6c66b

earlephilhower merged commit 1f3d501 into master Sep 15, 2023
12 checks passed

earlephilhower deleted the lwipconn branch September 15, 2023 02:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support wired network interfaces (W5500, W5100, ENC28J60) #1703

Support wired network interfaces (W5500, W5100, ENC28J60) #1703

earlephilhower commented Sep 9, 2023 •

edited

Loading

earlephilhower commented Sep 10, 2023

earlephilhower commented Sep 10, 2023

ndavi commented Sep 13, 2023 •

edited

Loading

earlephilhower commented Sep 13, 2023 •

edited

Loading

ndavi commented Sep 13, 2023 •

edited

Loading

earlephilhower commented Sep 13, 2023

ndavi commented Sep 13, 2023

earlephilhower commented Sep 13, 2023

earlephilhower commented Sep 14, 2023

ndavi commented Sep 14, 2023

earlephilhower commented Sep 14, 2023

earlephilhower commented Sep 14, 2023

earlephilhower commented Sep 14, 2023

earlephilhower commented Sep 14, 2023

ndavi commented Sep 14, 2023 •

edited

Loading

earlephilhower commented Sep 14, 2023

Ing-Dom commented Sep 15, 2023

earlephilhower commented Sep 15, 2023

Support wired network interfaces (W5500, W5100, ENC28J60) #1703

Support wired network interfaces (W5500, W5100, ENC28J60) #1703

Conversation

earlephilhower commented Sep 9, 2023 • edited Loading

earlephilhower commented Sep 10, 2023

earlephilhower commented Sep 10, 2023

ndavi commented Sep 13, 2023 • edited Loading

earlephilhower commented Sep 13, 2023 • edited Loading

ndavi commented Sep 13, 2023 • edited Loading

earlephilhower commented Sep 13, 2023

ndavi commented Sep 13, 2023

earlephilhower commented Sep 13, 2023

earlephilhower commented Sep 14, 2023

ndavi commented Sep 14, 2023

earlephilhower commented Sep 14, 2023

earlephilhower commented Sep 14, 2023

earlephilhower commented Sep 14, 2023

earlephilhower commented Sep 14, 2023

ndavi commented Sep 14, 2023 • edited Loading

earlephilhower commented Sep 14, 2023

Ing-Dom commented Sep 15, 2023

earlephilhower commented Sep 15, 2023

earlephilhower commented Sep 9, 2023 •

edited

Loading

ndavi commented Sep 13, 2023 •

edited

Loading

earlephilhower commented Sep 13, 2023 •

edited

Loading

ndavi commented Sep 13, 2023 •

edited

Loading

ndavi commented Sep 14, 2023 •

edited

Loading