Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 2022.07 - RC4 #258

Closed
78 of 88 tasks
chrysn opened this issue Aug 24, 2022 · 25 comments
Closed
78 of 88 tasks

Release 2022.07 - RC4 #258

chrysn opened this issue Aug 24, 2022 · 25 comments

Comments

@chrysn
Copy link
Member

chrysn commented Aug 24, 2022

This issue lists the status of all tests for the Release Candidate 4 of the 2022.07 release.

Specs tested:

@chrysn chrysn mentioned this issue Aug 24, 2022
88 tasks
@riot-ci
Copy link

riot-ci commented Aug 24, 2022

Test Report

Task Outcome
01. Task 02 - Unittests on native PASSED
01. Task 03 - Unittests on native separated [tests-analog_util] PASSED
01. Task 03 - Unittests on native separated [tests-base64] PASSED
01. Task 03 - Unittests on native separated [tests-bcd] PASSED
01. Task 03 - Unittests on native separated [tests-bitfield] PASSED
01. Task 03 - Unittests on native separated [tests-bloom] PASSED
01. Task 03 - Unittests on native separated [tests-bluetil] PASSED
01. Task 03 - Unittests on native separated [tests-checksum] PASSED
01. Task 03 - Unittests on native separated [tests-chunked_ringbuffer] PASSED
01. Task 03 - Unittests on native separated [tests-clif] PASSED
01. Task 03 - Unittests on native separated [tests-color] PASSED
01. Task 03 - Unittests on native separated [tests-core] PASSED
01. Task 03 - Unittests on native separated [tests-credman] PASSED
01. Task 03 - Unittests on native separated [tests-div] PASSED
01. Task 03 - Unittests on native separated [tests-dns_cache] PASSED
01. Task 03 - Unittests on native separated [tests-ecc] PASSED
01. Task 03 - Unittests on native separated [tests-fib] PASSED
01. Task 03 - Unittests on native separated [tests-fib_sr] PASSED
01. Task 03 - Unittests on native separated [tests-fmt] PASSED
01. Task 03 - Unittests on native separated [tests-frac] PASSED
01. Task 03 - Unittests on native separated [tests-gcoap] PASSED
01. Task 03 - Unittests on native separated [tests-gnrc_ipv6] PASSED
01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_hdr] PASSED
01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_nib] PASSED
01. Task 03 - Unittests on native separated [tests-gnrc_mac_internal] PASSED
01. Task 03 - Unittests on native separated [tests-gnrc_netif_pktq] PASSED
01. Task 03 - Unittests on native separated [tests-gnrc_sixlowpan_frag_vrb] PASSED
01. Task 03 - Unittests on native separated [tests-gnrc_udp] PASSED
01. Task 03 - Unittests on native separated [tests-hashes] PASSED
01. Task 03 - Unittests on native separated [tests-ieee802154] PASSED
01. Task 03 - Unittests on native separated [tests-inet_csum] PASSED
01. Task 03 - Unittests on native separated [tests-ipv4_addr] PASSED
01. Task 03 - Unittests on native separated [tests-ipv6_addr] PASSED
01. Task 03 - Unittests on native separated [tests-ipv6_hdr] PASSED
01. Task 03 - Unittests on native separated [tests-kernel_defines] PASSED
01. Task 03 - Unittests on native separated [tests-luid] PASSED
01. Task 03 - Unittests on native separated [tests-matstat] PASSED
01. Task 03 - Unittests on native separated [tests-mtd] PASSED
01. Task 03 - Unittests on native separated [tests-nanocoap] PASSED
01. Task 03 - Unittests on native separated [tests-nanocoap_cache] PASSED
01. Task 03 - Unittests on native separated [tests-netopt] PASSED
01. Task 03 - Unittests on native separated [tests-netreg] PASSED
01. Task 03 - Unittests on native separated [tests-phydat] PASSED
01. Task 03 - Unittests on native separated [tests-pkt] PASSED
01. Task 03 - Unittests on native separated [tests-pktbuf] PASSED
01. Task 03 - Unittests on native separated [tests-pktqueue] PASSED
01. Task 03 - Unittests on native separated [tests-printf_float] PASSED
01. Task 03 - Unittests on native separated [tests-priority_pktqueue] PASSED
01. Task 03 - Unittests on native separated [tests-rtc] PASSED
01. Task 03 - Unittests on native separated [tests-rtt_rtc] PASSED
01. Task 03 - Unittests on native separated [tests-saul_reg] PASSED
01. Task 03 - Unittests on native separated [tests-scanf_float] PASSED
01. Task 03 - Unittests on native separated [tests-seq] PASSED
01. Task 03 - Unittests on native separated [tests-sht1x] PASSED
01. Task 03 - Unittests on native separated [tests-sixlowpan] PASSED
01. Task 03 - Unittests on native separated [tests-sixlowpan_ctx] PASSED
01. Task 03 - Unittests on native separated [tests-sixlowpan_sfr] PASSED
01. Task 03 - Unittests on native separated [tests-sock_util] PASSED
01. Task 03 - Unittests on native separated [tests-timex] PASSED
01. Task 03 - Unittests on native separated [tests-tsrb] PASSED
01. Task 03 - Unittests on native separated [tests-turo] PASSED
01. Task 03 - Unittests on native separated [tests-uri_parser] PASSED
01. Task 03 - Unittests on native separated [tests-ut_process] PASSED
01. Task 03 - Unittests on native separated [tests-uuid] PASSED
01. Task 03 - Unittests on native separated [tests-vfs] PASSED
01. Task 03 - Unittests on native separated [tests-zptr] PASSED
01. Task 03 - Unittests on native separated [tests-ztimer64] PASSED
01. Task 03 - Unittests on native separated [tests-ztimer] PASSED
01. Task 04 - Unittests on iotlab-m3 PASSED
03. Task 01 - ICMPv6 multicast echo on native PASSED
03. Task 02 - ICMPv6 link-local echo on native PASSED
03. Task 03 - ICMPv6 link-local echo on native (1 hour) PASSED
03. Task 04 - ICMPv6 stress test on native (1 hour) PASSED
03. Task 05 - ICMPv6 stress test on native (neighbor cache stress) PASSED
03. Task 06 - ICMPv6 link-local echo on native (IPv6 fragmentation) PASSED
04. Task 01 - ICMPv6 link-local echo with iotlab-m3 PASSED
04. Task 02 - ICMPv6 multicast echo with iotlab-m3/samr21-xpro PASSED
04. Task 03 - ICMPv6 echo with large payload PASSED
04. Task 04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes PASSED
🟡 04. Task 05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 06 (Experimental)- ICMPv6 link-local echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee SKIPPED
🟡 04. Task 08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee SKIPPED
04. Task 09 - ICMPv6 stress test on iotlab-m3 PASSED
04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) PASSED
04. Task 11 (Experimental) - ICMPv6 stress test on nrf802154 PASSED
04. Task 12 (Experimental) - ICMPv6 multicast echo with iotlab-m3/nrf802154 PASSED
04. Task 13 (Experimental) - ICMPv6 link-local echo with iotlab-m3/nrf802154 PASSED
05. Task 01 - ICMPv6 echo unicast addresess on native (default route) PASSED
05. Task 02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route) PASSED
05. Task 03 - ICMPv6 echo unicast addresess on native (specific route) PASSED
05. Task 04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route) PASSED
06. Task 01 - UDP on iotlab-m3 PASSED
06. Task 02 - UDP on iotlab-m3 (UDP port compression) PASSED
06. Task 03 - UDP on native (non-existent neighbor) PASSED
06. Task 04 - UDP on iotlab-m3 (non-existent neighbor) PASSED
06. Task 05 - Empty UDP on native PASSED
06. Task 06 - Empty UDP on iotlab-m3 PASSED
07. Task 01 - ICMPv6 echo on iotlab-m3 with three hops (static route) PASSED
07. Task 02 - UDP on iotlab-m3 with three hops (static route) PASSED
07. Task 03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route) PASSED
07. Task 04 - UDP on iotlab-m3 with three hops (RPL route) PASSED
🟡 07. Task 05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route) SKIPPED
08. Task 01 - ICMPv6 echo between native and Linux PASSED
08. Task 08 - UDP between GNRC and lwIP on iotlab-m3 PASSED
09. Task 01 - CORD Endpoint PASSED
09. Task 02 - Confirmable retries [False-0] PASSED
09. Task 02 - Confirmable retries [True-1] PASSED
09. Task 03 - Block1 PASSED
09. Task 04 - Block2 PASSED
09. Task 05 - Observe registration and notification PASSED
10. Task 01 - Destination unreachable - no route to destination PASSED
10. Task 02 - Destination unreachable - Beyond scope of source address PASSED
10. Task 03 - Destination unreachable - address unreachable (target node address) PASSED
10. Task 04 - Destination unreachable - address unreachable (neighbor cache miss) PASSED
10. Task 05 - Destination unreachable - port unreachable PASSED
10. Task 06 - Destination unreachable - port unreachable (large payload) PASSED
10. Task 07 - Packet too big PASSED
10. Task 08 - Time exceeded - hop limit exceeded in transit PASSED
10. Task 09 - Parameter problem - erroneous header field encountered PASSED
10. Task 10 - IPv6-in-IPv6 encapsulation PASSED
11. Task 01 - LoRaWAN example FAILED
11. Task 02 - OTAA join procedure PASSED
11. Task 03 - ABP join procedure PASSED
11. Task 04 - LoRaWAN device parameters persistence PASSED
11. Task 05 - GNRC LoRaWAN (Over The Air Activation) [otaa] PASSED
🟡 11. Task 06 - GNRC LoRaWAN (ABP) [abp] SKIPPED

@chrysn
Copy link
Member Author

chrysn commented Aug 24, 2022

With backports for compiler errors that would show up on the user's side out of the way (RIOT-OS/RIOT#18488, RIOT-OS/RIOT#18485), I think this should be the final one.

RIOT-OS/RIOT#18479 will go into "known issues" with yet to be decided scope.

@jia200x please run the missing LoRA test (ABP is not automated); and IIRC you also have access to the 08-interop tests?

@jia200x
Copy link
Member

jia200x commented Aug 24, 2022

The ABP test does not run on TTN (see RIOT-OS/RIOT#16962).
And sure, I can try to run as many 08-interop tests as possible.

@chrysn
Copy link
Member Author

chrysn commented Aug 24, 2022

Native tests largely passed like in #257 (comment) -- but a new pkg has triggered static test failures in GCC. There is RIOT-OS/RIOT#18508 reported now, but considering how that'd stretch things out I'm putting this on "known issues". (While this is conceptually similar to the CCN-lite use-after-free for which I did go through another RC, the difference is that there it was UB on old compilers, and here it's just access out of bounds declared in the function argument, which AIU is not UB on its own in C).

Thus ticking the native box.

@leandrolanzieri
Copy link
Contributor

leandrolanzieri commented Aug 24, 2022

08. Task 11 - UDP exchange between iotlab-m3 and Zephyr - PASS
udp send fe80::2048:400:ae12:9f3b 4242 "RIOT testing"
2022-08-24 15:12:44,052 # udp send fe80::2048:400:ae12:9f3b 4242 "RIOT testing"
2022-08-24 15:12:44,056 # Success: sent 12 byte(s) to [fe80::2048:400:ae12:9f3b]:4242
> 2022-08-24 15:12:44,057 # PKTDUMP: data received:
2022-08-24 15:12:44,060 # ~~ SNIP  0 - size:  12 byte, type: NETTYPE_UNDEF (0)
2022-08-24 15:12:44,063 # 00000000  52  49  4F  54  20  74  65  73  74  69  6E  67
2022-08-24 15:12:44,066 # ~~ SNIP  1 - size:   8 byte, type: NETTYPE_UDP (4)
2022-08-24 15:12:44,068 #    src-port:  4242  dst-port:  4242
2022-08-24 15:12:44,069 #    length: 20  cksum: 0x5923
2022-08-24 15:12:44,071 # ~~ SNIP  2 - size:  40 byte, type: NETTYPE_IPV6 (2)
2022-08-24 15:12:44,073 # traffic class: 0x00 (ECN: 0x0, DSCP: 0x00)
2022-08-24 15:12:44,074 # flow label: 0x00000
2022-08-24 15:12:44,075 # length: 20  next header: 17  hop limit: 64
2022-08-24 15:12:44,077 # source address: fe80::2048:400:ae12:9f3b
2022-08-24 15:12:44,078 # destination address: fe80::a884:9937:4f86:7b4e
2022-08-24 15:12:44,080 # ~~ SNIP  3 - size:  24 byte, type: NETTYPE_NETIF (-1)
2022-08-24 15:12:44,082 # if_pid: 6  rssi: -38  lqi: 255
2022-08-24 15:12:44,082 # flags: 0x0
2022-08-24 15:12:44,083 # src_l2addr: 22:48:04:00:AE:12:9F:3B
2022-08-24 15:12:44,084 # dst_l2addr: AA:84:99:37:4F:86:7B:4E
2022-08-24 15:12:44,085 # ~~ PKT    -  4 snips, total size:  84 byte
udp send fe80::2048:400:ae12:9f3b 4242 "RIOT testing"
2022-08-24 15:12:44,868 # udp send fe80::2048:400:ae12:9f3b 4242 "RIOT testing"
2022-08-24 15:12:44,872 # Success: sent 12 byte(s) to [fe80::2048:400:ae12:9f3b]:4242
> 2022-08-24 15:12:44,881 # PKTDUMP: data received:
2022-08-24 15:12:44,883 # ~~ SNIP  0 - size:  12 byte, type: NETTYPE_UNDEF (0)
2022-08-24 15:12:44,886 # 00000000  52  49  4F  54  20  74  65  73  74  69  6E  67
2022-08-24 15:12:44,888 # ~~ SNIP  1 - size:   8 byte, type: NETTYPE_UDP (4)
2022-08-24 15:12:44,889 #    src-port:  4242  dst-port:  4242
2022-08-24 15:12:44,890 #    length: 20  cksum: 0x5923
2022-08-24 15:12:44,892 # ~~ SNIP  2 - size:  40 byte, type: NETTYPE_IPV6 (2)
2022-08-24 15:12:44,893 # traffic class: 0x00 (ECN: 0x0, DSCP: 0x00)
2022-08-24 15:12:44,894 # flow label: 0x00000
2022-08-24 15:12:44,896 # length: 20  next header: 17  hop limit: 64
2022-08-24 15:12:44,898 # source address: fe80::2048:400:ae12:9f3b
2022-08-24 15:12:44,900 # destination address: fe80::a884:9937:4f86:7b4e
2022-08-24 15:12:44,902 # ~~ SNIP  3 - size:  24 byte, type: NETTYPE_NETIF (-1)
2022-08-24 15:12:44,904 # if_pid: 6  rssi: -39  lqi: 255
2022-08-24 15:12:44,904 # flags: 0x0
2022-08-24 15:12:44,905 # src_l2addr: 22:48:04:00:AE:12:9F:3B
2022-08-24 15:12:44,906 # dst_l2addr: AA:84:99:37:4F:86:7B:4E
2022-08-24 15:12:44,907 # ~~ PKT    -  4 snips, total size:  84 byte
udp send fe80::2048:400:ae12:9f3b 4242 "RIOT testing"
2022-08-24 15:12:45,857 # udp send fe80::2048:400:ae12:9f3b 4242 "RIOT testing"
2022-08-24 15:12:45,873 # Success: sent 12 byte(s) to [fe80::2048:400:ae12:9f3b]:4242
> 2022-08-24 15:12:45,874 # PKTDUMP: data received:
2022-08-24 15:12:45,876 # ~~ SNIP  0 - size:  12 byte, type: NETTYPE_UNDEF (0)
2022-08-24 15:12:45,877 # 00000000  52  49  4F  54  20  74  65  73  74  69  6E  67
2022-08-24 15:12:45,877 # ~~ SNIP  1 - size:   8 byte, type: NETTYPE_UDP (4)
2022-08-24 15:12:45,878 #    src-port:  4242  dst-port:  4242
2022-08-24 15:12:45,878 #    length: 20  cksum: 0x5923
2022-08-24 15:12:45,879 # ~~ SNIP  2 - size:  40 byte, type: NETTYPE_IPV6 (2)
2022-08-24 15:12:45,880 # traffic class: 0x00 (ECN: 0x0, DSCP: 0x00)
2022-08-24 15:12:45,880 # flow label: 0x00000
2022-08-24 15:12:45,881 # length: 20  next header: 17  hop limit: 64
2022-08-24 15:12:45,881 # source address: fe80::2048:400:ae12:9f3b
2022-08-24 15:12:45,888 # destination address: fe80::a884:9937:4f86:7b4e
2022-08-24 15:12:45,888 # ~~ SNIP  3 - size:  24 byte, type: NETTYPE_NETIF (-1)
2022-08-24 15:12:45,889 # if_pid: 6  rssi: -39  lqi: 255
2022-08-24 15:12:45,889 # flags: 0x0
2022-08-24 15:12:45,889 # src_l2addr: 22:48:04:00:AE:12:9F:3B
2022-08-24 15:12:45,890 # dst_l2addr: AA:84:99:37:4F:86:7B:4E
2022-08-24 15:12:45,890 # ~~ PKT    -  4 snips, total size:  84 byte

I observed a couple of missing echoes from the zephyr node (fixed by a reboot of it), but the Zephyr would receive the RIOT packet.

@jia200x
Copy link
Member

jia200x commented Aug 24, 2022

08-interop. Task #4 - ICMPv6 echo between iotlab-m3 and Internet host through Linux with 6LowPAN - PASS
2022-08-24 15:21:11,922 # ping 2002:8d16:1b8e:28a1:263b:e40c:8ceb:f013
2022-08-24 15:21:11,938 # 12 bytes from 2002:8d16:1b8e:28a1:263b:e40c:8ceb:f013: icmp_seq=0 ttl=64 rssi=-61 dBm time=8.546 ms
2022-08-24 15:21:12,933 # 12 bytes from 2002:8d16:1b8e:28a1:263b:e40c:8ceb:f013: icmp_seq=1 ttl=64 rssi=-58 dBm time=9.508 ms
2022-08-24 15:21:13,938 # 12 bytes from 2002:8d16:1b8e:28a1:263b:e40c:8ceb:f013: icmp_seq=2 ttl=64 rssi=-58 dBm time=7.601 ms
2022-08-24 15:21:13,938 # 

@jia200x
Copy link
Member

jia200x commented Aug 24, 2022

08-interop. Task #5 - ICMPv6 echo between iotlab-m3 and Internet host through RIOT border router - PASS
2022-08-24 16:31:48,523 # ping  fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86
2022-08-24 16:31:48,572 # 12 bytes from fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86: icmp_seq=0 ttl=63 rssi=-51 dBm time=42.054 ms
2022-08-24 16:31:49,549 # 12 bytes from fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86: icmp_seq=1 ttl=63 rssi=-52 dBm time=23.975 ms
2022-08-24 16:31:50,541 # 12 bytes from fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86: icmp_seq=2 ttl=63 rssi=-52 dBm time=20.713 ms
2022-08-24 16:31:50,541 # 
2022-08-24 16:31:50,544 # --- fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 PING statistics ---
2022-08-24 16:31:50,547 # 3 packets transmitted, 3 packets received, 0% packet loss
2022-08-24 16:31:50,553 # round-trip min/avg/max = 20.713/28.914/42.054 ms

@jia200x
Copy link
Member

jia200x commented Aug 24, 2022

08-interop. Task #6 - UDP between iotlab-m3 and Internet host through RIOT border router - PASS
> 2022-08-24 16:36:20,934 # udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10
2022-08-24 16:36:20,935 # Success: send 10 byte to [fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86]:8000
udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10> 
2022-08-24 16:36:22,007 # udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10
2022-08-24 16:36:22,012 # Success: send 10 byte to [fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86]:8000
udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10> 
2022-08-24 16:36:23,337 # udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10
2022-08-24 16:36:23,340 # Success: send 10 byte to [fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86]:8000
> udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10
2022-08-24 16:36:28,759 # udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10
2022-08-24 16:36:28,775 # Success: send 10 byte to [fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86]:8000
udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10> 
2022-08-24 16:36:30,103 # udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 10
2022-08-24 16:36:30,107 # Success: send 10 byte to [fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86]:8000



~/ socat - UDP6-LISTEN:8000,reuseaddr,fork | xxd
00000000: 0c0c 0c0c 0c0c 0c0c 0c0c 0d0d 0d0d 0d0d  ................
00000010: 0d0d 0d0d 0e0e 0e0e 0e0e 0e0e 0e0e 0f0f  ................
00000020: 0f0f 0f0f 0f0f 0f0f 1010 1010 1010 1010  ................
00000030: 1010 1111 1111 1111 1111 1111 1212 1212  ................

@jia200x
Copy link
Member

jia200x commented Aug 24, 2022

08-interop. Task #7 - UDP between iotlab-m3 and Internet host through RIOT border router (200b payload) - PASS
2022-08-24 16:37:56,944 # udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 200
2022-08-24 16:37:56,962 # Success: send 200 byte to [fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86]:8000
2022-08-24 16:37:58,798 # udp send fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86 8000 200
2022-08-24 16:37:58,818 # Success: send 200 byte to [fd68:cdb2:4ca4:0:d5c2:ba65:1756:df86]:8000


00000000: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000010: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000020: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000030: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000040: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000050: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000060: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000070: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000080: 1313 1313 1313 1313 1313 1313 1313 1313  ................
00000090: 1313 1313 1313 1313 1313 1313 1313 1313  ................
000000a0: 1313 1313 1313 1313 1313 1313 1313 1313  ................
000000b0: 1313 1313 1313 1313 1313 1313 1313 1313  ................
000000c0: 1313 1313 1313 1313 1414 1414 1414 1414  ................
000000d0: 1414 1414 1414 1414 1414 1414 1414 1414  ................
000000e0: 1414 1414 1414 1414 1414 1414 1414 1414  ................
000000f0: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000100: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000110: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000120: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000130: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000140: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000150: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000160: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000170: 1414 1414 1414 1414 1414 1414 1414 1414  ................
00000180: 1414 1414 1414 1414 1414 1414 1414 1414  ................

@jia200x
Copy link
Member

jia200x commented Aug 24, 2022

08-interop. Task #2 - ICMPv6 echo between iotlab-m3 and Linux with 6LowPAN - PASS
2022-08-24 16:43:33,924 # ping fe80::243b:e40c:8ceb:f013
2022-08-24 16:43:33,940 # 12 bytes from fe80::243b:e40c:8ceb:f013%6: icmp_seq=0 ttl=64 rssi=-58 dBm time=6.449 ms
2022-08-24 16:43:34,935 # 12 bytes from fe80::243b:e40c:8ceb:f013%6: icmp_seq=1 ttl=64 rssi=-56 dBm time=6.747 ms
2022-08-24 16:43:35,945 # 12 bytes from fe80::243b:e40c:8ceb:f013%6: icmp_seq=2 ttl=64 rssi=-58 dBm time=7.658 ms
2022-08-24 16:43:35,946 # 
2022-08-24 16:43:35,948 # --- fe80::243b:e40c:8ceb:f013 PING statistics ---
2022-08-24 16:43:35,951 # 3 packets transmitted, 3 packets received, 0% packet loss
2022-08-24 16:43:35,952 # round-trip min/avg/max = 6.449/6.951/7.658 ms

@jia200x
Copy link
Member

jia200x commented Aug 24, 2022

08-interop. Task #3 - UDP exchange between iotlab-m3 and Contiki - PASS
2022-08-24 17:23:22,947 # ping fe80::0212:4b00:0422:9db8
2022-08-24 17:23:22,952 # 12 bytes from fe80::212:4b00:422:9db8%6: icmp_seq=0 ttl=64 rssi=-26 dBm time=7.350 ms
2022-08-24 17:23:23,954 # 12 bytes from fe80::212:4b00:422:9db8%6: icmp_seq=1 ttl=64 rssi=-26 dBm time=8.602 ms
2022-08-24 17:23:24,950 # 12 bytes from fe80::212:4b00:422:9db8%6: icmp_seq=2 ttl=64 rssi=-26 dBm time=7.351 ms
2022-08-24 17:23:24,950 # 
2022-08-24 17:23:24,953 # --- fe80::0212:4b00:0422:9db8 PING statistics ---
2022-08-24 17:23:24,957 # 3 packets transmitted, 3 packets received, 0% packet loss
2022-08-24 17:23:24,958 # round-trip min/avg/max = 7.350/7.767/8.602 ms

@jia200x
Copy link
Member

jia200x commented Aug 24, 2022

I ran all interest tests I could. Could someone run the ones with Wifi and ESP?

@chrysn
Copy link
Member Author

chrysn commented Aug 24, 2022 via email

@chrysn
Copy link
Member Author

chrysn commented Aug 24, 2022

Ran compile-and-test locally with tests on IoT-Lab for iotlab-m3 and samr21; largely as in the last test:

  • Some builds failed due to missing local tools / libraries. (That's OK -- we want to test local tools, because on the images things are tested elsewhere)
  • LVGL builds fail on SAMR21 ("Error: unaligned opcodes detected in executable segment"). This is known upstream, and likely an error specific to the GCC version I'm using (arm-none-eabi-gcc (15:10.3-2021.07-4) 10.3.1 20210621 (release)).
  • Some just glitched (and worked when tested again)
  • riotboot, as before, would need a bootloader as far as I understand
  • tests/periph_timer_short_relative set looks like it can't pass at all and that's OK -- see tests/periph_timer_short_relative_set results are misleading RIOT#18511 but not release critical
  • tests/gnrc_rpl is not expected to run on anything but native

Thus, I'm ticking the 02-tests/#2 and 02-tests/#3

@benpicco
Copy link
Contributor

benpicco commented Aug 24, 2022

Task #12 - ICMPv6 echo between Border Router with WiFi uplink to named Internet host

passed (esp32-wroom-32)
2022-08-24 22:08:02,677 # Iface  9  HWaddr: 3E:71:BF:9E:13:FC  Channel: 1 
2022-08-24 22:08:02,680 #           L2-PDU:249  MTU:1280  HL:64  RTR  
2022-08-24 22:08:02,685 #           RTR_ADV  6LO  Source address length: 6
2022-08-24 22:08:02,687 #           Link type: wireless
2022-08-24 22:08:02,693 #           inet6 addr: fe80::3c71:bfff:fe9e:13fc  scope: link  VAL
2022-08-24 22:08:02,700 #           inet6 addr: 2001:9e8:1406:2bfc:3c71:bfff:fe9e:13fc  scope: global  VAL
2022-08-24 22:08:02,703 #           inet6 group: ff02::2
2022-08-24 22:08:02,705 #           inet6 group: ff02::1
2022-08-24 22:08:02,709 #           inet6 group: ff02::1:ff9e:13fc
2022-08-24 22:08:02,710 #           
2022-08-24 22:08:02,715 # Iface  10  HWaddr: 3C:71:BF:9E:13:FC  Channel: 1  Link: up 
2022-08-24 22:08:02,719 #           L2-PDU:1500  MTU:1492  HL:255  RTR  
2022-08-24 22:08:02,722 #           Source address length: 6
2022-08-24 22:08:02,725 #           Link type: wireless
2022-08-24 22:08:02,731 #           inet6 addr: fe80::3e71:bfff:fe9e:13fc  scope: link  VAL
2022-08-24 22:08:02,738 #           inet6 addr: 2001:9e8:1406:2b00:3e71:bfff:fe9e:13fc  scope: global  VAL
2022-08-24 22:08:02,740 #           inet6 group: ff02::2
2022-08-24 22:08:02,743 #           inet6 group: ff02::1
2022-08-24 22:08:02,747 #           inet6 group: ff02::1:ff9e:13fc
2022-08-24 22:08:02,748 #           
> ping 2600::
2022-08-24 22:08:49,032 # ping 2600::
2022-08-24 22:08:49,245 # 12 bytes from 2600::: icmp_seq=0 ttl=50 time=207.473 ms
2022-08-24 22:08:50,262 # 12 bytes from 2600::: icmp_seq=1 ttl=50 time=224.702 ms
2022-08-24 22:08:51,164 # 12 bytes from 2600::: icmp_seq=2 ttl=50 time=127.042 ms
2022-08-24 22:08:51,165 # 
2022-08-24 22:08:51,168 # --- 2600:: PING statistics ---
2022-08-24 22:08:51,172 # 3 packets transmitted, 3 packets received, 0% packet loss
2022-08-24 22:08:51,177 # round-trip min/avg/max = 127.042/186.405/224.702 ms
> ping riot-os.org
2022-08-24 22:08:52,656 # ping riot-os.org
2022-08-24 22:08:52,740 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=0 ttl=59 time=18.946 ms
2022-08-24 22:08:53,744 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=1 ttl=59 time=23.529 ms
2022-08-24 22:08:54,877 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=2 ttl=59 time=155.917 ms
2022-08-24 22:08:54,878 # 
2022-08-24 22:08:54,880 # --- riot-os.org PING statistics ---
2022-08-24 22:08:54,885 # 3 packets transmitted, 3 packets received, 0% packet loss
2022-08-24 22:08:54,890 # round-trip min/avg/max = 18.946/66.130/155.917 ms

Task #13 - ICMPv6 echo between ESP and named Internet host through Linux with 6LowPAN

passed (esp8266-esp-12x)
2022-08-24 22:09:30,406 # Iface  11  HWaddr: EE:FA:BC:5F:7E:5F  Channel: 1 
2022-08-24 22:09:30,411 #           L2-PDU:249  MTU:1280  HL:64  RTR  
2022-08-24 22:09:30,414 #           RTR_ADV  6LO  Source address length: 6
2022-08-24 22:09:30,416 #           Link type: wireless
2022-08-24 22:09:30,422 #           inet6 addr: fe80::ecfa:bcff:fe5f:7e5f  scope: link  VAL
2022-08-24 22:09:30,430 #           inet6 addr: 2001:9e8:1406:2bfc:ecfa:bcff:fe5f:7e5f  scope: global  VAL
2022-08-24 22:09:30,433 #           inet6 group: ff02::2
2022-08-24 22:09:30,436 #           inet6 group: ff02::1
2022-08-24 22:09:30,439 #           inet6 group: ff02::1:ff5f:7e5f
2022-08-24 22:09:30,441 #           inet6 group: ff02::1a
2022-08-24 22:09:30,442 #           
2022-08-24 22:09:30,444 #           Statistics for Layer 2
2022-08-24 22:09:30,447 #             RX packets 0  bytes 0
2022-08-24 22:09:30,452 #             TX packets 0 (Multicast: 0)  bytes 10704
2022-08-24 22:09:30,455 #             TX succeeded 184 errors 0
2022-08-24 22:09:30,458 #           Statistics for IPv6
2022-08-24 22:09:30,463 #             RX packets 184  bytes 29360
2022-08-24 22:09:30,467 #             TX packets 188 (Multicast: 187)  bytes 10554
2022-08-24 22:09:30,470 #             TX succeeded 188 errors 0
2022-08-24 22:09:30,471 # 
> ping 2600::
2022-08-24 22:09:34,339 # ping 2600::
2022-08-24 22:09:34,483 # 12 bytes from 2600::: icmp_seq=0 ttl=49 time=137.506 ms
2022-08-24 22:09:35,480 # 12 bytes from 2600::: icmp_seq=1 ttl=49 time=134.470 ms
2022-08-24 22:09:36,478 # 12 bytes from 2600::: icmp_seq=2 ttl=49 time=132.859 ms
2022-08-24 22:09:36,478 # 
2022-08-24 22:09:36,480 # --- 2600:: PING statistics ---
2022-08-24 22:09:36,486 # 3 packets transmitted, 3 packets received, 0% packet loss
2022-08-24 22:09:36,491 # round-trip min/avg/max = 132.859/134.945/137.506 ms
> ping riot-os.org
2022-08-24 22:09:40,617 # ping riot-os.org
2022-08-24 22:09:40,814 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=0 ttl=58 time=161.448 ms
2022-08-24 22:09:41,878 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=1 ttl=58 time=221.431 ms
2022-08-24 22:09:42,707 # 12 bytes from 2001:67c:254:b0b0::1: icmp_seq=2 ttl=58 time=46.882 ms
2022-08-24 22:09:42,707 # 
2022-08-24 22:09:42,710 # --- riot-os.org PING statistics ---
2022-08-24 22:09:42,715 # 3 packets transmitted, 3 packets received, 0% packet loss
2022-08-24 22:09:42,720 # round-trip min/avg/max = 46.882/143.253/221.431 ms

Ping from Linux also works

PING 2001:9e8:1406:2bfc:ecfa:bcff:fe5f:7e5f(2001:9e8:1406:2bfc:ecfa:bcff:fe5f:7e5f) 56 data bytes
64 bytes from 2001:9e8:1406:2bfc:ecfa:bcff:fe5f:7e5f: icmp_seq=26 ttl=63 time=95.3 ms
64 bytes from 2001:9e8:1406:2bfc:ecfa:bcff:fe5f:7e5f: icmp_seq=27 ttl=63 time=34.4 ms
64 bytes from 2001:9e8:1406:2bfc:ecfa:bcff:fe5f:7e5f: icmp_seq=28 ttl=63 time=23.6 ms
64 bytes from 2001:9e8:1406:2bfc:ecfa:bcff:fe5f:7e5f: icmp_seq=29 ttl=63 time=40.3 ms

Unfortunately the esp8266 hangs very quickly after being exposed to the local network - but this is not a new issue.

@chrysn
Copy link
Member Author

chrysn commented Aug 25, 2022

Ran a bunch of other examples to satisfy 99-compile-and-test-one-board:

  • on microbit-v2: skald_eddystone, gcoap, nimble_scanner, saul
  • on stk3700: saul; tested tests/periph_rtc, tests/periph_gpio_ll (with adaequate Makefile.stk3700)
  • on hifive1 (built in docker): default, rust-hello-world
    Ticking 01-ci/[WIP] 2015.08: multi-hop test on iotlab testbed #1 as that was covered by the last backport running through CI.

@chrysn
Copy link
Member Author

chrysn commented Aug 25, 2022

The remaining ticks are all either

  • experimental -- OK to not be ticked
  • LoRaWAN (ABP) -- can not be tested with current TTN
  • LoRaWAN example -- this failed during CI at startup (no 'This is RIOT'); I'm starting the CI job again.

Looking good; unless the LoRaWAN example keeps breaking, or someone stumbles over something critical, this will be it, once the release notes are through.

@miri64
Copy link
Member

miri64 commented Aug 25, 2022

  • LoRaWAN (ABP) -- can not be tested with current TTN

Can be, but is flaky, see #255.

  • LoRaWAN example -- this failed during CI at startup (no 'This is RIOT'); I'm starting the CI job again

Probably should be marked as flaky as well.

@chrysn
Copy link
Member Author

chrysn commented Aug 25, 2022

It failed again, but it worked in RC3 without any relevant changes in that area, so I'd consider this flaky based on the above.

Thus, all tests either passed or are expected to (sometimes) fail; it's only the release notes now (which I'd let gather feedback over the weekend).

@miri64
Copy link
Member

miri64 commented Aug 25, 2022

It failed again, but it worked in RC3 without any relevant changes in that area, so I'd consider this flaky based on the above.

yeah, during testing #255, @jia200x and I noticed, that if the lorawan nodes on the testbed "decide" to not to work, they will stay that way for a while :-/... so maybe it doesn't even make sense to mark the test as flaky, in the pytest sense (i.e. rerun immediately if failed).

@jia200x
Copy link
Member

jia200x commented Aug 25, 2022

It failed again, but it worked in RC3 without any relevant changes in that area, so I'd consider this flaky based on the above.

yeah, during testing #255, @jia200x and I noticed, that if the lorawan nodes on the testbed "decide" to not to work, they will stay that way for a while :-/... so maybe it doesn't even make sense to mark the test as flaky, in the pytest sense (i.e. rerun immediately if failed).

Yup... it's because of iotlab nodes randomly disconnecting. This test always succeeds when ran locally.

@gschorcht
Copy link

gschorcht commented Aug 26, 2022

On Wed, Aug 24, 2022 at 08:25:46AM -0700, José Alamos wrote:
I ran all interest tests I could. Could someone run the ones with Wifi and ESP?

@gschorcht, could you help here maybe?

Unfortunately not, I'm on vacation. Although I have some hardware with me, I don't have the required WiFi infrastructure here. But, I have use WiFi so often in last month that I'm convinced that it works in 2022.07. Maybe @benpicco might confirm.

@benpicco
Copy link
Contributor

I already ran the test - #258 (comment)

@gschorcht
Copy link

I already ran the test - #258 (comment)

Ah ok, I see. Sorry for the noise, a smartphone is obviously not a good choice for checking emails, following links in emails, and keeping a track of what's going on..

@chrysn
Copy link
Member Author

chrysn commented Sep 23, 2022

Should have been closed with the release. Thanks everyone for helping out!

@chrysn chrysn closed this as completed Sep 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants