Release 2018.10 - RC2

[jia200x]

This issue lists the status of all tests for the Release Candidate 2 of the 2018.10 release.

Specs tested:

• 01-ci
  • Task #01 - Compile test @jia200x
  • Task #02 - Unittests on native @jia200x
  • Task #03 - Unittests on native separated @jia200x
  • Task #04 - Unittests on iotlab-m3 @jia200x
• 02-tests
  • Task #01 - All tests on native
  • Task #02 - Subset of tests on iotlab-m3
  • Task #03 - Subset of tests on samr21-xpro
• 03-single-hop-ipv6-icmp
  • Task #01 - ICMPv6 multicast echo on native
  • Task #02 - ICMPv6 link-local echo on native
  • Task #03 - ICMPv6 link-local echo on native (1 hour) @jia200x
• 04-single-hop-6lowpan-icmp (@miri64)
  • Task #01 - ICMPv6 link-local echo with iotlab-m3 (1% packet loss)
  • Task #02 - ICMPv6 multicast echo with iotlab-m3/samr21-xpro (0% packet loss from both sides)
  • Task #03 - ICMPv6 echo with large payload (3% packet loss)
  • Task #04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes (1% packet loss from both sides)
  • Task #05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/remote
  • Task #06 (Experimental) - ICMPv6 link-local echo with samr21-xpro/remote
  • Task #07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee (3% packet loss arduino-zero => samr21-xpro, 6% packet loss samr21-xpro => arduino-zero)
  • Task #08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee 15 minutes (3% packet loss arduino-zero => samr21-xpro, 1% packet loss samr21-xpro => arduino-zero)
• 05-single-hop-route @jia200x
  • Task #01 - ICMPv6 echo unicast addresess on native (default route)
  • Task #02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route)
  • Task #03 - ICMPv6 echo unicast addresess on native (specific route)
  • Task #04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route)
• 06-single-hop-udp @leandrolanzieri
  • Task #01 - UDP on iotlab-m3 1% packet loss
  • Task #02 - UDP on iotlab-m3 (UDP port compression) 0.5% packet loss
• 07-multi-hop @MrKevinWeiss
  • Task #01 - ICMPv6 echo on iotlab-m3 with three hops (static route)
  • Task #02 - UDP on iotlab-m3 with three hops (static route)
  • Task #03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route)
  • Task #04 - UDP on iotlab-m3 with three hops (RPL route)
• 08-interop
  • Task #01 - ICMPv6 echo between native and Linux
  • Task #02 - ICMPv6 echo between iotlab-m3 and Linux with 6LowPAN
  • Task #03 - ICMPv6 echo between iotlab-m3 and Contiki (@kYc0o already tested it in RC-1. I will use the same results since nothing changed here)
  • Task #04 - ICMPv6 echo between iotlab-m3 and Internet host through Linux with 6LowPAN
  • Task #05 - ICMPv6 echo between iotlab-m3 and Internet host through RIOT border router
  • Task #06 - UDP between iotlab-m3 and Internet host through RIOT border router
  • Task #07 - UDP between iotlab-m3 and Internet host through RIOT border router (200b payload)
  • Task #08 - UDP between GNRC and lwIP on iotlab-m3
  • Task #09 - UDP between GNRC and emb6 on iotlab-m3
  • Task #10 - UDP between lwIP and emb6 on iotlab-m3
• 09-coap (Experimental) @kb2ma
  • Task #01 - CORD Endpoint
  • Task #02 - Confirmable retries
  • Task #03 - Block1
  • Task #04 - Block2
  • Task #05 - Observe registration and notification
• 99-compile-and-test-one-board
  • Task #01 - Run tests on different hardwares

[jia200x]

@kb2ma do you have time to re-run the 09-coap tasks again? :)

[kb2ma]

No problem, @jia200x. I'll should get to it by Wednesday morning.

[jia200x]

No problem, @jia200x. I'll should get to it by Wednesday morning.

Awesome! Thank you so much!

[jia200x]

waiting for 01/01. Unittests pass in all variants

[miri64]

I don't have access to a remote, so 4.5 and 4.6 needs to be done by someone else.

[jia200x]

I don't have access to a remote, so 4.5 and 4.6 needs to be done by someone else.

We can run it here at HAW

[miri64]

Nothing interesting to report for task 4.

[jia200x]

GNRC - emb6 - lwip triade works as expected in the interop tests

[kb2ma]

09-coap works as expected.

[jia200x]

Native tests:

Failures during test:
- [tests/driver_my9221](tests/driver_my9221/test.failed)
- [tests/pkg_cayenne-lpp](tests/pkg_cayenne-lpp/test.failed)
- [tests/pkg_lora-serialization](tests/pkg_lora-serialization/test.failed)

driver_my9221 is expected to fail on native (it shouldn't run IMO though). pkg_cayenne-lpp and pkg_lora-serialization fail in GCC8 because of floating point implementation. Not blocking for the release though.

I think we should implement the classic AssertAlmostEqual for these tests soon-ish

[miri64]

FYI, @bergzand found a serious bug in RIOT-OS/RIOT#10246 (comment). Luckily in its current version RIOT-OS/RIOT#10246 fixes that (though very ugly, but later reversible).

[jia200x]

Output of compile test:

Errors:

Outcome:
        success: arduino_hello-world, asymcute_mqttsn, bindist, ccn-lite-relay, cord_ep, cord_epsim, default, dtls-echo, emcute_mqttsn, filesystem, gcoap, gnrc_border_router, gnrc_minimal, gnrc_networking, gnrc_networking_mac, gnrc_tftp, hello-world, ipc_pingpong, javascript, lorawan, lua_REPL, lua_basic, nanocoap_server, ndn-ping, nimble_gatt, openthread, posix_sockets, riot_and_cpp, saul, skald_eddystone, skald_ibeacon, timer_periodic_wakeup, bench_msg_pingpong, bench_mutex_pingpong, bench_runtime_coreapis, bench_sched_nop, bench_sizeof_coretypes, bench_thread_flags_pingpong, bench_thread_yield_pingpong, bench_timers, bitarithm_timings, bloom_bytes, board_calliope-mini, board_microbit, buttons, can_trx, cb_mux, cb_mux_bench, conn_can, cortexm_common_ldscript, cpp11_condition_variable, cpp11_mutex, cpp11_thread, cpu_efm32_features, driver_adcxx1c, driver_ads101x, driver_adt7310, driver_adxl345, driver_apa102, driver_at, driver_at30tse75x, driver_at86rf2xx, driver_ata8520e, driver_bh1750, driver_bmp180, driver_bmx055, driver_bmx280, driver_dht, driver_ds1307, driver_ds18, driver_dsp0401, driver_dynamixel, driver_enc28j60, driver_encx24j600, driver_feetech, driver_fxos8700, driver_grove_ledbar, driver_hd44780, driver_hdc1000, driver_hih6130, driver_hts221, driver_ina220, driver_io1_xplained, driver_isl29020, driver_isl29125, driver_jc42, driver_kw2xrf, driver_l3g4200d, driver_lc709203f, driver_lis2dh12, driver_lis3dh, driver_lis3mdl, driver_lpd8808, driver_lps331ap, driver_lsm303dlhc, driver_lsm6dsl, driver_mag3110, driver_mma7660, driver_mma8x5x, driver_mpl3115a2, driver_mpu9150, driver_mq3, driver_my9221, driver_nrf24l01p_lowlevel, driver_nvram_spi, driver_pcd8544, driver_pir, driver_pn532, driver_pulse_counter, driver_rn2xx3, driver_sdcard_spi, driver_servo, driver_sht1x, driver_si114x, driver_si70xx, driver_soft_spi, driver_srf02, driver_srf04, driver_srf08, driver_sx127x, driver_tcs37727, driver_tmp006, driver_tsl2561, driver_tsl4531x, driver_vcnl40x0, driver_veml6070, driver_xbee, eepreg, emb6, events, evtimer_msg, evtimer_underflow, external_module_dirs, fault_handler, float, fmt_print, gnrc_gomach, gnrc_ipv6_ext, gnrc_ipv6_nib_6ln, gnrc_lwmac, gnrc_ndp, gnrc_netif, gnrc_sixlowpan, gnrc_sock_dns, gnrc_sock_ip, gnrc_sock_udp, gnrc_tcp_client, gnrc_tcp_server, gnrc_udp, irq, isr_yield_higher, leds, libc_newlib, libfixmath, libfixmath_unittests, lwip, lwip_sock_ip, lwip_sock_tcp, lwip_sock_udp, malloc, mcuboot, memarray, minimal, mpu_stack_guard, msg_avail, msg_send_receive, msg_try_receive, mutex_order, mutex_unlock_and_sleep, netdev_test, netstats_l2, nhdp, od, periph_adc, periph_cpuid, periph_dac, periph_dma, periph_eeprom, periph_flashpage, periph_gpio, periph_hwrng, periph_i2c, periph_pm, periph_pwm, periph_qdec, periph_rtc, periph_rtt, periph_spi, periph_timer, periph_uart, pipe, pkg_cayenne-lpp, pkg_cmsis-dsp, pkg_fatfs, pkg_fatfs_vfs, pkg_hacl, pkg_heatshrink, pkg_jsmn, pkg_libb2, pkg_libcoap, pkg_libcose, pkg_libhydrogen, pkg_lora-serialization, pkg_micro-ecc, pkg_micro-ecc-with-hwrng, pkg_microcoap, pkg_minmea, pkg_monocypher, pkg_oonf_api, pkg_semtech-loramac, pkg_tiny-asn1, pkg_tinycbor, pkg_tinycrypt, pkg_tweetnacl, pkg_u8g2, pkg_ucglib, pkg_umorse, posix_semaphore, posix_time, ps_schedstatistics, pthread, pthread_barrier, pthread_cleanup, pthread_condition_variable, pthread_cooperation, pthread_rwlock, pthread_tls, puf_sram, rmutex, rng, saul, sched_testing, shell, slip, sntp, socket_zep, ssp, struct_tm_utility, sys_arduino, thread_basic, thread_cooperation, thread_exit, thread_flags, thread_flags_xtimer, thread_flood, thread_msg, thread_msg_block_w_queue, thread_msg_block_wo_queue, thread_msg_seq, thread_priority_inversion, thread_race, trace, trickle, unittests, warn_conflict, xtimer_drift, xtimer_hang, xtimer_longterm, xtimer_msg, xtimer_msg_receive_timeout, xtimer_now64_continuity, xtimer_periodic_wakeup, xtimer_remove, xtimer_reset, xtimer_usleep, xtimer_usleep_short
        failed: gnrc_ipv6_nib
Errors: 0 Warnings: 0

The gnrc_ipv6_nib example fails to build on waspmote-pro. It's the same problem as described in RIOT-OS/Release-Specs#76 (comment), and it's solved with RIOT-OS/RIOT#9646. So, no need to worry about it

[cladmi]

I ran some scan-build to test the new docker images and it reports actual bugs like (netif == NULL) ? netif->pid : 0); for make -C examples/gnrc_networking scan-build

I will try running it for the boards supporting llvm and do a report.

[miri64]

I ran some scan-build to test the new docker images and it reports actual bugs like (netif == NULL) ? netif->pid : 0); for make -C examples/gnrc_networking scan-build

I'm doing a fix for the native reports as we speak. @jia200x do we do another release candidate for all those last minute GNRC stuff?

[miri64]

See RIOT-OS/RIOT#10350.

[jia200x]

I'm doing a fix for the native reports as we speak. @jia200x do we do another release candidate for all those last minute GNRC stuff?

Yes. Also for the latest fixes

[cladmi]

Unfortunately I did not have time to run and analyze them before the week-end.

[miri64]

Who tested 8.6 and 8.7 because the RIOT border router on samr21-xpro crashes for me in the current release branch oO

[jia200x]

Who tested 8.6 and 8.7 because the RIOT border router on samr21-xpro crashes for me in the current release branch oO

I tested it. It didn't crash in any of the RC. Let me re-check in any case

[miri64]

(It also crashes on RC2, I checked)

[miri64]

All-clear: it was the broken samr21-xpro I was using ^^"

[jia200x]

Already in RC3