RaspberryMatic 3.61.7.20211218

[jens-maus]

This is release 3.61.7.20211218 of RaspberryMatic which is a major release with the following changes:

_{Hier klicken für deutschsprachige 🇩🇪🇦🇹🇨🇭 Version des ChangeLogs/Diskussionsbeitrag}

🚧 Changes:

_{For all changes, see the full commit log.}

CCU/HomeMatic service changes:

• updated OCCU version to 3.61.7-2 integrating full compatibility to the CCU3 3.61.7 firmware which comes with the following changes:
  • updated ReGaHss logic engine to version R1.00.0388.0228 (Dec 10 2021) coming with the following changes:
    • fixed a bug in the central GetHistoryData() function which returned invalid structured text if the content itself contained the reserved characters ; or \t (cf. ReGaHss: GetHistoryData() returns invalid structured text if content contains ";" or "\t" #1587).
  • HmIP-PSM: added missing translation for FREQUENCY_STATUS parameter of channel 6.
  • HmIP-eTRV-E-S: added device integration for new silver version of device.
  • HmIP-MP3P: fixed issue that not more than one mp3 was playable in WebUI programs.
• integrated a new 4.4.16 coprocessor firmware for the USB-based HmIP-RFUSB rf module sold by eQ3/ELV. This new "dualcopro" firmware released by eQ3 finally supports to use the HmIP-RFUSB as a dual BidCos/HomeMatic and homematicIP RF communication device and bumps it to the same feature set like the GPIO-based RPI-RF-MOD. This means, that the HmIP-RFUSB USB stick can now finally communicate with traditional "HomeMatic" devices as well as "homematicIP" devices and also supports all "Advanced homematicIP features" which are required to connect a HmIP-HAP or HmIPW-DRAP LAN gateway device.
  PLEASE NOTE: This new dualcopro firmware is only available for the HmIP-RFUSB sticks distributed by eQ3/ELV and NOT for the HmIP-RFUSB-TK which is the so-called "Telekom" version of the same RFUSB stick but meant to be used/supported only in combination with a Telekom SpeedPort router.
• integrated a new version of the generic_raw_uart kernel driver which comes with the necessary low latency uart support for allowing to use the HmIP-RFUSB as a dualcopro device. Special thanks have to go to @alexreinert for adapting his raw uart kernel driver to the new dualcopro capabilities of the HmIP-RFUSB. Without these changes and his modifications to the detect_radio_module tool, the dualcopro support for the HmIP-RFUSB could not have been implemented.
• slightly optimized the firmware update routines for the RPI-RF-MOD and HmIP-RFUSB.
• fixed a bug in the firmware update routine for a HmIP-RFUSB where an incorrect variable was used, thus a firmware update always failed (HmIP-RFUSB docker log shows "Updating Homematic RF-Hardware: HMIP-RFUSB: 4.2.14=>2.8.6, ERROR ( != 2.8.6)" #1516).
• updated the mod_authn_rega lighttpd ReGa authentication module to work with newer lighttpd versions included in buildroot 2021.11 and newer.
• updated integrated Mediola NEO Server addon to latest 2.8.0 version.

WebUI changes:

• added a new 0160-WebUI-Fix-TranslationStartRelease WebUI bugfix patch which fixes the mixed up translation for the new "Start long key press" and "Stop long key press" translations.
• added new 0159-WebUI-Fix-JSONRpcResponses WebUI bugfix patch which makes sure that all JSON-RPC functions will actually return a valid json reponse even thought they don't have any result value (null). This however seems to be required for proper json-rpc processing in the WebUI (ssh kann bei frischem System (Nightly-Snapshot) nicht sauber über die WebUI aktiviert werden #1593).
• added new 0158-WebUI-Fix-SysProtocolCharEncoding WebUI bugfix patch which fixes escape issues in the system protocol display when SysVars contain ' characters (Systemprotokoll wird nicht angezeigt / System protocol not shown #1585, fixes #1585 #1586, @jp112sdl).
• added new 0157-WebUI-Fix-WrongJSONQuotes WebUI bugfix patch which fixes incorrectly quoted JSON responses for the createBool, createEnum and createFloat JSON-RPC API calls (add patch 0157-WebUI-Fix-WrongJSONQuotes #1580, @jp112sdl).
• added new 0156-WebUI-ImprovedConfirmDeletePopup WebUI patch which improves the confirmation dialogs for sysvars & co so that the name of the object that is supposed to be deleted will be listed as well (add patch 0156-WebUI-ImprovedConfirmDeletePopup #1579, @jp112sdl).
• integrated a fix in 0097-WebUI-SetSysvarValueByUser WebUI patch which will remove the double State() uses when changing the content of a system variable via the WebUI (Ändern von SV-Werten via WebUI führt zu Mehrfacheinträgen im Systemprotokoll und Falsch-Triggern in Programmen #1568, fixes 1568 #1569, @jp112sdl).
• integrated fixes for the 0154-WebUI-DynamicResizing WebUI patch for improved dynamic resizing (fixes for dynamic resizing #1564).
• integrated a javascript error fix in 0134-WebUI-Add-ShowInternalSysvars WebUI patch which did not check if form is null before trying to get its length (Fehlermeldungen in der Browserkonsole wenn unter "Status und Bedienung > Geräte" bei eingeblendeten "systeminternen Kanälen" die HM-RCV-50 BidCoS-RF angewählt wird #1547)
• added common favicon statements also to logout.htm (Improve logout.htm in 0036-WebUI-ImprovedFavIcons.patch as well #1559).
• reworked the whole 0144-WebUI-ControlForMaintenanceChannel WebUI patch from scratch so that we can display all commonly known maintenance datapoints of all BidCos and HmIP devices correctly (Maintenance channel 0 is no longer showing temperature for HmIP-PCBS since 3.61.5.20211113 #1538).
• added new 0153-WebUI-DynamicResizing WebUI patch which replaces the cumbersome and semi-automatically generated content resizing in the WebUI with plain CSS based responsive resizing which does not require manual calculations. This should make resizing the WebUI less tricky and also potentially a bit faster (0153-WebUI-dynamic-resizing #1550, @ptweety).
• added new 0155-WebUI-Add-HmIP-RFUSB-Support WebUI patch which adds all necessary changes to get the HmIP-RFUSB supported in the WebUI as a full-fledged Advanced IP Routing Gateway device similar to a RPI-RF-MOD. This will bring up the CarrierSense measures and creates a dedicated maintenance device and :0 channel from which additional parameters can be queried.
• added new 0153-WebUI-Fix-ImproveImgUpdates WebUI patchset which uses CSS methods to load the main logo and its 'red' variant. In addition, the dot images for the alarm and service messages will only be updated if there are any changes detected. This should slightly reduce the amount of regular work to be done in a timer event (Remove img requests spam for Logo, AlarmMsg and ServiceMsg on webserver #1552, @Steinweber).
• integrated minor optimizations for 0135-WebUI-Add-ControlPanel-AdvancedSettings WebUI patch to not use Javascript statements to set document.title (0135 web UI add control panel advanced settings #1551, @jp112sdl).
• added possibility to specify a dedicated "system name" for a RaspberryMatic instance and added it to 0135-WebUI-Add-ControlPanel-AdvancedSettings WebUI patch so that a user can define an own system name now which will be shown in the browser title/tab and the startpage (WebUI: Systemname vergeben können #548, benutzerdefinierter Systemname #548 #1544, @jp112sdl).
• when uninstalling CCU-Addons an error was always reported even thought the uninstall worked out fine (Jede Deinstallation von Zusatzsoftware endet mit Fehler #1536, Fix for #1536 #1543, @jp112sdl).
• added a new 0152-WebUI-SysvarSelectionPopupInternalToggle WebUI patch which adds functionality to allow to show/hide internal system variables in program selection popups.
• fixed a minor bug in 0057-WebUI-ImprovedDutyCycleDisplay WebUI patch where a closing > tag was missing in the Dutycycle table (Invalid HTML in DutyCycle display on start page #1540).
• removed the fixed height:20px; CSS statement for the StdButton CSS class so that the buttons will automatically resize if there is not enough space for content (WebUI Buttons - LAN Gateway - Display  #1537).
• integrated a new upstream bugfix in 0146-WebUI-ImprovedStartpageLayout WebUI patch which should make the firmware update display non-clickable for non admins, thus the WebUI should not freeze anymore (Klick auf Update-Link als "Benutzer" lässt WebUI einfrieren #1533).
• integrated a fix in 0039-WebUI-Fix-Style-Glitches which brings back the accidentially hidden device highlights (Kanal-Farbmarkierung z.B. bei Tastern fehlt #1534).
• added new feature to 0146-WebUI-ImprovedStartpageLayout allowing to directly click on the time&date to bring up the respective config popup (@jp112sdl).
• added new 0151-WebUI-Add-DeviceRestoreConfigButton WebUI feature patch which implements a new "Restore Config" button placed in the device settings page and allows to restore the whole device config of a device in one run, e.g. after an old backup has been restored which contains other device config settings (@jp112sdl).
• modified 0146-WebUI-ImprovedStartpageLayout WebUI patch to allow to edit system variables also directly from the startpage and favorite display.
• added more style fixes to 0039-WebUI-Fix-Style-Glitches to use the same table header style in all tables throughout the WebUI.
• integrated a fix for incorrect standard sorting in 0145-WebUI-NaturalSorting WebUI patch (system variables in descending order (Z-A) in selection popup #1519).
• integrated missing fix for 0029-WebUI-Fix-Variable-selection-and-filter modifications WebUI patch.

Operating system changes:

• updated buildroot Linux environment to latest stable 2021.11 version.
• added support for RTL8812AU network hardware for ODROID platform (cf. rtl8812au: use buildroot provided aircrack-ng driver (#1660) home-assistant/operating-system#1669).
• improved the OVA build environment to generate *.ova files with a more VMware compatible manifest *.mf file included (cf. Adjust ova to support VMware ESXi (#826) home-assistant/operating-system#1674).
• updated qemu-guest-agent to latest 6.1.0 version for improved compatibility to OVA-based qemu-based environments.
• output ssh startup info way earlier so that potential error messages can be more intuitively linked to the start.
• updated the rpi-eeprom package to use the latest stable pieeprom-2021-11-22.bin RaspberryPi4 EEPROM version.
• updated latest stable RaspberryPi kernel which will bring all necessary RaspberryPi 2 Zero W changes/adaptions.
• reworked deploy.sh to not test for Raspbian but for a RaspberryPi system in general (Raspberrymatic on Docker install "Paket linux-headers-5.10.63-v8 kann nicht gefunden werden" #1582).
• added sysfs mount in inittab. This fixes a problem where the fsck call returned an invalid LABEL error because sysfs was not available at the time of execution of fsck.
• updated RaspberryPi firmware to latest 1.20211118 which includes the necessary upstream dtb files for the new RaspberryPi 2 Zero W.
• updated tailscale to latest 1.18.0 version.
• updated monit to latest 5.29.0 version.
• updated upstream Linux kernel to latest 5.10.84 LTS version.

👪 Contributors (alphabetically):

• @alexreinert
• @jens-maus
• @jp112sdl
• @ptweety
• @Steinweber

📝 Support:

For support on installation and help please visit the following web pages:

RaspberryMatic - Documentation 🇺🇸
RaspberryMatic - Discussions 🇺🇸
RaspberryMatic - Dokumentation 🇩🇪
RaspberryMatic - Forum 🇩🇪

📦 Download:

The following installation archives can be downloaded for different hardware platforms. To verify their integrity a sha256 checksum is provided as well. You can either upload these files using the WebUI-based update mechanism or unarchive them to e.g. flash the included *.img files on a fresh installation media (e.g. microSD card):

• RaspberryPi4 Model B, RaspberryPi Compute Module 4, RaspberryPi 400 – (installation):
  _{📦 RaspberryMatic-3.61.7.20211218-rpi4.zip
  SHA256: 8ebb1146c76a33f357c5a43ce0d14f8b2b40e50e87522bb49e37e5ff6390de59}

• CCU3, ELV-Charly, RaspberryPi3 Model B+, RaspberryPi3 Model B, RaspberryPi3 Model A+, RaspberryPi Compute Module 3, RaspberryPi Compute Module 3 lite, RaspberryPi Zero 2 W – (installation):
  _{📦 RaspberryMatic-3.61.7.20211218-rpi3.zip
  SHA256: a6177642917d12d3b924b7ffd5171fcaa800e2949736b92d8f88cda46cd2ee70
  📦 RaspberryMatic-3.61.7.20211218-ccu3.tgz (only for initial CCU3 Firmware -> RaspberryMatic Upgrade)
  SHA256: 4cbe9e3e9b471b91f33de4ae4a49aa718b34494a55dbee97b1545212872662b5}

• RaspberryPi2 Model B – (installation):
  _{📦 RaspberryMatic-3.61.7.20211218-rpi2.zip
  SHA256: fe71a6c6623b870421c4e3891cb0b24067ff6944012700cc7fed26282b89270c}

• RaspberryPi Zero W, RaspberryPi Zero, RaspberryPi Compute Module 1, RaspberryPi1 (A+/B+) – (installation):
  _{📦 RaspberryMatic-3.61.7.20211218-rpi0.zip
  SHA256: 0b5d7e2162bf6bf999593d270679f37fd2420af7cc84b2f27396035e2ca86722}

• TinkerBoard S, TinkerBoard – (installation):
  _{📦 RaspberryMatic-3.61.7.20211218-tinkerboard.zip
  SHA256: 6f75853d15294fbe0440abcd0edfef3ed92efece1bbf31d7d892ea82a9f9296c}

• ODROID-N2/N2+/C4/C2 – (installation):
  _{📦 RaspberryMatic-3.61.7.20211218-odroid-n2.zip
  SHA256: 791eab64b69b5a6cdcc24ec1ec7b3b55b84a24b5c7147e15da28f92a0674638c
  📦 RaspberryMatic-3.61.7.20211218-odroid-c4.zip
  SHA256: 07d269ed9c64134b668754943875dc8166f6c70665650848ffb8c420cdd2cee7
  📦 RaspberryMatic-3.61.7.20211218-odroid-c2.zip
  SHA256: 1f4a76dfa6ef6e14f0cff5627c61bb964e4cc292ab6fdbcb78b1ce61f134c207}

• Intel NUC – (installation):
  _{📦 RaspberryMatic-3.61.7.20211218-intelnuc.zip
  SHA256: 60d77785d52c871e2c90e58c478e58b2b28ee2af1900c97511773b95ec060315}

• Open Virtual Appliance (OVA) – (ESXi, Proxmox, VirtualBox, Synology, QNAP, Workstation Player, QEmu, UNRAID, HyperV):
  _{📦 RaspberryMatic-3.61.7.20211218-ova.zip
  SHA256: 3defca425bbcc0ae38878139c4291b887a3161040f6a4afc3338bf128944eb3b
  📦 RaspberryMatic-3.61.7.20211218.ova (only for initial OVA installation)
  SHA256: 32af8736a1f0ff380bb57c69c3e2f195fb53a2a7017d349646d4defd2c98b68f}

• Docker / Open Container Initiative (OCI) – virtual appliance (installation):
  _{📦 RaspberryMatic-3.61.7.20211218-oci_amd64.tgz (amd64/x86_64)
  SHA256: c048fc1c82282688e3f0a400bab8a15d9119d54ead29aa2cd60c70ad724600be
  📦 RaspberryMatic-3.61.7.20211218-oci_arm64.tgz (arm64/aarch64)
  SHA256: ec4ac299ca35fc993347fc987ae730d0c1770ede8148d9f23ed8bee1b3aa82f0
  📦 RaspberryMatic-3.61.7.20211218-oci_arm.tgz (arm/armhf)
  SHA256: b6690677e0fa005bf6065746a006ea1e385e23829fce87ce6f87dfa0e7960a4a}

• Kubernetes / K8s – virtual appliance:
  _{see install documentation}

• Home Assistant Add-on – virtual appliance:
  _{see install documentation}

---

This discussion was created from the release RaspberryMatic 3.61.7.20211218.