[mqtt] Simplify homeassistant thing types, and use AbstractStorageBasedTypeProvider #16600

ccutrer · 2024-04-01T16:05:11Z

This is a rather large PR, but is built of a few commits with essentially two distinct goals. I have not tested them individually, and there are lots of interactions between the two goals, which is why I did not disentangle them.

This first goal is to simplify and optimize thing and channel types for mqtt.homeassistant. This fixes #14295. This in itself has two user visible changes:

Thing types have not actually changed, but thing UIDs have. Auto Discovery will no longer use the per-thing thing type in the thing ID, but will use the "base" thing type (available in the binding's XML) of mqtt:homeassistant in its ID. In practice, newly discovered things will get UIDs such as mqtt:homeassistant:mosquitto:awair-element-63611 instead of mqtt:homeassistant_awair_2Delement_2D63611:mosquitto:awair_2Delement_2D63611 (unnecessary escaping has also been removed). Existing things will not change their UID. I'm not sure if this is considered a breaking change, since existing things will not change at all. But if you were to delete the thing, and re-add it from discovery, it would end up with a different ID. This new behavior matches how mqtt.homie works.
Channel IDs have been simplified. First, if a component has no name (in the Home Assistant configuration), it will not generate a channel group at all. Second, the channel group name is now only the component's name, instead of including the entire thing ID in the channel group, it is now only the component name (which already has to be unique within a Home Assistant device, so not a problem there). The channel group type (which is unseen by the user) remains the fully unique id, which includes the thing id. Third, if a component only has a single channel, it also doesn't create a channel group, just a single channel for that component without a group.
For this change, I did a bit of hoop-jumping to pass a property through from auto-discovery, so that existing things will not have their channels change, only newly discovered and created things will use this new channel type. Again, I'm not sure if this counts as a breaking change. If it does, I'm possibly inclined to not use this hinky property, and simply apply the new channel IDs to all things. An example change is from mqtt:homeassistant_zigbee2mqtt_5F0x8cf681fffe32e58e:mosquitto:zigbee2mqtt_5F0x8cf681fffe32e58e:0x8cf681fffe32e58e_5Fbattery_5Fzigbee2mqtt#sensor to mqtt:homeassistant:mosquitto:zigbee2mqtt_5F0x8cf681fffe32e58e:sensor_battery

The second goal is relatively simple - use AbstractStorageBasedTypeProvider. Because the thing and channel group types are dynamic, they don't exist in the binding's XML. But since openHAB 4.0, if a thing type isn't available at startup, it will leave the thing in an unknown state for two minutes before it initializes it. By using AbstractStorageBasedTypeProvider, the dynamic types will be persisted to JSON storage, and thus available immediately at startup, avoiding the two minute delay. This fixes #15577

...ing.mqtt.generic/src/main/java/org/openhab/binding/mqtt/generic/MqttChannelTypeProvider.java

...c/main/java/org/openhab/binding/mqtt/homeassistant/internal/component/AbstractComponent.java

....binding.mqtt.homie/src/main/java/org/openhab/binding/mqtt/homie/internal/homie300/Node.java

...c/main/java/org/openhab/binding/mqtt/homeassistant/internal/component/AbstractComponent.java

lolodomo · 2024-04-06T16:22:16Z

@ccutrer : did you notice the build failed ?

ccutrer · 2024-04-06T16:23:34Z

I did. And I've gotten to the point that I can reproduce the failures locally, but have not yet had time to troubleshoot them (or address @antroids comments).

ccutrer

I've addressed (most) the review comments, but not looked at itests. The suggested changes have also caused (non-test) test failures - I think related to some of the foreach changes, possibly due to needing to mock different methods. I don't have time today to look into any of that, but will when I can

...c/main/java/org/openhab/binding/mqtt/homeassistant/internal/component/AbstractComponent.java

lsiepel · 2024-06-14T19:24:50Z

Gentle ping @antroids if im correct @ccutrer is waiting on your response.

ccutrer · 2024-06-14T19:27:02Z

While I do have an outstanding question (about preferred amount of backwards compatibility), the ball is in my court to fix unit tests after recent changes to address review comments. Unfortunately I don't know when I'll have time to dig in to this PR. Life has been excessively busy lately, leaving little time for hobbies.

openhab-bot · 2024-07-09T17:48:06Z

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/mqtt-binding-beta/157129/1

Instead, use state descriptions for the varying parts, which means that there is not an explosion of channel types for every single Home Assistant channel, which can cause performance issues if you have a lot of devices/channels Signed-off-by: Cody Cutrer <cody@cutrer.us>

This has two important impacts, and one not-as-important-but-still- beneficial impact: * ThingTypes are not created for Things that are never actually created, but only discovered, reducing bloat. * Avoids warnings about things types not being in the registry * Thing IDs end up like mqtt:homeassistant:broker:mydevice instead of mqtt:homeassistant_mydevice:broker:mydevice Note that already-created things will maintain their original ID, and it's only new things created from auto-discovery that will have the new, shorter thing IDs. Signed-off-by: Cody Cutrer <cody@cutrer.us>

This is triggered by a property only set by discovery, so will only apply to newly created things. Channel groups are now named after simply the component type and object_id, if possible, and without additional encoding, instead off encoding the unique_id. Most devices will encode the component type and device identifiers into the unique_id, and of arbitrary format, so it is far more complicated than necessary, when a channel group is already only within the scope of the thing/device. Many component types that can only ever have one channel (button, switch, sensor, select, scene, etc.) will not be in a channel group anymore. In this case their channel id is what their group id would have been, without the repetitiveness of "sensor_<unique_id>#sensor". Signed-off-by: Cody Cutrer <cody@cutrer.us>

Includes both homie and homeassistant dynamic things. It also fixes all ordering issues in both bindings in order to be consistent in how types are persisted: * For Homie, Nodes (channel groups) and Properties (channels) are ordered in the way they are defined in $nodes and $properties * For Home Assistant, Components are ordered by label. This includes both single channel components that are not in a channel group, as well as channel groups. We also ensure that on the Thing itself non-grouped channels consistently sort before grouped channels. Signed-off-by: Cody Cutrer <cody@cutrer.us>

Signed-off-by: Cody Cutrer <cody@cutrer.us>

the ternary accidentally got swapped Signed-off-by: Cody Cutrer <cody@cutrer.us>

Signed-off-by: Cody Cutrer <cody@cutrer.us>

ccutrer

I rebased to get changes from main, and tests should pass now. 🤞 this is ready to go.

...c/main/java/org/openhab/binding/mqtt/homeassistant/internal/component/AbstractComponent.java

openhab-bot · 2024-07-26T20:16:19Z

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/problem-adding-a-switch-to-a-dimmer/155304/19

lsiepel

LGTM.

The binding documentation could use some examples or maybe file based configuration. Not sure if that makes sense, but this PR might be a good moment to update/extend the documentation.
Please create a notice about the semi-non-breaking change in the https://github.com/openhab/openhab-distro/pulls
Do you want to create a PR with removing the obsolete parts and have that as a draft waiting for the next stable or how do you want to make sure a follow up is taking place?

I'll wait for @antroids to confirm all his comments are addressed.

ccutrer · 2024-07-30T17:17:02Z

The binding documentation could use some examples or maybe file based configuration. Not sure if that makes sense, but this PR might be a good moment to update/extend the documentation.

Indeed, it's quite light on examples. But I'm suspicious that file-based configuration does not work for Home Assistant at all. And I'm also aware of other issues surrounding discovery that I want to fix. I plan on investigating all that - and updating the docs as I confirm things work - after this PR is submitted. This PR is already large and complex enough, most anything else I do will conflict with it. It's also difficult to test because it relies on changes across both mqtt.generic and mqtt.homeassistant, so I can't just update a single bundle - you have to update them all, and then restart openHAB.

Please create a notice about the semi-non-breaking change in the https://github.com/openhab/openhab-distro/pulls

openhab/openhab-distro#1675

Do you want to create a PR with removing the obsolete parts and have that as a draft waiting for the next stable or how do you want to make sure a follow up is taking place?

I've got it ready at ccutrer@dfe751a. Once this PR merges, I'll rebase it, then open a draft PR against this repo.

ccutrer · 2024-08-12T14:59:06Z

ping @antroids

Signed-off-by: Cody Cutrer <cody@cutrer.us> Co-Authored-By: Anton Kharuzhy <publicantroids@gmail.com>

.../java/org/openhab/binding/mqtt/homeassistant/internal/handler/HomeAssistantThingHandler.java

...ssistant/src/main/java/org/openhab/binding/mqtt/homeassistant/internal/ComponentChannel.java

Signed-off-by: Cody Cutrer <cody@cutrer.us>

antroids

LGTM
Let's get rid of this PR 😁

I actually still use file-based configuration, but IMO it's just a feature for backward compatibility.

…edTypeProvider (openhab#16600) * [mqtt.homeassistant] don't dynamically generate channel types Signed-off-by: Cody Cutrer <cody@cutrer.us>

…edTypeProvider (openhab#16600) * [mqtt.homeassistant] don't dynamically generate channel types Signed-off-by: Cody Cutrer <cody@cutrer.us> Signed-off-by: Patrik Gfeller <patrik.gfeller@proton.me>

…edTypeProvider (openhab#16600) * [mqtt.homeassistant] don't dynamically generate channel types Signed-off-by: Cody Cutrer <cody@cutrer.us>

ccutrer requested a review from antroids as a code owner April 1, 2024 16:05

lolodomo added the enhancement An enhancement or new feature for an existing add-on label Apr 3, 2024

antroids requested changes Apr 3, 2024

View reviewed changes

ccutrer commented Apr 18, 2024

View reviewed changes

ccutrer added 8 commits July 26, 2024 13:05

fix spotless warning

c377dcd

Signed-off-by: Cody Cutrer <cody@cutrer.us>

address itests compilation errors

cb2a0ec

Signed-off-by: Cody Cutrer <cody@cutrer.us>

address review comments

9c1f7f0

Signed-off-by: Cody Cutrer <cody@cutrer.us>

fix errors introduced by last commit

c835743

the ternary accidentally got swapped Signed-off-by: Cody Cutrer <cody@cutrer.us>

ccutrer force-pushed the mqtt-homeassistant-simplify-types branch from b54d5da to c835743 Compare July 26, 2024 19:19

fix itest for new channel names

d2685f2

Signed-off-by: Cody Cutrer <cody@cutrer.us>

ccutrer commented Jul 26, 2024

View reviewed changes

...c/main/java/org/openhab/binding/mqtt/homeassistant/internal/component/AbstractComponent.java Show resolved Hide resolved

lsiepel requested a review from antroids July 30, 2024 11:18

lsiepel approved these changes Jul 30, 2024

View reviewed changes

ccutrer mentioned this pull request Jul 30, 2024

[mqtt.homeassistant] Add future breaking change alert for Home Assistant openhab/openhab-distro#1675

Merged

This was referenced Jul 30, 2024

[mqtt.homeassistant] Discovered things never get removed from the inbox #17183

Open

[mqtt][homeassistant][zigbee2mqtt] State missing in homeassistant #16633

Open

[mqtt.homeassistant] Color channel not created for Hue lights via zigbee2mqtt #17184

Closed

Use ChannelType ID constants, not strings

a88d40b

Signed-off-by: Cody Cutrer <cody@cutrer.us> Co-Authored-By: Anton Kharuzhy <publicantroids@gmail.com>

antroids requested changes Aug 12, 2024

View reviewed changes

.../java/org/openhab/binding/mqtt/homeassistant/internal/handler/HomeAssistantThingHandler.java Outdated Show resolved Hide resolved

...ssistant/src/main/java/org/openhab/binding/mqtt/homeassistant/internal/ComponentChannel.java Outdated Show resolved Hide resolved

remove unused variables

4c90379

Signed-off-by: Cody Cutrer <cody@cutrer.us>

antroids approved these changes Aug 12, 2024

View reviewed changes

lsiepel merged commit 73a3f15 into openhab:main Aug 12, 2024
5 checks passed

lsiepel added this to the 4.3 milestone Aug 12, 2024

ccutrer deleted the mqtt-homeassistant-simplify-types branch August 12, 2024 20:29

ccutrer mentioned this pull request Oct 2, 2024

[mqtt.homeassistant] Fix newStyleChannels #17491

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[mqtt] Simplify homeassistant thing types, and use AbstractStorageBasedTypeProvider #16600

[mqtt] Simplify homeassistant thing types, and use AbstractStorageBasedTypeProvider #16600

ccutrer commented Apr 1, 2024

lolodomo commented Apr 6, 2024

ccutrer commented Apr 6, 2024

ccutrer left a comment

lsiepel commented Jun 14, 2024 •

edited

Loading

ccutrer commented Jun 14, 2024

openhab-bot commented Jul 9, 2024

ccutrer left a comment

openhab-bot commented Jul 26, 2024

lsiepel left a comment •

edited

Loading

ccutrer commented Jul 30, 2024

ccutrer commented Aug 12, 2024

antroids left a comment

[mqtt] Simplify homeassistant thing types, and use AbstractStorageBasedTypeProvider #16600

[mqtt] Simplify homeassistant thing types, and use AbstractStorageBasedTypeProvider #16600

Conversation

ccutrer commented Apr 1, 2024

lolodomo commented Apr 6, 2024

ccutrer commented Apr 6, 2024

ccutrer left a comment

Choose a reason for hiding this comment

lsiepel commented Jun 14, 2024 • edited Loading

ccutrer commented Jun 14, 2024

openhab-bot commented Jul 9, 2024

ccutrer left a comment

Choose a reason for hiding this comment

openhab-bot commented Jul 26, 2024

lsiepel left a comment • edited Loading

Choose a reason for hiding this comment

ccutrer commented Jul 30, 2024

ccutrer commented Aug 12, 2024

antroids left a comment

Choose a reason for hiding this comment

lsiepel commented Jun 14, 2024 •

edited

Loading

lsiepel left a comment •

edited

Loading