Supervised: AppArmor issue with Raspberrymatic addon after upgrading to Debian 12

[sbyx]

Describe the issue you are experiencing

Since upgrading from Debian 11 to 12 I'm encountering an issue with the Raspberrymatic addon which is unable to start due to an issue with AppArmor. This was running fine with Debian 11 and identical HA components and same Addon version.

This is from the addon log:

Jun 30 16:24:31 de838cd8-raspberrymatic user.err : Could not create socket
Jun 30 16:24:57 de838cd8-raspberrymatic user.err rfd: Could not bind to TCP port 32001

Disabling apparmor with "apparmor=0" onm kernel command line works around this issue and makes the addon run correctly again.

What type of installation are you running?

Home Assistant Supervised

Which operating system are you running on?

Debian

Steps to reproduce the issue

1. Install HA supervised with Debian 11
2. Upgrade to Debian 12
3. Install latest homeassistant-supervised.deb
4. Failure appears

Anything in the Supervisor logs that might be useful for us?

No error is supervisor log.
Related log entries for this addon:


23-06-30 16:52:11 INFO (SyncWorker_1) [supervisor.docker.addon] Starting Docker add-on ghcr.io/jens-maus/raspberrymatic with version 3.69.7.20230626
23-06-30 16:52:37 INFO (MainThread) [supervisor.hardware.monitor] Detecting HardwareAction.ADD hardware /dev/eq3loop - None
23-06-30 16:52:37 INFO (MainThread) [supervisor.docker.addon] Added cgroup permissions 'c 238:0 rwm' for device /dev/eq3loop to addon_de838cd8_raspberrymatic
23-06-30 16:52:40 INFO (MainThread) [supervisor.hardware.monitor] Detecting HardwareAction.ADD hardware /dev/mmd_hmip - None
23-06-30 16:52:40 INFO (MainThread) [supervisor.hardware.monitor] Detecting HardwareAction.ADD hardware /dev/mmd_bidcos - None
23-06-30 16:52:40 INFO (MainThread) [supervisor.docker.addon] Added cgroup permissions 'c 238:1 rwm' for device /dev/mmd_hmip to addon_de838cd8_raspberrymatic
23-06-30 16:52:40 INFO (MainThread) [supervisor.docker.addon] Added cgroup permissions 'c 238:2 rwm' for device /dev/mmd_bidcos to addon_de838cd8_raspberrymatic

### System Health information

## System Information

version | core-2023.6.3
-- | --
installation_type | Home Assistant Supervised
dev | false
hassio | true
docker | true
user | root
virtualenv | false
python_version | 3.11.4
os_name | Linux
os_version | 6.1.0-9-amd64
arch | x86_64
timezone | Europe/Berlin
config_dir | /config

<details><summary>Home Assistant Community Store</summary>

GitHub API | ok
-- | --
GitHub Content | ok
GitHub Web | ok
GitHub API Calls Remaining | 4096
Installed Version | 1.32.1
Stage | running
Available Repositories | 1287
Downloaded Repositories | 16

</details>

<details><summary>Home Assistant Cloud</summary>

logged_in | true
-- | --
subscription_expiration | 10. Oktober 2023 um 02:00
relayer_connected | true
relayer_region | eu-central-1
remote_enabled | true
remote_connected | true
alexa_enabled | false
google_enabled | true
remote_server | eu-central-1-4.ui.nabu.casa
certificate_status | ready
can_reach_cert_server | ok
can_reach_cloud_auth | ok
can_reach_cloud | ok

</details>

<details><summary>Home Assistant Supervisor</summary>

host_os | Debian GNU/Linux 12 (bookworm)
-- | --
update_channel | stable
supervisor_version | supervisor-2023.06.4
agent_version | 1.5.1
docker_version | 20.10.24+dfsg1
disk_total | 467.0 GB
disk_used | 13.8 GB
healthy | true
supported | true
supervisor_api | ok
version_api | ok
installed_addons | deCONZ (6.20.0), RaspberryMatic CCU (3.69.7.20230626), Mosquitto broker (6.2.1), Studio Code Server (5.8.0), Home Assistant Google Drive Backup (0.111.1), Watchdog Dev (0.0.10), ESPHome (2023.6.3)

</details>

<details><summary>Dashboards</summary>

dashboards | 2
-- | --
resources | 6
views | 19
mode | storage

</details>

<details><summary>Recorder</summary>

oldest_recorder_run | 23. Juni 2023 um 09:58
-- | --
current_recorder_run | 30. Juni 2023 um 16:48
estimated_db_size | 1209.51 MiB
database_engine | sqlite
database_version | 3.41.2

</details>

### Supervisor diagnostics

_No response_

### Additional information

_No response_

[sbyx]

Adding apparmor kernel log:

[  335.095168] audit: type=1400 audit(1688207445.848:417): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-1" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  335.095822] audit: type=1400 audit(1688207445.848:418): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-2" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  335.097214] audit: type=1400 audit(1688207445.848:419): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-1" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  335.098668] audit: type=1400 audit(1688207445.852:420): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-2" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  340.090651] audit: type=1400 audit(1688207450.844:421): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-2" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  340.091015] audit: type=1400 audit(1688207450.844:422): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-1" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  340.092491] audit: type=1400 audit(1688207450.844:423): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-2" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  340.093039] audit: type=1400 audit(1688207450.844:424): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-1" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  340.094404] audit: type=1400 audit(1688207450.848:425): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-2" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
[  340.094805] audit: type=1400 audit(1688207450.848:426): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-1" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"

[agners]

I am assuming the cgroup v1 switch successfully works on Debian 12 (e.g. /etc/default/grub has systemd.unified_cgroup_hierarchy=false)?

It seems that some kind of socket operation is denied.

[  335.095168] audit: type=1400 audit(1688207445.848:417): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-1" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"

@jens-maus maybe you have some insights here?

[sbyx]

Yes it is not a cgroup issue, I checked /proc/cmdline. It is weird that it worked with identical config and components on Debian 11. Main difference between 11 and 12 seems to be Apparmor 2.x vs 3. Especially since I was running same kernel versions (I e. back ported 6.10 kernel for Bullseye before).

[agners]

Could be related to AppArmor 3.x, however, we do use AppArmor 3.x on HAOS as well, and I haven't seen such reports.

Also, I don't think that the regular AppArmor profiles are used by Docker.

What could be a issue is the new version of Docker: What version of Docker are you using? E.g. what is docker info reporting?

[jens-maus]

It seems that some kind of socket operation is denied.

[  335.095168] audit: type=1400 audit(1688207445.848:417): apparmor="DENIED" operation="create" profile="de838cd8_raspberrymatic" pid=11312 comm="Timer-1" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"

@jens-maus maybe you have some insights here?

No sorry, haven't seen this yet and have no idea what might be the root cause. As usual: HA Supervised installations are unfortunately very prone to these kind of OS related issues and especially in cases where new major versions like Debian11->12 are arriving ppl commonly popup with these kind of issues. That's why HA supervised installations should IMHO really be discouraged more and more and suggestions should be raised that ppl should use HomeAsssistantOS instead ;)

[sbyx]

I gave up on it already and switched to HAOS in VM in the meantime which works fine. If you don't want to look into it further we can close this.

[agners]

I gave up on it already and switched to HAOS in VM in the meantime which works fine. If you don't want to look into it further we can close this.

Thanks for the update.

Yeah without exact information e.g. what Docker version is in use its kinda hard to investigate. I'll close it for now, if someone else experiences the same issue, feel free to speak up so we can resume investigation.

[lmagyar]

FYI: Downgrading AppArmor from 3.0.8-3 to 2.13.6-10 maybe can solve this issue also (it helped in a similar situation), for steps see lmagyar/homeassistant-addon-mariadb-inmemory#44 (comment)

[sbyx]

Thanks @lmagyar that confirms my initial suspicion.

[sbyx]

@lmagyar you may want to try adding a line "network," to your apparmor policy (or more specific network rules). This is what seems to do the trick for me. I don't know why it wasn't necessary on Debian 11 and is on 12 now.

[jens-maus]

@sbyx Please provide more detail on the specific network, change you did to your apparmor policy. Please demonstrate by example.

[sbyx]

In short I moved from HAOS/Supervised to individually managed containers with podman on top of Debian 12. While trying to run your raspberrymatic container unprivileged I essentially ran into the same issue as with HA supervised under Debian 12 (permission denied when binding / connecting sockets). What did the trick was adding a "network" directive into the apparmor profile. See my example here (5th line in the profile block):

#include <tunables/global>

profile ccu flags=(attach_disconnected,mediate_deleted) {
  #include <abstractions/base>

  file,
  mount fstype in (tmpfs, devtmpfs),
  network,

  capability chown,
  capability dac_override,
  capability net_bind_service,
  capability setgid,
  capability setuid,
  capability sys_admin,
  capability sys_nice,

  /bin/** ix,
  /usr/bin/** ix,
}

[agners]

There are reports of changes in how AppArmor behaves. It seems that with AppArmor 3.0 network rules got enabled in Debian 12/Bookworm: https://forums.whonix.org/t/sdwdate-apparmor-profile-broken-on-bookworm/13521

What I do wonder is why this is not showing up on Home Assistant OS. After all, Home Assistant OS 10.x is using AppArmor 3.1.