Run uStreamer via a launcher

[mtlynch]

This changes the uStreamer installation so that the uStreamer systemd service runs a uStreamer launcher script rather than running uStreamer directly.

The launcher script, at runtime, reads a set of configuration files, translates those configuration files to uStreamer's command-line flags, and launches uStreamer with those flags.

In the previous implementation of this role, clients needed to re-run Ansible in order to change any of uStreamer's settings (e.g., frame rate, quality) because Ansible was responsible for generating the command-line string to launch uStreamer.

In this implementation, clients can change uStreamer's settings without using Ansible. Clients can simply change settings in the uStreamer launcher's config files and restart the uStreamer service. When the service restarts, the uStreamer launcher will pick up the new configuration.

This reduces the time to change video settings by 91%.

See a rough proof of concept in TinyPilot Pro: https://github.com/tiny-pilot/tinypilot-pro/pull/701

Implementation notes

• files/launch deliberately doesn't use Ansible role variables because I want to make it easy to move this script to Debian packages in the future.
• The design of having the config files as ordered files in /opt/ustreamer-launcher/configs.d/ is to make it easier to translate the logic to Debian packages in the future.
  • The uStreamer Debian package will be responsible for placing /opt/ustreamer-launcher/configs.d/000-defaults.yml.
  • The TinyPilot Debian package will be responsible for placing /opt/ustreamer-launcher/configs.d/100-tinypilot.yml.
  • This is similar to how nginx works with sites-enabled so that different Debian packages can own files that affect uStreamer's behavior without colliding on trying to own the same files.
• I'm leaving in a debug play, which we normally don't do, but I think we should take advantage of it more.
  • It makes it a lot easier to debug the role, so it's like logging that's just helpful even when we're not actively debugging.
• To avoid complicated coordination of merge order between this PR and the TinyPilot PR, we're leaving in the systemd-config tags and tracking their removal in Remove systemd-config tags #90

Benefits

If we switch TinyPilot over to using this role (which should be pretty straightforward), we get a slew of benefits:

• Increases performance by 91%
  • In my tests, changing uStreamer settings through TinyPilot's video settings dialog went from 17.9s to 1.6s
• Reduces our dependency on Ansible
  • We are adding some more Ansible code here, but I wrote it so that it would be easy to translate to a uStreamer Debian package in the future.
  • We're getting rid of a big dependency in that we no longer need Ansible to change uStreamer settings, which was an annoying obstacle before.
• Decreases fragility
  • It used to be that the /opt/tinypilot-privileged/scripts/update-video-settings script pointed to a file in /opt/tinypilot-updater, which was confusing and brittle.
    • We eventually want to get to a place where the installer files don't need to hang around after installation/update is finished, and this gets us closer to that.
• Backwards-compatible
  • Code written for the previous version of the role should still work on this one, though it's easy for clients to take advantage of the new design.

Drawbacks

• Values from /home/tinypilot/settings.yml will appear in both 000-defaults.yml and 100-tinypilot.yml
  • This isn't a problem with this role but more in how we've implemented the TinyPilot Ansible role on top of it.
  • The TinyPilot role reads from /home/tinypilot/settings.yml and uses those values in the Ansible execution of the TinyPilot and uStreamer role.
  • That means if there are uStreamer variables in /home/tinypilot/settings.yml, they get copied to /opt/ustreamer-launcher/configs.d/000-defaults.yml whenever the uStreamer role runs.
  • The TinyPilot Debian installer also symlinks /opt/ustreamer-launcher/configs.d/100-tinypilot.yml to /home/tinypilot/settings.yml because legacy installations wrote their settings to settings.yml, and it's hard to change at this point.
  • Our design accounts for this, as 100-tinypilot.yml overrides settings in 000-defaults.yml, but it's still a bit untidy.

[mtlynch]

Automated comment from CodeApprove ➜

⏳ @jdeanwallace please review this Pull Request

[jdeanwallace]

Automated comment from CodeApprove ➜

---

In: files/launch:

> Line 55
  yaml_value="$(cat /opt/ustreamer-launcher/configs.d/* | yq "${yaml_path}")"

I'm confused about how this bash command works. I did some local testing with multiple yaml files and yq doesn't merge the yaml files by default:

$ ls /tmp/*.yml
/tmp/a.yml  /tmp/b.yml
$ cat /tmp/*.yml
---
ustreamer_persistent: yes

---
ustreamer_persistent: off

$ cat /tmp/*.yml | yq '.ustreamer_persistent'
yes
---
off

I followed these yq docs to merge all the yaml files together:

$ cat /tmp/*.yml | yq eval-all '. as $item ireduce ({}; . * $item ) | .ustreamer_persistent'
off

---

In: files/launch:

> Line 111
/opt/ustreamer/ustreamer "${USTREAMER_ARGS[@]}"

Can we use exec here so that it replaces the current process?

---

In: tasks/install_launcher.yml:

> Line 9
    ustreamer_yq_architecture: "{% if ansible_architecture == 'x86_64' %}amd64{% elif ansible_architecture == 'armv7l' %}arm{% elif ansible_architecture == 'aarch64' %}arm64{% else %}{{ ansible_architecture }}{% endif %}"

(optional) To avoid the if-else statements, we could also define the architecture mapping as a dictionary. For example:

# This can be defined as a fact or as a default value in `defaults/main.yml`
- name: define ansible to yq architecture mapping
  set_fact:
    ustreamer_yq_arch_map:
       x86_64: amd64
       armv7l: arm
       aarch64: arm64

# This can be defined as a fact or as a default value in `defaults/main.yml`
- name: canonicalize yq binary architecture
  set_fact:
    ustreamer_yq_arch: "{{ ustreamer_yq_arch_map[ansible_architecture] | default(ansible_architecture) }}"

---

In: tasks/install_launcher.yml:

> Line 26
- name: create uStreamer launcher configs folder

(optional) We could also collapse the creation of the above 2 directories into a single task using loop. For example:

- name: create uStreamer launcher directories
  template:
    path: "{{ item }}"
    state: directory
    owner: "{{ ustreamer_user }}"
    group: "{{ ustreamer_group }}"
  loop:
    - "{{ ustreamer_launcher_dir }}"
    - "{{ ustreamer_launcher_configs_dir }}"

---

In: vars/main.yml:

> Line 15
ustreamer_launcher_script: /opt/ustreamer-launcher/launch

Can we use the ustreamer_launcher_dir variable to build the path? Something like:

ustreamer_launcher_script: "{{ ustreamer_launcher_dir }}/launch"

---

👀 @mtlynch it's your turn please take a look

[mtlynch]

Automated comment from CodeApprove ➜

---

In: files/launch:
Whoops, that's embarrassing. Thanks for catching that!

I tested for this issue and found a similar solution, but I must have just applied it on the command-line during testing and forgot to integrate it back into the script.

I blame it on holiday brain.

---

In: files/launch:
Sure, that sounds fine. What's the advantage in this case? I get the idea of replacing the process as opposed to starting a subshell, but I'm not clear on the implications.

---

In: tasks/install_launcher.yml:
Oh, nice! I like that. Fixed.

---

In: tasks/install_launcher.yml:
Oh, yeah this is so much cleaner. Thanks!

---

In: vars/main.yml:
Sure, fixed.

---

👀 @jdeanwallace it's your turn please take a look

[jdeanwallace]

Automated comment from CodeApprove ➜

⏳ Approval Pending (3 unresolved comments)
Approval will be granted automatically when all comments are resolved

Looks good and I've tested it on device. Changing video settings is super fast now!

---

In: files/launch:
From what I understand, using exec cuts out the middleman process and preserves termination signals which I think is important for systemd to make restart decisions.

---

In: files/launch:

> Line 45
MERGED_YAML="$(yq eval-all '. as $item ireduce ({}; . * $item )' \

(nit): Can we mark MERGED_YAML as readonly after setting its value?

---

In: files/launch:

> Line 82
     [[ "${yaml_value_lowercase}" != 'no' ]]; then

(nit, optional): Would this if statement become clearer if we matched the positive condition? For example:

  if [[ "${yaml_value_lowercase}" == 'true' ]] && \
     [[ "${yaml_value_lowercase}" == 'on' ]] && \
     [[ "${yaml_value_lowercase}" == 'yes' ]] && \
     [[ "${yaml_value_lowercase}" == 'y' ]]; then

(Reference: Possible yaml boolean values)

---

In: tasks/install_launcher.yml:

> Line 79
    mode: '0755'

Can we restart uStreamer if the launcher file changes? Something like:

notify: restart uStreamer

---

👀 @mtlynch it's your turn please take a look

[mtlynch]

Automated comment from CodeApprove ➜

---

In: files/launch:
Oh yeah, that's a good idea. I was thinking that anything that isn't false is implicitly true (e.g., ustreamer_persistent: banana) counts as true, but we don't need to be so permissive, and it's clearer to just allow the explicit things we expect.

---

In: files/launch:

> Line 45
MERGED_YAML="$(yq eval-all '. as $item ireduce ({}; . * $item )' \

Ah, right. Fixed.

---

In: tasks/install_launcher.yml:

> Line 81
    mode: '0755'

Oh, good point. I added it to the "write uStreamer runtime variables to file" play as well.

[jdeanwallace]

Automated comment from CodeApprove ➜

Approved: I have approved this change on CodeApprove and all of my comments have been resolved.