Use bullseye templates on 4.1 and and use apt-test+nightlies repos for dev apt-test+main packages for staging

[eaon]

Description of Changes

Towards #600

Draft PR for now, but this will include a collection of 4.1 bullseye template related changes to salt state logic, as well as @legoktm's work from the bullseye-nightlies branch. It also removes the qa-switch.sh utility (at least for now) as it would introduce unnecessary complexity for keeping 4.0 compatibility, and at least for this upcoming stage of testing it isn't useful to us and could probably be implemented in a cleaner way if we need it again in the future.

Fixes #787

Testing

Qubes OS R4.1 only:

• make dev installs without errors, with bullseye based templates which in turn use bullseye+nightlies repositories
• make test in dom0 passes except for 1 failure about a securedrop-workstation-grsec that is not up to date (seems to be a nightly package/apt problem, the package hasn't been changed, there's no negative effects to this failure)

Checklist

• I have updated MANIFEST.in and rpm-build/SPECS/securedrop-workstation-dom0-config.spec
• I would appreciate help with the documentation

[sssoleileraaa]

Test Plan

• make dev installs
• make test passes for both 4.0 and 4.1
  ^ Should be sufficient before we enter phase 1 QA, right?

[sssoleileraaa]

https://apt-test.freedom.press/pool/nightlies/x/xpp/ is up

[sssoleileraaa]

Deployment

• Once merged, follow https://github.com/freedomofpress/securedrop-workstation/wiki/Building-securedrop-workstation-dom0-config-RPM-package to get the new dom0 config package up on yum-test
• Prod deployment is being tracked in Release SDW template and dom0-config for Qubes 4.1 securedrop-yum-prod#33

[cfm]

As of 62224e7:

Qubes OS R4.0

• make dev installs without errors, with buster based templates which in turn use buster+nightlies repositories

Exits 20:

Add SecureDrop export device handling to sys-usb
sys-usb:
  
  Summary for sys-usb
  -----------
  Succeeded: 0
  Failed:   0
  -----------
  Total states run:    0
  Total run time:  0.000 ms

• make test in dom0 passes

Qubes OS R4.1

• make dev installs without errors, with bullseye based templates which in turn use bullseye+nightlies repositories
• make test in dom0 passes

With the following patch, passes except for tests:

• test_sd_export_package_installed
• test_mime_types
• test_all_sd_vms_uptodate
• test_debian_keyring_config

diff --git a/tests/test_dom0_rpm_repo.py b/tests/test_dom0_rpm_repo.py
index 99cb395..5362e58 100644
--- a/tests/test_dom0_rpm_repo.py
+++ b/tests/test_dom0_rpm_repo.py
@@ -2,6 +2,14 @@ import json
 import unittest
 
 
+DEBIAN_VERSION = "bullseye"
+FEDORA_VERSION = "f32"
+with open("/etc/qubes-release") as qubes_release:
+    if "R4.0" in qubes_release.read():
+        DEBIAN_VERSION = "buster"
+        FEDORA_VERSION = "f25"
+
+
 class SD_Dom0_Rpm_Repo_Tests(unittest.TestCase):
 
     pubkey_wanted = ""
@@ -9,8 +17,8 @@ class SD_Dom0_Rpm_Repo_Tests(unittest.TestCase):
     pubkey_actual = "/etc/pki/rpm-gpg/RPM-GPG-KEY-securedrop-workstation"  # noqa
     pubkey_wanted_prod = "sd-workstation/securedrop-release-signing-pubkey-2021.asc"
     pubkey_wanted_test = "sd-workstation/apt-test-pubkey.asc"
-    yum_repo_url_prod = "https://yum.securedrop.org/workstation/dom0/f25"
-    yum_repo_url_test = "https://yum-test.securedrop.org/workstation/dom0/f25"
+    yum_repo_url_prod = f"https://yum.securedrop.org/workstation/dom0/{FEDORA_VERSION}"
+    yum_repo_url_test = f"https://yum-test.securedrop.org/workstation/dom0/{FEDORA_VERSION}"
 
     def setUp(self):
         # Enable full diff output in test report, to aid in debugging
diff --git a/tests/test_vms_exist.py b/tests/test_vms_exist.py
index 90cc499..0d7295c 100644
--- a/tests/test_vms_exist.py
+++ b/tests/test_vms_exist.py
@@ -5,7 +5,7 @@ from qubesadmin import Qubes
 from base import WANTED_VMS
 
 
-EXPECTED_KERNEL_VERSION = "4.14.241-grsec-workstation"
+EXPECTED_KERNEL_VERSION = "5.15.41-grsec-workstation"  # whoops; make conditional!
 
 DEBIAN_VERSION = "bullseye"
 with open("/etc/qubes-release") as qubes_release:
diff --git a/tests/test_vms_platform.py b/tests/test_vms_platform.py
index 31d7e2a..25917c5 100644
--- a/tests/test_vms_platform.py
+++ b/tests/test_vms_platform.py
@@ -6,13 +6,15 @@ from qubesadmin import Qubes
 from base import WANTED_VMS, CURRENT_FEDORA_TEMPLATE
 
 
+BULLSEYE_STRING = "Debian GNU/Linux 11 (bullseye)"
+BUSTER_STRING = "Debian GNU/Linux 10 (buster)"
 with open("/etc/qubes-release") as qubes_release:
     if "R4.1" in qubes_release.read():
-        SUPPORTED_SD_PLATFORMS = ["Debian GNU/Linux 11 (bullseye)"]
+        SUPPORTED_SD_PLATFORMS = [BULLSEYE_STRING]
     else:
-        SUPPORTED_SD_PLATFORMS = ["Debian GNU/Linux 10 (buster)"]
+        SUPPORTED_SD_PLATFORMS = [BUSTER_STRING]
 
-SUPPORTED_WHONIX_PLATFORMS = ["Debian GNU/Linux 11 (bullseye)"]
+SUPPORTED_WHONIX_PLATFORMS = [BULLSEYE_STRING]
 
 
 apt_url = ""
@@ -29,10 +31,9 @@ class SD_VM_Platform_Tests(unittest.TestCase):
             if "environment" not in config:
                 config["environment"] = "dev"
 
-            if "buster" in SUPPORTED_SD_PLATFORMS:
+            dist = "bullseye"
+            if BUSTER_STRING in SUPPORTED_SD_PLATFORMS:
                 dist = "buster"
-            elif "bullseye" in SUPPORTED_SD_PLATFORMS:
-                dist = "bullseye"
 
             if config["environment"] == "prod":
                 self.apt_url = FPF_APT_SOURCES.format(dist=dist, component="main")

[eloquence]

Trying to make dev from this branch, I get many Jinja syntax errors of the form:

  [CRITICAL] Rendering SLS 'base:sd-logging-setup' failed: Jinja syntax error: expected token ',', got 'for'; line 4
  
  ---
  # -*- coding: utf-8 -*-
  # vim: set syntax=yaml ts=2 sw=2 sts=2 et :
  
  {% if grains['id'] in [name.format(grains['oscodename']) for name in ["securedrop-workstation-{}", "sd-small-{}-template", "sd-large-{}-template"]] %}    <======================
  include:
    - fpf-apt-repo

It looks like the for syntax used here is erroring out at least on my system, if that's working for you/others, let's do a bit of a version comparison.

[sssoleileraaa]

Running through the test plan in #786 (comment) for Qubes 4.1, and will update here in case there are any issues.

[eloquence]

I took the liberty to remove a tiny print statement (8060842), for testing purposes it should not matter whether you're including that commit or not.

[eloquence]

Should this be removed, or updated to remove the Bullseye template?

[eaon]

So what's happening here is that while the qubes-template-securedrop-workstation-bullseye rpm exists, it's not installed via DNF and doesn't show up when you list installed packages. As far as I can tell, the qvm-template tool installs the relevant template files but then discards the downloaded package, which saves space when compared to 4.0. So the keep template functionality would have to be implemented in a different manner to avoid repeated downloads. I'll file a new ticket for this, but I don't think it's a problem for the first 4.1 release as it's a developer convenience feature

[eloquence]

• make dev installs without errors, with bullseye based templates which in turn use bullseye+nightlies repositories

• ❌ make test in dom0 passes except for 1 failure about a securedrop-workstation-grsec that is not up to date (seems to be a nightly package/apt problem, the package hasn't been changed, there's no negative effects to this failure)

• I can reproduce the securedrop-workstation-grsec issue. I've opened [4.1] securedrop-workstation-grsec package not upgradeable #793 for that, and added a hypothesis about what might be causing it.

• I get one additional test failure related to the MIME tests (test_mime_types). Our test fixture hasn't been updated yet to account for the eog.desktop -> org.gnome.eog.desktop changes in Update securedrop-workstation-config for bullseye, refrain from building buster nightlies securedrop-builder#349 and checks for eog.desktop explicitly:

securedrop-workstation/tests/vars/sd-viewer.mimeapps

Lines 24 to 28 in f1f3a88

	image/jpeg=eog.desktop
	image/gif=eog.desktop
	image/tiff=org.gnome.Evince.desktop
	image/png=eog.desktop
	image/svg+xml=eog.desktop

[eloquence]

Beyond that, things are looking great so far.

• Updater is happily updating Bullseye templates
• Auto-attach for USB device is working
• securedrop-client is starting without AppArmor denials

(I've not tested client functionality yet as this workstation is provisioned with a dummy configuration for now.)

[cfm]

Beyond @eloquence's findings, I'm able to sign off on the test plan as below! I'm approving but leaving for @eaon or @creviera to do the merging honors tomorrow. :-)

• make dev installs without errors, with bullseye based templates which in turn use bullseye+nightlies repositories

After make clean, make dev passed after several runs encountered transient failures:

• in DNS resolution (probably due to something weird with my sys-net); and

• from qvm-sync-appmenus in sd-devices-template-sync-appmenus.

• make test in dom0 passes except for 1 failure about a securedrop-workstation-grsec that is not up to date (seems to be a nightly package/apt problem, the package hasn't been changed, there's no negative effects to this failure)

With d2a2a52. Following 3181092, this entails that SD_Viewer_Tests.test_mime_types() will begin to fail on buster. If this is not acceptable, this commit should be refactored to parameterize the test.

[sssoleileraaa]

Thanks everyone for all the hard work 🚀