Attempt to create Qubes-managed template VM with grsec kernel

[eloquence]

As part of the template hardening epic (#21), we should attempt the workaround described here:

QubesOS/qubes-issues#3178 (comment)

to create a Qubes-managed template VM with a grsec kernel. This will increase our understanding of the scope/complexity of additional hardening work.

[emkll]

A first pass at a working grsec config in a Qubes HVM template appears to be successful. WIP kernel config can be found in https://github.com/freedomofpress/ansible-role-grsecurity-build/compare/qubes-workstation
There are a significant number of options set y or m when compared with the existing 4.14 workstation config, so one of the next steps should be to investigate and remove some of these parameters.

How to install the kernel in a Qubes Template (tested in Qubes 4.0):

• Clone the Qubes Debian template
• Build the kernel (or download the debs), copy linux-image and linux-headers to the newly created template vm
• Open a terminal in this template VM:

sudo apt install libelf-dev
sudo dpkg -i <linux-image>.deb <linux-headers>.deb
sudo dkms autoinstall -k <kernel name, e.g. 4.14.53-grsec>
# the previous command should exit without returning anything
update-grub2

shutdown the template. Now the template should now be ready for use.

Run the following commands in dom0 for the template and for all machines based on the template (when creating new vms):

# change from PVH to HVM
qvm-prefs <vm-name> virt_mode hvm
# Remove the default qubes kernel, it will use the VM's kernel instead
qvm-prefs <vm-name> kernel ''

After some non-exhaustive testing, qvm-copy-to-vmdoes not work due to /proc restrictions put in place by grsecurity.

The next step will be to ensure all Qubes integration/functionality works with these kernels and investigate the provisioning/distribution aspect forms

• installing paxctld to set flags on binaries
• automate the qvm-prefs flow described above
• ensure all required Qubes integrations are allowed

[conorsch]

Nits on testing instructions

Requesting clarification in the testing instructions. Instead of this:

Clone the Qubes Debian template

We should instruct testers to run qvm-clone debian-9 test-grsec-kernels, and trust Qubes to set the class appropriately. At first I tried qvm-create test-grsec-kernels --template debian-9 --label red --class TemplateVM, which failed with a rather cryptic error message.

It's also worth noting that the dmks autoinstall -k command will respond to tab completion. 😇

Finally, update-grub2 is merely a symlink to update-grub, so I don't see the value in using the suffix. If there's a forward-compatible motivation at play here, please let me know.

Results

Working as described! 🎉 After halting-and-starting the new TemplateVM, I confirmed it was running 4.14.52-grsec by running uname -r. Next, I created an AppVM based on the template:

qvm-create --template test-grsec-kernels test-grsec-kernels-child --class AppVM --property virt_mode=hvm --property kernel='' --label green

and confirmed that machine was running 4.14.52-grsec on boot, as well. So far, so good! Have not yet explored setting PaX flags.

[emkll]

Successully went through the entire decryption flow using grsec template for both sd-svs and sd-svs-disp VMs with this branch.

Thesd-workstation-grsec template was configured as above, with the following incremental changes:

• apt install nautilus to get the Files shortcut to get parity with the Fedora template (The debian-9 template did not have Files)
• apt install paxctld
• Add the following line in /etc/paxctld.conf, as the qubes RPC mechanism uses gnome-terminal-server on the VM to run commands:

/usr/lib/gnome-terminal/gnome-terminal-server   m

More obscure formats will require additional PaX flags.

[emkll]

On topic of memory usage/overhead of using HVM instead of PVH, I used xentop to monitor the MAXMEM usage of VMs. Opened several submissions within grsec-based disposable vm, and MAXMEM would hover around 600MB per disposable VM, and 130MB for the associated disp-vm. Non-grsec (PVH-based) VMs (e.g. sd-decrypt, sd-gpg) hover at around 500MB per VM.

[eloquence]

Since this was a timeboxed research task, closing. We can now decide what the next step for integrating with the current workstation architecture should be.