Ensure TemplateVMs are shut down prior to rebooting

[emkll]

Status

Ready for Review

Description of Changes

Maybe fixes #498.

Should they call sd-log during shutdown sequence or otherwise, they may interfere with the reboot order and introduce failures. ALso move sd-app to beginning of shutdown sequence.

Since sd-log receives qubes-rpc calls from pretty much every AppVM and TemplateVM we provision. If that's the case, changed we've introduced in #487 could explain why we have been seeing this error more often, but also why it has been inconsistent across our installs: we do not control the state (powered on/off) of TemplateVMs. Should a template send logs to sd-log and triggering an out-of-order reboot. It would make sense to ensure any VM that logs to be shut down prior to sd-log, and start sd-log before all other VMs.

Testing

[eloquence]

Run 1:

• No templates running
• No USBs attached
• Updates not forced, but some fedora-30 updates due from yesterday
• No errors

[eloquence]

Run 2:

• sd-app-buster-template running
• zlib downgraded to force fedora-30 update
• No USBs attached
• No errors

[eloquence]

Run 3:

• sd-app-buster-template running
• zlib downgraded to force fedora-30 update
• No USBs attached
• No errors

[eloquence]

Run 4:

• sd-app-buster-template running
• zlib downgraded to force fedora-30 update
• No USBs attached
• No errors

One more.

[eloquence]

Run 5.

• sd-app-buster-template running
• zlib downgraded to force fedora-30 update
• No USBs attached
• "Security updates cannot be applied at this time" but no libxenlight error

(Unrelated to this PR. UPDATES_FAILED=3 reported for sd-whonix so I'm guessing Tor flakiness. Point in favor of moving away from updates over Tor soon, IMO, if necessary by eliminating Whonix dependency - post-initial consolidation, of course.)

All runs were done using the version of Updater.py in this branch, by manually replacing the copies in /opt and /srv in dom0.

[eloquence]

I don't see any regressions and the reasoning for the changes makes sense to me, so 👍 from me to promoting from draft status for final review.

[conorsch]

@emkll Nudge on ready-for-review status here. Ideally we'd get in #523 first, but that's not ready-for-review, and this one (arguably 😃) is.

[conorsch]

Haven't performed functional review of these changes myself, but given the copious testing by @eloquence & @rocodes (see #498 (comment)), quite comfortable merging based on visual review.

The test updates in #523 will need to be rebased post-merge, I'll get on that shortly.