Disable v2 onion addresses on restore on Focal

[kushaldas]

Status

Ready for review

Description of Changes

Fixes #5676 , towards #5731

We filter out any v2 onion address related line from /etc/tor/torrc
and also the directories from /var/lib/tor/services. This will
happen only on Focal. On Xenial, everything stays the same.

Testing

• Checkout this branch in Tails
• Restore a v2 + v3 backup into a Xenial prod, nothing should change
• Create a Focal prod vm following steps at Prod vms on focal #5669 with both v2+v3
• Restore the same backup from tails to the Focal VM.

Deployment

Any special considerations for deployment? Consider both:

1. Upgrading existing production instances.
2. New installs.

Checklist

If you made changes to the server application code:

• Linting (make lint) and tests (make test) pass in the development container

If you made changes to securedrop-admin:

• Linting and tests (make -C admin test) pass in the admin development container

If you made changes to the system configuration:

• Configuration tests pass

If you made non-trivial code changes:

• I have written a test plan and validated it for this PR

Choose one of the following:

• I have opened a PR in the docs repo for these changes, or will do so later
• I would appreciate help with the documentation
• These changes do not require documentation

If you added or updated a code dependency:

Choose one of the following:

• I have performed a diff review and pasted the contents to the packaging wiki
• I would like someone else to do the diff review

[lgtm-com]

This pull request introduces 3 alerts when merging c98ad06 into 2e513b1 - view on LGTM.com

new alerts:

• 3 for Variable defined multiple times

[lgtm-com]

This pull request introduces 3 alerts when merging fafb675 into 2e513b1 - view on LGTM.com

new alerts:

• 3 for Variable defined multiple times

[codecov-io]

Codecov Report

Merging #5677 (fafb675) into develop (2e513b1) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #5677   +/-   ##
========================================
  Coverage    85.38%   85.38%           
========================================
  Files           51       51           
  Lines         3709     3709           
  Branches       464      464           
========================================
  Hits          3167     3167           
  Misses         441      441           
  Partials       101      101           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e513b1...fafb675. Read the comment docs.

[eloquence]

(Marked as release blocker for 1.8.0.)

[kushaldas]

Rebased today.

[lgtm-com]

This pull request introduces 3 alerts when merging 31f6091 into 65e8d6b - view on LGTM.com

new alerts:

• 3 for Variable defined multiple times

[rmol]

This left Xenial services untouched. On Focal it removed all v2 services but ssh. Seems like that should go as well.

[kushaldas]

I started working on the updating of the PR, and now wondering about the scenario of v2 ssh. My update will assume that v3 ssh is already setup.

[kushaldas]

Rebased against develop.

[lgtm-com]

This pull request introduces 5 alerts when merging e5fa74b into f3e51b4 - view on LGTM.com

new alerts:

• 5 for Variable defined multiple times

[lgtm-com]

This pull request introduces 5 alerts when merging 1066cd0 into 88ac049 - view on LGTM.com

new alerts:

• 5 for Variable defined multiple times

[zenmonkeykstop]

Tried using this to restore a v2+v3 Xenial backup onto a v3-only Focal system. It fails at the compare_torrc.py step, as the configurations are obviously different, but this scenario should probably be supported.

My expectation would be that the migration states should go as follows:

backup \ target	v2 Xenial	v2+v3 Xenial	v3 Xenial or Focal
v2	Yes	No (restore then migrate)	yes with --skip-tor
v2+v3	No	Yes	yes but v3-only
v3	No	Yes but v3-only	Yes

(implicit above is that there are no valid Focal targets with v2)

• The v2 backup row is covered with current code.
• The v2+v3 backup row is not covered, needs changes to compare_torrc.py logic and logic to apply only the v3 config.
• the v3 backup row is not covered, needs same changes as v2+v3

In all cases the docs need refreshing (see freedomofpress/securedrop-docs#133 ).

Implementation notes

• We can probably get away without the script to clean up the servers' torrc files, as the install playbook is run as part of migrations and it should ensure that torrc is correct (but does it? goood question).
• There is existing logic to skip Tor configs altogether using the exclude field:

  securedrop/install_files/ansible-base/roles/restore/tasks/main.yml

  Line 62 in 1066cd0

  exclude: "var/lib/tor,etc/tor/torrc"

  - a similar approach could be used to selectively exclude v2 configurations This could be cleaner than removing v2 files after copying them to the server.
• Changes to compare_torrc.py (or a different approach) would be required for the case where v2+v3 backups are applied to a v3 target - instead of erroring out it could indicate that v3 was available on the server, and allow the restore to proceed with the v3 info from the backup.

[lgtm-com]

This pull request introduces 5 alerts when merging 05d83c7 into da9dbf2 - view on LGTM.com

new alerts:

• 5 for Variable defined multiple times

[lgtm-com]

This pull request introduces 5 alerts when merging 8835810 into 23bf5f8 - view on LGTM.com

new alerts:

• 5 for Variable defined multiple times

[kushaldas]

I tested with v2/v3 and only v3 restore. Also fixed the merge conflict in CI. Now, need at least more person to make sure that I did not loose anything after merge conflict. Maybe @zenmonkeykstop

[zenmonkeykstop]

Conflict resolution looks good based on visual review. Approving.

ssh has been addressed