1.8.0 Test Plan

1.8.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report.

If you have submitted a QA report already for a 1.8.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the 1.8.0-specific changes as well as changes since the previous release candidate.

There are OS-specific sections in the test plan - make sure you complete the appropriate section based on the server OS in your chosen test scenario.

Environment

• Install target:
• Server OS:
• Tails version:
• Test Scenario:
• SSH over Tor:
• Onion service version:
• Release candidate:
• General notes:

Basic Server Testing

• I can access both the source and journalist interfaces
• I can SSH into both machines over Tor
• AppArmor is loaded on app
  • 0 processes are running unconfined
• AppArmor is loaded on mon
  • 0 processes are running unconfined
• Both servers are running grsec kernels
• iptables rules loaded
• OSSEC emails begin to flow after install
• OSSEC emails are decrypted to correct key and I am able to decrypt them
• After installing the testinfra dependencies, all tests in ./securedrop-admin verify are passing:
  • Install dependencies on Admin Workstation with cd ~/Persistent/securedrop && ./securedrop-admin setup -t
  • Run tests with ./securedrop-admin verify (this will take a while)
  • Remove test dependencies: rm -rf admin/.venv3/ && ./securedrop-admin setup
• QA Matrix checks pass

Command Line User Generation

• Can successfully add admin user and login

Administration

• I have backed up and successfully restored the app server following the backup documentation
• If doing upgrade testing, make a backup on 1.7.1 and restore this backup on this release candidate
• "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
• Can successfully add journalist account with HOTP authentication

Application Acceptance Testing

Source Interface

Landing page base cases

• JS warning bar does not appear when using Security Slider high
• JS warning bar does appear when using Security Slider Low

First submission base cases

• On generate page, refreshing codename produces a new 7-word codename
• On submit page, empty submissions produce flashed message
• On submit page, short message submitted successfully
• On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
• On submit page, file less than 500 MB submitted successfully

Returning source base cases

• Nonexistent codename cannot log in
• Empty codename cannot log in
• Legitimate codename can log in
• Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

• Can log in with 2FA tokens
• incorrect password cannot log in
• invalid 2fa token cannot log in
• 2fa immediate reuse cannot log in
• Journalist account with HOTP can log in

Index base cases

• Filter by codename works
• Starring and unstarring works
• Click select all selects all submissions
• Selecting all and clicking "Download" works

Individual source page

• You can submit a reply and a flashed message and new row appears
• You cannot submit an empty reply
• Clicking "Delete Source Account" and the source and docs are deleted
• You can click on a document and successfully decrypt using application private key

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

• The Updater GUI appears on boot
• Updating occurs without issue

1.8.0 release-specific changes (common)

• V2 SSH only configured when v2 services are enabled #5718

  • If SSH-over-Tor was enabled and v2 onion services were not enabled during installation:
  • the v2 onion service configuration in /var/lib/tor/services/ssh was not created on either the Application or Monitor Server
  • the file /etc/tor/torrc does not contain HiddenServiceVersion 2 on either the Application or Monitor Server
  • (optional) OSSEC alerts related to v2 onion services are not triggered

• SSHd config updates #5666

  • After installation, ssh access to both servers works without issue in either SSH-over-Tor or SSH-over-LAN (depending on chosen config)
  • No OSSEC alerts are generated including the text Error: Unable to load host key: /etc/ssh/ssh_host_dsa_key (#5660)

• Safe deletion #5770

  • With Tor Browser's security setting at "standard", sources' files and messages can be deleted on the All Sources page :

    • log into the SI and submit multiple messages/files
    • log into the JI and click Delete on the All Sources page without selecting any sources' checkboxes
      • a server call is not made, and a modal is displayed under the Delete button asking the user to select one or more checkboxes.
    • select the checkbox for the source created above in the "All Sources" page and Click Delete..:
      • a modal is displayed under the delete button giving the option to delete files and messages, delete source accounts, or cancel - the number of sources selected is also displayed.
    • click Cancel
      • The source entry is present and its file/message counts are unchanged
    • ensure that the source is selected and click Delete.. again, then click Files and Messages
      • A success flash message is displayed
      • The source is still present and its file/message counts are both 0
    • in the SI, submit a message
      • The message is submitted successfully
      • in the JI, when the All Sources page is refreshed the message count is now 1.
      • clicking on the source codename opens the source page, the message is listed and can be downloaded.
      • on the source page, a reply can be successfully sent to the source
    • Return to the All Sources page, select the source, and choose Delete > Files and Messages
      • The source is present and counts are 0
      • clicking through to the source page works and no files/messages/replies are listed.
    • In the SI, submit some more messages/files, then log out, create a new source account, and submit more messages. Repeat to create a total of 3 sources with submissions.
    • In the JI, return to the All Sources page.
    • select two sources, choose Delete > Files and Messages
      • both sources are present with zeroed file/message counts
      • the third source is present and its counts are unchanged (and non-zero)

  • With TBB security set to "standard", source accounts can be deleted with a double confirm on the All Sources page:

    • log into the SI, recording the source codename, and submit multiple messages/files
    • log into the JI and select the checkbox for the source created above in the "All Sources" page
    • Click Delete..:
      • a modal is displayed under the delete button giving the option to delete files and messages, delete source accounts, or cancel, the count of selected sources is also displayed
    • Click Source Accounts
      • A second explanatory modal is displayed giving the option to cancel or delete source accounts
    • Click Yes, Delete Source Accounts
      • a success flash message is displayed and the source account is removed from the listing
      • the source's files are all queued for deletion on the server
      • the source's database entry is deleted
      • the sources' reply key is deleted.
    • return to the SI and attempt to log in as the source:
      • the source codename is not found.
      • In the SI, log in with a new account submit some more messages/files, then log out, create a new source account, and submit more messages. Repeat to create a total of 3 sources with submissions.
    • return to the JI and open the All Sources page
    • select two sources, choose Delete > Source Accounts > Yes, Delete Source Accounts
      • a success flash message is displayed
      • the two sources selected are deleted from the All sources page and the server (store/db/reply key)
      • the remaining source is unaffected.

  • With TBB security at "safest", the test cases above pass with the the following exceptions:

    • the selected source count is not displayed on the initial deletion modal when deleting files and messages or source accounts on the All Sources page
    • the modals are centered in the page, not displayed under the delete button on the All sources page
    • a flash error message is displayed instead of the error modal when the user clicks Delete on All Sources with nothing selected.

• Empty files are no longer created for disconnected database entries #5724

  • Log in to the Source Interface as a new source. Submit one message.
  • Connect to the Application Server over SSH, navigate to the source's directory under /var/lib/securedrop/store and delete the file of the message you just submitted.
  • Back in the Source Interface, submit another two messages, waiting a few seconds between them.
  • On the Application Server, verify that the source's directory only contains two files (2-... and 3-...) and that their timestamps are identical.

• Remove cloud-init package during installation #5771

  • When the command ssh app apt list --installed | grep cloud-init is run via an Admin Workstation terminal, it returns an empty string.
  • When command ssh mon apt list --installed | grep cloud-init is run via an Admin Workstation terminal, it returns an empty string.

• Install release-upgrader in prepare-servers role (#5792)

  • When the command ssh app apt list --installed | grep release-upgrader is run via an Admim Workstation terminal, it returns ubuntu-release-upgrader-core/{focal,xenial} (depending on the server OS)
  • When the command mon app apt list --installed | grep release-upgrader is run via an Admim Workstation terminal, it returns ubuntu-release-upgrader-core/{focal,xenial} (depending on the server OS)

• Update Tor to 0.4.5.6 #5803

  • When the command ssh app tor --version is run via an Admim Workstation terminal, it returns "Tor Version 0.4.5.6."
  • When the command ssh mon tor --version is run via an Admim Workstation terminal, it returns "Tor Version 0.4.5.6."

• LTS upgrade prompt is disabled #5786

  • The command ssh app cat /etc/update-manager/release-upgrades | grep "Prompt=never" | wc -l outputs 1 when run from the Adminm Workstation terminal
  • The command ssh app cat /etc/update-manager/release-upgrades | grep "Prompt=lts" | wc -l outputs 0 when run from the Adminm Workstation terminal
  • The command ssh mon cat /etc/update-manager/release-upgrades | grep "Prompt=never" | wc -l outputs 1 when run from the Adminm Workstation terminal
  • The command ssh mon cat /etc/update-manager/release-upgrades | grep "Prompt=lts" | wc -l outputs 0 when run from the Adminm Workstation terminal

• Update and annotate Apache configuration #5797

  • Check the Source Interface headers from an Admin Workstation terminal using the command curl -I http://<onion>, where <onion> is the SI onion address. The response should include the following:
    • X-Frame-Options: DENY
    • Referrer-Policy: same-origin
    • X-XSS-Protection: 1; mode=block
    • Content-Security-Policy: default-src 'none'; script-src 'self'; style-src 'self'; img-src 'self'; font-src 'self';
    • X-Download-Options: noopen
    • Cache-Control: no-store
  • Repeat the command for the Journalist Interface onion address:
    • The header values are the ame as for the SI with the exception of Referrer-policy, whose value should be no-referrer

• Check for updates before most securedrop-admin commands #5788

  • On an Admin Workstation with persistence unlocked and an admin password set:
    • Open a terminal and change directory to ~/Persistent/securedrop
    • Ensure the code is on the most recent 1.8.0 RC tag with git status, switching if necessary with , e.g., git checkout 1.8.0-rc1.
    • Run the command ./securedrop-admin logs. Verify that it does an update check, does not run the subcommand, prints an error, and exits with exit code 1 ( check with echo $?)
      • Verify that the error message above correctly reflects the state of the repository (latest version is 1.7.1) and your checkout (HEAD detached at 1.8.0-rc1).
    • Delete the most recent tags locally (git tag -d 1.8.0-rc1 && git tag -d 1.7.1). Retag your current HEAD as 1.7.1 with an annotated tag (git tag -a 1.7.1 -m 'TEST TAG ONLY'). This tells the updater that you are using the expected tag even though you are on 1.8.0-rc1.
    • Run ./securedrop-admin logs. Confirm that the command prints "All updates applied" and proceeds to fetch logs.
    • Delete your test tag with git tag -d 1.7.1, restore the tags from the server with git fetch --tags --all, and check out the latest RC again with, e.g., git checkout 1.8.0-rc1
    • Run ./securedrop-admin logs again, confirming that the error is displayed and the subcommand not run
    • Run ./securedrop-admin --force logs. Confirm that the version check is skipped and logs are fetched.
    • (Optional) Repeat the check for other ./securedrop-admin subcommands and verify that version checks are performed.

1.8.0 release-specific changes (Xenial only)

• End-of-life messaging#5789

  • When logged into the Journalist Interface, a banner is displayed with information on the April 30 date and a link to the blog advisory.
  • When visiting the Source Interface, the interface is enabled
  • If v2 is enabled, neither the Source Interface nor the Journalist Interface display a v2-related warning banner.
  • on the Application Server, edit the file /var/www/securedrop/server_os.py, changing XENIAL_EOL_DATE value to date(2021,2,23), and restart Apache with the command sudo systemctl restart apache2
    • When logged into the Journalist Interface, a banner is displayed informing you that the Source Interface is disabled and linking to the blog advisory.
    • When visiting the Source Interface, a message is displayed saying that it is disabled, and you cannot log in or create a new source account.

• IPv6 disabled in init in Focal only#5810

  • In an SSH session on the Application Server via ssh app, the commands below have the following output:
    • sudo ip -4 addr: you should see two addresses, one for localhost and one for the ethernet device.
    • sudo ip -6 addr: you may see address information, for localhost, ethernet, or both.
    • sudo cat /proc/cmdline: you should NOT see "ipv6.disable=1" in the output.
    • sudo ip6tables -S: a brief list of "DROP" policies. Each line you see should have "DROP", but no lines should have "ALLOW".

• Repeating the process above on the Monitor Server, you should see the same results.

1.8.0 release-specific changes (Focal only)

• Focal support added #4728

  • A fresh install using Focal as the base OS completed successfully
  • If a migration from an existing backup was performed as part of testing:
    • The data restoration was completed successfully, including data, submissions, and JI accounts
    • If the backup file included v2 onion service configurations, they were not carried over to the Focal install. #5677

• Update Kernel to 5.4.97 for Focal #5785

  • When the command ssh app uname -r is run via the Admin Workstation terminal, it outputs 5.4.97-grsec-securedrop
  • When the command ssh mon uname -r is run via the Admin Workstation terminal, it outputs 5.4.97-grsec-securedrop

• End-of-life messaging#5789

  • When logged into the Journalist Interface the EOL banner is not displayed.
  • When visiting the Source Interface, the interface is enabled
  • Neither the Source Interface nor the Journalist Interface display a v2-related warning banner.
  • on the Application Server, edit the file /var/www/securedrop/server_os.py, changing XENIAL_EOL_DATE value to date(2021,2,23), and restart Apache with the command sudo systemctl restart apache2
    • When logged into the Journalist Interface the EOL banner is not displayed.
    • When visiting the Source Interface, the interface is enabled

• resolvconf is not present on focal #5809

  • When the command ssh app apt list --installed | grep resolvconf is run via an Admin Workstation terminal, it returns an empty string.
  • When the command ssh mon apt list --installed | grep resolvconf is run via an Admin Workstation terminal, it returns an empty string.
  • When the command ssh app dig freedom.press is run via an Admin Workstation terminal:
    • it should succeed.
    • The SERVER line at the bottom should contain the IP address of the DNS server configured via ./securedrop-admin sdconfig (e.g. 8.8.8.8)

• Remove aptitude and disable install-recommends #5793

  • When the command ssh app apt list --installed | grep aptitude is run via an Admin Workstation terminal, it should return an empty string
  • When the command ssh mon apt list --installed | grep aptitude is run via an Admin Workstation terminal, it should return an empty string
  • When the command ssh app sudo apt install vlc is run via an Admin Workstation terminal:
    • It should complete successfully
    • The subsequent command ssh app apt list --installed | grep vlc-l10n should return an empty string

• IPv6 disabled in init in Focal only #5810

• In an SSH session on the Application Server via ssh app, the commands below have the following output:

  • sudo ip -4 addr: you should see two addresses, one for localhost and one for the ethernet device.
  • sudo ip -6 addr: you should see no output.
  • sudo cat /proc/cmdline: you should see "ipv6.disable=1" in the output.
  • sudo ip6tables -S: you should see an error about functionality not being supported.

• Repeating the process above on the Monitor Server, you should see the same results.

• replace ntp with systemd-timesyncd #5806https://github.com/freedomofpress/securedrop/issue/5806)

  • Confirm that ntp and ntpdate are not installed on the Application Server with the Admin Workstation command ssh app apt list --installed | egrep (ntp|ntpdate) - it should not return any package listings
  • Confirm that ntp and ntpdate are not installed on the Monitor Server with the Admin Workstation command ssh mon apt list --installed | egrep (ntp|ntpdate) - it should not return any package listings
  • confirm that time has been synchronized to NTP servers on both machines:
    • ssh app timedatectl show and ssh mon timedatectl show should both contain NTPSynchronized=yes
    • ssh app timedatectl show-timesync and ssh mon timedatectl show should both contain ServerName=ntp.ubuntu.com, with an NTPMessage indicating that the server has been reached

• Use paxctld, not paxctl on Focal #5808

  • When the command ssh app apt list --installed | grep paxctl/focal is run via an Admin Workstation terminal, it should return an empty string
  • When the command ssh mon apt list --installed | grep paxctl/focal is run via an Admin Workstation terminal, it should return an empty string
  • When the command ssh app apt list --installed | grep paxctld/focal is run via an Admin Workstation terminal, it should return a listing for paxctld
  • When the command ssh mon apt list --installed | grep paxctld/focal is run via an Admin Workstation terminal, it should return a listing for paxctld
  • When the command ssh app systemctl status paxctld is run, its output should indicate that paxctld is active.

• Replace cron-apt with unattended-upgrades #5684 RC2 or later only

  • the Admin Workstation command ssh app unattended-upgrades --dry-run works without returning errors
  • the Admin Workstation command ssh app unattended-upgrades -d works without returning errors, and the Application Server log at /var/logs/unattended-upgrades.log contains no errors
  • If a later RC version was available overnight, it has been applied automatically
  • The system was rebooted automatically at or close to the time specified via `./securedrop-admin sdconfig

• v2 services cannot be installed on Focal #5819

  • run ./securedrop-admin sdconfig, choosing to enable v2 onion services but leaving all other settings unchanged.
    • When ./securedrop-admin install is run, it errors out immediately after the prepare-servers role with an message including Please run sdconfig again, disabling v2 services.

RC3/RC3 specific testing

• If Focal, the unattended-upgrades related timers work, see https://github.com/freedomofpress/securedrop/pull/5855 for test plan
• If Xenial, ensure timers do not exist in xenial, and cron-apt configuration still works

Preflight testing

Basic testing

• Install or upgrade occurs without error
• Source interface is available and version string indicates it is 1.8.0
• A message can be successfully submitted

Tails

• The updater GUI appears on boot
• The update successfully occurs to 1.8.0
• After reboot, updater GUI no longer appears