-
Notifications
You must be signed in to change notification settings - Fork 687
2.1.0 Test Plan
For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report.
If you have submitted a QA report already for a 2.1.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the release-specific changes as well as changes since the previous release candidate.
If you are testing the upgrade scenario, you should create source and journalist accounts before performing the upgrade - some test cases require existing accounts.
In order to test #5988 you will need to enable HTTPS on the source interface. To do so using self-signed certs:
- Use the command
cd ~/Persistent/securedrop && make self-signed-https-certs
on the Admin Workstation to generate the necessary files - Choose the appropriate options while running
./securedrop-admin sdconfig
- Accept the self-signed cert when prompted with the HTTPS warning in Tor Browser.
- Install target:
- Tails version:
- Test Scenario:
- SSH over Tor:
- Release candidate:
- General notes:
- After installing the testinfra dependencies, all tests in
./securedrop-admin verify
are passing:- Install dependencies on Admin Workstation with
cd ~/Persistent/securedrop && ./securedrop-admin setup -t
- Run tests with
./securedrop-admin verify
(this will take a while) - Remove test dependencies:
rm -rf admin/.venv3/ && ./securedrop-admin setup
- Install dependencies on Admin Workstation with
- QA Matrix checks pass
- Can successfully add admin user and login
- I have backed up and successfully restored the app server following the backup documentation
- If doing upgrade testing, make a backup on 2.0.2 and restore this backup on this release candidate
- "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
- Can successfully add journalist account with HOTP authentication
- JS warning bar does not appear when using Security Slider high
- JS warning bar does appear when using Security Slider Low
- On generate page, refreshing codename produces a new 7-word codename
- On submit page, empty submissions produce flashed message
- On submit page, short message submitted successfully
- On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
- On submit page, file less than 500 MB submitted successfully
- Nonexistent codename cannot log in
- Empty codename cannot log in
- Legitimate codename can log in
- Returning user can view journalist replies - need to log into journalist interface to test
- Can log in with 2FA tokens
- incorrect password cannot log in
- invalid 2fa token cannot log in
- 2fa immediate reuse cannot log in
- Journalist account with HOTP can log in
- Filter by codename works
- Starring and unstarring works
- Click select all selects all submissions
- Selecting all and clicking "Download" works
- You can submit a reply and a flashed message and new row appears
- You cannot submit an empty reply
- Clicking "Delete Source Account" and the source and docs are deleted
- You can click on a document and successfully decrypt using application private key
After updating to this release candidate and running securedrop-admin tailsconfig
- The Updater GUI appears on boot
- Updating to 2.0.2 is successful
-
- Verify that the SI lookup page HTML validates correctly (e.g. by using a Firefox addon)
-
#5695 - scrypt and sessions refactor
- Verify that no codename- or session-related errors were observed during application acceptance testing
- On the Source Interface, create a new user, log out, return to the home page without clearing the Tor Browser identity, and repeat this process at least 5 times. Verify that you do not observe an error wwith the text "you have been logged out due to inactivity" while repeating these steps.
- If possible, log in to the Source Interface simultaneously as separate users from multiple instances of the Tor Browser, submit messages from each user, and reply to the users from the Journalist Interface.
- Verify that each user has a distinct codename in the SI and journalist designation in the JI
- Verify that no errors were observed on the Source Interface in any user session
- Verify that submitted messages appear on the correct source page on the JI
- Verify that sources receive the correct reply in the SI and that the reply can be read
- On the application server, verify that each source has a distinct GPG key fingerprint, e.g. with the command
sudo -u www-data gpg --homedir /var/lib/securedrop/keys -k
- (Upgrade-only) Verify that a source user created before the upgrade can still log in using their existing codename, that their journalist designation is unchanged, and that they can still read existing and new replies
-
- Option 1, low effort: review automated accessibility checks
- Verify that Source Interface functionality and appearance is unchanged
- Using the Web Developer tools' Accessibility Inspector and check for issues: All, verify that accessibility issues have been significantly reduced on the SI
/lookup
page vs the corresponding page onhttps://demo-source.securedrop.org
- Compare other pages on the SI in the same way - verify that issues have been removed and no new issues introduced.
- Option 2, medium effort: step through submission flow in screen reader
- On Tails, activate the screen reader from the accessibility menu
- Step through the submission flow at
https://demo-source.securedrop.org
and pay attention to how the experience feels - Step through the submission flow in your QA instance and compare how the experience feels
- Option 1, low effort: review automated accessibility checks
-
- verify that the TOTP secret and corresponding QRcode displayed via
manage.py add-admin
on the application server are 32 chars long and can be used to generate valid TOTP codes via the Google Authenticator mobile app - verify that the TOTP secret and QRCode generated via the JI during user creation are also 32 chars long and valid
- (upgrade-only) verify that an existing 16-char shared secret can still be used to generate valid TOTP codes and log in
- (upgrade-only) verify that when a user account with an existing 16-char shared secret has its OTP secret regenerated, the new OTP shared secret is 32 chars long and valid.
- (HOTP-only) verify that an account using HOTP can still be created and used to log in.
- verify that the TOTP secret and corresponding QRcode displayed via
-
#5696 - no JS in user delete modal
- Set the Tor Browser security level to Safest
- In the Journalist Interface, verify that a confirmation modal dialog is displayed when a user is selected for deletion
- Choose Cancel and verify that the user was not deleted
- Select the user for deletion again and choose Delete. Verify that the user was deleted.
-
- (HTTPS-only) On the Source Interface index page, verify via the Tor Browser Security Tab that the connection is encrypted not only as an Onion Service but also via TLS 1.3
- (HTTPS-only) On the application server, inspect
/etc/apache2/sites-enabled/source.conf
and verify that it contains the lineSSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.2
- Install testssl.sh
sudo apt install -y testssl.sh -t buster-backports
in Tails, then runtestssl <onion url>
. Confirm that only TLSv1.3 is provided; all earlier protos are not.
-
- verify that the old signing key with fingerprint
22245C81E3BAEB4138B36061310F561200F4AD77
is not present on the application or monitor servers, e.g. by running the commandfor s in app mon; do ssh $s sudo apt-key list; done
from the Admin workstation - Verify that the new signing key
2359 E653 8C06 13E6 5295 5E6C 188E DD3B 7B22 E6A3
is presenton the servers
- verify that the old signing key with fingerprint
-
- Verify that no OSSEC alerts related to fwupd were received
-
#5909 - Manually-transferred backup
- Back up the instance using
./securedrop-admin backup
- Copy the backup file to the application server using e.g.
scp sd-backup.tar.gz app:/tmp/sd-backup-transfer.tar.gz
- Verify that the command
./securedrop-admin restore --no-transfer sd-backup-transfer.tar.gz
completes successfully, and that the local backup file is not transferred to the server during the playbook run.
- Back up the instance using
-
#6110 - Repair Tails installer
- (Tails version 4.18 or lesser):
- Verify that
./securedrop-admin tailsconfig
completes successfully and the Tails OS Updater starts without displaying errors - Verify that Tails can be successfully updated to the latest version.
- Verify that
- (Tails latest version):
- Verify that the command
./securedrop-admin tailsconfig
completes successfully without triggering the Tails OS Updater. - on the admin workstation, edit
/etc/os-release
to change the TAILS_VERSION_ID to a pre-4.19 version - Run
touch /usr/local/etc/ssl/certs/tails.boum.org-CA.pem
- Run `./securedrop-admin tailsconfig and verify that it completes successfully
- Verify that the contents of this file were added to
/usr/local/etc/ssl/certs/tails.boum.org-CA.pem
- Verify that the command
- (Tails version 4.18 or lesser):
- Install or upgrade occurs without error
- Source interface is available and version string indicates it is 2.1.0
- A message can be successfully submitted
- The updater GUI appears on boot
- The update successfully occurs to 2.1.0
- After reboot, updater GUI no longer appears