Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finalize document export approach #84

Closed
joshuathayer opened this issue Apr 9, 2018 · 21 comments
Closed

Finalize document export approach #84

joshuathayer opened this issue Apr 9, 2018 · 21 comments

Comments

@joshuathayer
Copy link
Contributor

After review / redaction, how does a document make it on to a USB drive?

  • from SVS vm?
  • from (a new) export vm?
  • from dispvm

Consider that we can permanently attach a USB device to a non-disposable VM, so UX might be best choosing the export or SVS options. In that case, how does the user easily copy redacted/reviewed submissions to the appropriate VM?

@harlo
Copy link

harlo commented May 23, 2018

yes, you can script attaching a USB device to a disposable VM; i wouldn't know off the top of my head what's the safest and most-efficient way to do it yet, though.

once you get that sorted, it's pretty simple to leverage basic luks-encrypted utilities via nautilus or veracrypt (if you include it in the disposable vm template) to push media into "cold storage"

@eloquence
Copy link
Member

There is a first proposal for an architecture in the current README in this repository (in short: copy files to sd-export from sd-svs, attach USB device there and copy to it).

Is this the architecture we want to use? There are two major problems the final architecture should mitigate against:

  • user errors resulting in failed exports. The attach/detach process can be error-prone, even with use of qubes-usb-proxy. To the extent that devices can be permanently attached to a trusted VM, that would significantly simplify things.

  • decrypted files being forgotten about in QubesIncoming within a permanent export VM. This could be mitigated against by automatic removal on boot, for example, or by use of disposable VMs for the export.

Let's explore/test some additional options. If we do use disposable VMs, we'll have to think more about batch exports to avoid user frustration.

@eloquence eloquence added this to the 0.1.0beta milestone Nov 28, 2018
@eloquence eloquence changed the title decide on document export approach Finalize document export approach Nov 28, 2018
@emkll
Copy link
Contributor

emkll commented Nov 30, 2018

DisposableVMs seem perfectly suited for this task, and I think automatically attaching it would make sense (and especially would guard against a user attaching the usb device to the wrong vm, like sd-svs). The challenge would be having some sort of device whitelist to know which type of device or device id to attach (qvm-usb doesn't expose the type of the device, only description).

The most straightforward implementation would likely have sd-svs act as (or delegate to) an admin-vm that would handle the creation of a DispVM, svs sends files to said DispVM, and admin vm attaches drive to dispVM, formats/encrypts the drive, copies the files and unmounts the drive.

Redaction/Sanitization:

In the current model, if they are copied from sd-svs, they are likely not redacted/sanitized, so there may need to be an export flow from the sd-svs-disp that edits/redacts and sends those to sd-export

Encryption

Whatever is running in sd-export will need to handle encrypting the data, and encrypting the device is preferable to encrypting the single files. Veracrypt (as stated above) does have a CLI interface, and would be the only multi-platform device encryption utility. We would likely need to host a .deb in that case.

@ninavizz
Copy link
Member

ninavizz commented Dec 2, 2018

As the users experience the export flow, I'd like for the whole thing to feel natural enough for our parents to walk-up and feel completely comfortable with it.

I appreciate that security and Qubes mandate extra precautions; I do. I also know that EVERYONE wants to see a primo UX result from all our hard work.

In service to that shared goal, I submit the above nudge on technical dependencies that may impose extra steps in the export process' UX. Just a nudge. I know you guys have a lot of work to do. :)

Our users are unlikely to be major consumers of FOSS products, or maker/DIY-minded (as most American FOSS consumers tend to be)... so that extra push from us in service to their comforts, will go a long way wrt building the confidence to dive deeper into technical unknowns—which, bigger-picture, is the broader goal in shaping safer behaviors in folks.

@airelemental
Copy link

airelemental commented Dec 18, 2018

Another option may be the new persistent-name DispVMs introduced in Qubes 4.0.

Like classic DispVM, they are stateless and are based on a DVM template. Unlike classic DispVM, they have a persistent name, e.g. disp-sds-export, and metadata. Once created, you can open its qubes settings from CLI (qubes-vm-settings disp-sds-export) and permanently assign pci devices (such as usb controller).

marmarek describes the creation process here: https://groups.google.com/d/msg/qubes-users/LB9f9OSWCzM/Grsk5VDjCQAJ

@eloquence
Copy link
Member

For the 4/17-5/1 sprint, @eloquence will provide a more detailed summary of the currently UX-prototyped workflow and the underlying assumptions re: Qubes VMs; @emkll will then do an initial technical and security investigation, aiming for a first iteration implementation proposal we can consider for the n+1 or n+2 sprint.

@ninavizz
Copy link
Member

ninavizz commented May 1, 2019

^ @emkll requested the latest prototype(s) for Export, yesterday. The two options being considered are linkable w/ brief prototype-specific details, here: https://github.com/freedomofpress/securedrop-client/wiki/Styleguide%28ish%29#export-functionality

Yep, I updated the Briefcase flow to reflect the latest VisDe updates, too. Still holding-out hope that it might get in for Beta, but totally understood if not. :)

@eloquence
Copy link
Member

So far, we have mainly focused on the "briefcase" flow, where you essentially have a staging area for exports within the SecureDrop client, then spin up a disposable VM when you're ready to export.

In the UX meeting last week we discussed whether it might be possible to have a much more lightweight workflow for managing exports.

If it is feasible to initiate a disposable VM for each session (either client login session, or Qubes login session) which has the required USB device access, any files exported during the session could be exported to the same VM. That would eliminate the need for an export staging area in the client, significantly simplifying implementation. It would also improve performance given the reduced number of VM spin-up/tear-down cycles.

@emkll, @rmol, @redshiftzero I would suggest that we plan to research that specific question through some targeted prototyping in Qubes; does that make sense to y'all?

@emkll
Copy link
Contributor

emkll commented May 13, 2019

Following several discussions with @redshiftzero and @rmol , as well as all the excellent input provided above, I propose we proceed with the following, unless anyone has any objections:

sd-client in sd-svs

  • User clicks on export in the client
  • communicates to sd-export-disp via qrexec
  • generates or prompts for a symmetric "export" key
  • documents are encrypted (GPG would be simplest) as they are exported to the export VM
  • Script in sd-export-disp mounts drive, copies files, and unmounts drive

sd-export-template

  • For now a basic debian-9 or securedrop-workstation template should be sufficient
  • An sd-export template might help in the future, if we want to add onionshare, veracrypt or other export options such as printer drivers.

sd-export-disp (persistent named DispVM suggested by @airelemental above, and based on sd-export-template)

  • mounts the usb device must be attached first, see below
  • receives encrypted submission from dispvm
  • mounts the export drive
  • copies encrypted submission from dispvm
  • unmounts export drive
  • returns status to sd-client:
    • no export device found
    • success
    • fail

export-proposal

If it is feasible to initiate a disposable VM for each session [...]

I think a disposable VM for each session could work, if we take care to mount/unmount the drive each time we perform a write to the disk. We don't want to introduce further complexity around adding disk management state to the export VM, as the files aren't exported to the VM but rather written to an external device mounted in that VM. We must ensure that USB devices that are unplugged without being ejected and result in corrupted exported data. Note however that if we want to incorporate convert-to-trusted-pdf in this flow, this itself will introduce a dispvm cycle.

Before mounting the USB device, dom0 needs to attach it to a VM. So far, there seems to be three ways to achieve this:
1: qvm-pci method described above, which will expose the entire usb controller to the dispvm. This means that no other usb devices could be used by the workstation, unless going through the sd-export-disp, or unless toggled in dom0
2. instructing users to use the dom0 widget to mount the drive in sd-export-template
3. set udev rules (per-vendor?) to start the dispVM (if required) and qvm-usb attach to the dispvm.

Given the potential for user error, options 1 or 3 seem most prudent, and 2 and 3 provide the most flexibility.

@ninavizz
Copy link
Member

ninavizz commented May 13, 2019

@emkll It would be ideal if we could have one/singular sd-disp-export VM for all documents to be saved to, from the Client—for printing, redaction or safe-PDF tools use, onionshare, or saving to a USB drive. Am I reading your assessment correctly, that you'd rather have a different disp-VM for printing vs saving things to USBs and Onionshare? That seems like it'd be very confusing to users... tho admittedly, not as confusing as using Qubes' devices widget.

I've grown increasingly fond of the idea of having one single "Briefcase" disp-VM, that all files for use with peripheral devices or onionshare, get saved to. As you noted in your recommendation, each one would be ephemeral to each authenticated session—but a single destination disp-VM will spare users the cognitive/process burden of deciding how they wish to act upon files before exporting them... and may also spare users the need to multi-export to multiple VMs (printing and saving). If I understand your recommendation correctly. The device auto-attach is pretty important too, though. Is it a binary one-or-the-other to keep security high? Could a "safe" printer get designated for each workstation laptop—as that one peripheral is unlikely to ever change?

Yeah, journalists love to print—but we also don't know if that's because treeware is the only tool they currently have for scrubbing metadata, their only means to share files at the moment, or other motivations.

Looking forward to studying your recc more closely/carefully!

@emkll
Copy link
Contributor

emkll commented May 13, 2019

It would be ideal if we could have one/singular sd-disp-export VM for all documents to be saved to, from the Client—for printing, redaction or safe-PDF tools use, onionshare, or saving to a USB drive. Am I reading your assessment correctly, that you'd rather have a different disp-VM for printing vs saving things to USBs and Onionshare? That seems like it'd be very confusing to users... tho admittedly, not as confusing as using Qubes' devices widget.

The intent is for sd-export to be disposable (its state shouldn't persist across sessions/reboots), and used for both printing and exporting to a USB drive. I think it would be best to abstract the details away from the user, and not persist any data in the sd-export vm: it should be stateless. Introducing state will likely be complex to maintain from a technical perspective. In the context of the export flow (in the case described above, usb drives), we want the export to be an atomic operation initiated from the client in sd-svs and requiring no further interaction from the user. The user should not even be aware of sd-export.

Could a "safe" printer get designated for each workstation laptop—as that one peripheral is unlikely to ever change?

Yes, we could probably set per device (vendor/model, but no serial number) rules to auto-attach to the export VM.

The printing flow is still a bit unclear to me (I am not sure if one could trigger a print via CLI to avoid user interaction in sd-export other than them changing the print settings like duplex or other).

@eloquence
Copy link
Member

Thanks, @emkll, for putting this together. The idea of an essentially stateless "pass-through" VM for export definitely has a lot of appeal. I think we'll want to contrast it with a "staging area" VM which could serve as a starting point for redaction, printing, or export. Perhaps these two concepts are even complementary.

Either way, automatically attaching storage devices would be a huge win for usability. Our research shows that the device attachment process is quite confusing; even with guiders, users struggle to make sense of it, and may also fundamentally misconstrue what it is they're doing.

Questions:

  • Do you see GPG encryption as mandatory for export? So far we've considered it optional. Would this encryption be to a journalist key? If so, how would it be configured? It's worth noting that different exports may go to different recipients.

  • What would be your biggest concerns with an approach that uses a session-persistent staging area instead of a stateless pass-through VM? (Export launches Nautilus, the user does what they want with exported files -- copy to storage, print, etc.) Is it the potential mixing of files from different sources, or are there other technical or security concerns as well?

@ninavizz
Copy link
Member

Very cool, thx for the clarification @emkll. Yes, we're in total agreement then, on general interests (my expectations are/were also that it'd be disposable per session‚ tho I probably confused that via incorrectly attributing tying that to authentication).

@emkll
Copy link
Contributor

emkll commented May 13, 2019

Do you see GPG encryption as mandatory for export? So far we've considered it optional. Would this encryption be to a journalist key? If so, how would it be configured? It's worth noting that different exports may go to different recipients.

Yes, given how easy it is to lose/misplace a usb device, I think we absolutely must enforce encryption of the device and/or the exported submission in some way. While full disk encryption is probably best to reduce metadata should the disk be lost/compromised, there are several other ways we can encrypt these exports:

  1. We can do classic SecureDrop-style transfer keys with LUKS, but the multi-platform support is problematic
  2. We could use VeraCrypt for encrypting the device but might be complex from an automation perspective
  3. We can use GPG with keypairs, and the key management issue you bought up might be a problem
  4. We can use GPG in symmetric mode (effectively using a passphrase and providing compatibility across a wide-range of systems). I propose we implement this solution as part of the prototype export flow for the reasons described as well as ease of implementation.

What would be your biggest concerns with an approach that uses a session-persistent staging area instead of a stateless pass-through VM? (Export launches Nautilus, the user does what they want with exported files -- copy to storage, print, etc.) Is it the potential mixing of files from different sources, or are there other technical or security concerns as well?

I think the "export launches Nautilus" flow you describe sounds interesting, but would be curious to see how well this will work in practice. Provided comprehensive training and documentation this approach seems more flexible for an end-user. However, this VM will possibly have network connectivity (onionshare), it would be best to keep this on the rails and automate as much as possible to minimize potential user error, given the complexity of Qubes.

@eloquence
Copy link
Member

So, thinking this through a bit more, @emkll, I think that the approach you describe would work well with the "add to briefcase" workflow prototyped here (click pink arrows in top right to advance). In this scenario, initiating a batch "Export" operation would do the following:

  • prompt or generate a passphrase for the user
  • ask the user to insert the storage device

Once they have done both, the export would be performed via the passthrough VM as described in your diagram, and encrypted symmetrically via gpg using the provided or generated passphrase.

For more advanced users, we could offer an "export to file manager" option. This would copy the files to the VM (using the same disposable VM template) instead of directly to the storage device. On that VM, the user could then attach/detach devices, and perform other operations such as "convert to trusted PDF".

Rationale

While I was hoping we'd be able to avoid the full complexity of the briefcase flow, in almost any export workflow that involves multiple files, without an "add to briefcase" workflow within the client, we'll likely have to prompt the user multiple times in annoying ways (passphrase, "attach storage device now", etc.), and we might also have to incur the disposable VM launch penalty multiple times.

The "Add to briefcase" flow has the big advantage that whatever we need to ask the user, we can ask them exactly once -- before they start an export operation, whether it includes 1 file or 1000.

Additional considerations

  • Network exports: Secure network exports via tools like OnionShare will most likely be out of scope for beta, but if we do want to support them, perhaps it would make sense to do that via a separate VM template? That way, we could keep "printer/storage device access" and "network access" mutually exclusive.

  • On rails vs. free-form: I agree that there's a lot of appeal in keeping the process on rails. Where I think this will get complicated is in anticipating the kind of processing steps a journalist may want to run on a document. For example, converting a file to a trusted PDF makes it much harder to work with, and may not be the first step a journalist wants to take. An export may involve multiple file formats that require different conversion steps. The journalist may want to archive files, or concatenate them, or split them up.

    This way lies potential madness -- checkbox nightmares, or "pipeline" dreams. I think our focus for now should be bundling the right tools and making them accessible in a secure way. Over time we can add guides and workflows that are optimized for the desire paths users typically want to follow.

    That's why I think a file manager option for more advanced users may be a good place to start from, alongside a "simplest possible export" scenario: encrypt + copy to USB.

@eloquence
Copy link
Member

For the 5/15-5/30 sprint, we committed to a CLI-only proof-of-concept implementation of the passthrough VM approach described in Mickael's comment, to aid in further UX prototyping that is more closely aligned with what's technically feasible. That should ideally include the automatic mounting of a storage device, but the scope of the proof-of-concept will be constrained by a 16 hour timebox. @rmol and @emkll will likely drive this proof-of-concept.

A video (screencast or cell phone recording) would be very helpful for UX purposes.

@eloquence
Copy link
Member

eloquence commented May 21, 2019

Had a quick chat w/ @redshiftzero @rmol @emkll today. Recap of my understanding based on this conversation:

  • After the user initiates an export, we should be able to auto-detect if the export VM is ready and a storage device is attached and can be successfully mounted.

  • Since the VM doesn't hold any data, we don't have to spin it up / shut it down for each export, so the actual export operation should be very fast (of course depending on file size).

  • Let's aim to validate those assumptions in the CLI-only experimental export flow that @rmol and @emkll will collaborate on during this sprint.

  • If both of these assumptions hold true, we may yet be able to avoid implementing a "Briefcase" flow for collecting files. The user would just click the "Export" action for any individual file. If the storage device is ready, we will just go ahead and encrypt & copy the file (I'm assuming we can show "busy" / "done" indicators for any individual export). If not, we'll prompt the user.

Open Questions: Passphrase Management

We had consensus that we want to generate a diceware passphrase for the user, and that the passphrase would likely be per-workstation (not per-user, per-session, or per-export). The export passphrase mainly mitigates against operational security failures in managing the storage device.

That said, the following open questions remain:

  • When does the user get informed about the per-workstation export passphrase? (During provisioning? First-run client configuration? First export?)

  • How do we want to advise the user to store the passphrase? (They may want to take the export device with them to another room, so most likely will just write it down if we don't advise them to do something else.)

  • What does the user do when they want to cycle the per-workstation export passphrase for security reasons? Is that a must-do feature for a production rollout?

  • If we do support cycling, do we have to ensure that a record is kept of previously used passphrases, so that the user does not lose access to previously encrypted files?

Input from both a security and a UX perspective on these questions would be appreciated. I'll take a first stab of my own at that in a follow-up comment.

Beyond single file export

We've agreed for now to treat single file export as the first iteration. Support for printing or other advanced features ("trusted PDF" etc.) may slip into post-beta territory. That's a big deal for newsrooms, who'd have to copy file to other machines for purposes like printing, so I would like to treat printing at least as a should feature, one we'll try very hard to keep in scope. But let's see what hurdles we encounter in this first iteration before making a final decision.

@eloquence
Copy link
Member

eloquence commented May 29, 2019

@emkll @rmol I've blocked away some time for us to go over the workflow prototyped in #259, as well as other findings from your spike (currently scheduled for 6/5). What would be super-helpful prior to then is to have a series of screenshots, photos, or a cell phone video sequence documenting the export workflow described in #259, to help think through the user experience we may want to build around that. Is that something either of you would be able to provide?

@eloquence
Copy link
Member

eloquence commented Jun 21, 2019

Capturing my understanding of the current consensus on open questions:

When does the user get informed about the per-workstation export passphrase? (During provisioning? First-run client configuration? First export?)

The user would be prompted on the first export of a given client session to enter the passphrase, after which it would be retained for the length of that session. (Detail to resolve: how to handle multiple drives with different passphrases.)

How do we want to advise the user to store the passphrase? (They may want to take the export device with them to another room, so most likely will just write it down if we don't advise them to do something else.)

We would recommend that they use their existing password manager, most likely on their phone.

What does the user do when they want to cycle the per-workstation export passphrase for security reasons?

Because the workstation would not retain export passphrases beyond the duration of a session, this is not an issue.

Since we have a starting point from which to iterate in master, and a broadly agreed upon design proposal as well (freedomofpress/securedrop-ux#57), I'm proposing that we close this ticket, and work with more tightly scoped tickets from here on.

@ninavizz
Copy link
Member

ninavizz commented Jun 25, 2019

Logic in latest IxD that will navigate users through export. Plopping here per @emkll request.

Below, from current wireframes:

image

@eloquence
Copy link
Member

Closing this issue per previous comment as I think we have broad consensus on the high level architecture and rough UX flow for the first implementation. For cross-cutting discussion for print & export, and to track all related issues, I've opened #290.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants