Avoid deadlocks

[vkvam]

Generalise some mechanism to shut down all threads when one thread/process fails.

[vkvam]

Brute force could by a keep-alive signal back from pipeline while running everything in a Process.

[vkvam]

All threads are now wrapped in a FileGeneratorResponse, allowing something other than the FileGenerator itself to start/observe/join threads.

[vkvam]

One remaining problem to overcome is that it will be impossible for a thread to trigger an exception on the main thread while the main thread is locked in a read() call.

An option could be to wrap the read() call with a queue that requests results from a child thread doing the actual read call.

The read call (on the main thread), could periodically check for exceptions on any thread and raise them.