Begin a major rewrite with rrq

[wlandau]

rrq is a Redis-based task queue for R. It manages communication between the controller and the workers. More specifically, it brokers tasks, dynamically down-scales workers, and handles errors. But it does not start the Redis server, and it can only launch workers on the local machine. My goal is to make crew focus on these last two pieces. In the upcoming phase of development, crew will serve as a launcher for rrq and a future-like unifying R interface to many different backend computing environments that run the workers. I will start with traditional schedulers like SGE and SLURM, where it is standard practice to trust the local network and establish direct Redis connections between the controller and the workers. Later on (outside the scope of this first issue) I will work on a more secure communication layer between the user and the controller in cases where ports cannot be safely exposed (AWS EC2, Batch, etc.)

[wlandau]

The body of rrq_worker_main() looks useful:

> rrq:::rrq_worker_main
function (args = commandArgs(TRUE)) 
{
    dat <- rrq_worker_main_args(args)
    worker <- rrq_worker_from_config(dat$queue_id, dat$config, 
        dat$name, dat$key_alive)
    worker$loop()
    invisible()
}

[wlandau]

probably can just call rrq_worker$new(...)

[wlandau]

I actually think in addition to {rrq}-based persistent workers, we could also have queues with completely transient workers. This might scale better in some cases, and it would make it easier to work with just the web APIs to monitor jobs. This approach would also let me warm up to using AWS Batch on a serious basis before I need to worry about the network programming that the {rrq} stuff requires. And it would make use for the work I put into the code base in the commits leading up to f6a5769.

[wlandau]

I know we have a mirai backend now (as of #29), but I also plan to implement the same with rrq because of shikokuchuo/mirai#26.

[wlandau]

If #30 and #31 work out when I test them this upcoming week, we can fully commit to mirai without any need for rrq/Redis

[wlandau]

Because of the recent major improvements in mirai, I doubt we will need rrq going forward.