RQ Retry is a package that adds retry functionality to the RQ queueing system. It can retry failed jobs immediately or optionally schedule them to retry after a delay using RQ Scheduler.
pip install rq-retry
pip install rq-scheduler # optional
Run worker process:
rqworker -w rq_retry.RetryWorker
rqscheduler # optionalThe RetryWorker performs work like a regular RQ Worker. In addition it periodically checks for jobs on the failed queue and requeues them to be run again. If a job fails too many times it is moved to the dead letter queue and will not be tried again.
RQ Retry can be configured with environment variables.
| Name | Default | Description |
|---|---|---|
| RQ_RETRY_MAINT_INTERVAL | 30 | How often, at most, to check for failed jobs. The check is not guaranteed to happen every RQ_RETRY_MAINT_INTERVAL seconds. See Maintenance Tasks. |
| RQ_RETRY_MAX_TRIES | 3 | Maximum number of time a job may be attempted. This includes the first time the job was run. So a value of 3 will try the job 2 times. A value of 1 or less disables retry. Zero has no special meaning. |
| RQ_RETRY_DELAYS | 5 | How long to delay each job retry in seconds. This can be a single float value or a comma separated list of floats. A simple exponential backoff can be achieved by using a value like 3,10,60. This causes the first retry of each job to be delayed 3 seconds, the second retry 10 seconds, and the third and subsequent retries to be delayed 60 seconds. To disable, set to any non numeric value like disabled. When delays are disabled jobs are requeued immediately without using RQ Scheduler. |
| RQ_RETRY_DEAD_LETTER_QUEUE | dead_letter_queue | name of dead letter queue |
rqworker is a command line wraper around an RQ Worker. It configures a Worker instance and then calls its work() method which loops indefinitely looking for jobs and running them. Workers also perform maintenance tasks periodically in their work loop. In the case of RetryWorker maintenance tasks include retrying failed jobs.
This means that RetryWorker failed job handling is only performed when the RetryWorker wakes up to handle a "regular" job from a queue. For this reason it is suggested that RetryWorker be used as a drop in replacement for Worker. In an active system this will ensure that failed jobs are retried promptly.
Pull requests welcome.
Getting started:
make install-dev
make testNote for conda users
If you're using conda some tox envs may not be able to run tests. To solve this update your PATH so that no python, python3, etc. points at a Python managed by conda. For example on OS X:
export PYTHON_ROOT="/Library/Frameworks/Python.framework/Versions/3.4"
export PATH="$PYTHON_ROOT/bin:$PATH"
make install-dev
make test