A Dramatiq broker that can be used with Microsoft Azure queue services.
Heavily inspired by Dramatiq SQS, this package currently implements a broker for Azure Storage Queue. An implementation for Azure Service Bus is planned... eventually.
pip install dramatiq-azure
pip install dramatiq-azure[identity] # for passwordless authentication
import dramatiq
from dramatiq.middleware import AgeLimit, TimeLimit, Callbacks, Pipelines, Prometheus, Retries
from dramatiq_azure import ASQBroker
broker = ASQBroker(
dead_letter=True,
middleware=[
Prometheus(),
AgeLimit(),
TimeLimit(),
Callbacks(),
Pipelines(),
Retries(min_backoff=1000, max_backoff=900000, max_retries=96),
],
)
dramatiq.set_broker(broker)
The following authentication methods are supported by the broker:
- Connection string based:
AZURE_STORAGE_CONNECTION_STR
environment variable must be set. If this variable is not set, passwordless authentication will be used. Creating a connection string for your Azure account is documented here. - Passwordless (token-based) authentication (Recommended):
AZURE_STORAGE_ACCOUNT_NAME
environment variable must be set.
The list of other mandatory variables depends on where the app is being run. More information can be found here.
The following environment variables can be used to configure the broker:
AZURE_STORAGE_CONNECTION_STR
: Azure Storage connection string;AZURE_STORAGE_ACCOUNT_NAME
/AZURE_ACCOUNT_NAME
: Azure Storage account name;AZURE_ENDPOINT_SUFFIX
: Azure Storage endpoint suffix;AZURE_SSL
: Whether to use SSL for the connection;AZURE_QUEUE_ACCOUNT_URL
: Azure Storage account URL;DRAMATIQ_ASQ_MIN_TIMEOUT
: The minimum time to wait between polls in second.
Tests require a running Azurite instance. You can easily launch azurite
through Docker.
docker run -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite
Run the test suite
pytest
Found an itch you know how to scratch? PR welcome (just remember to read the contribution guide) !