taskq is brought to you by ⭐ uptrace/uptrace. Uptrace is an open source and blazingly fast distributed tracing tool powered by OpenTelemetry and ClickHouse. Give it a star as well!
- Redis, SQS, IronMQ, and in-memory backends.
- Automatically scaling number of goroutines used to fetch (fetcher) and process messages (worker).
- Global rate limiting.
- Global limit of workers.
- Call once - deduplicating messages with same name.
- Automatic retries with exponential backoffs.
- Automatic pausing when all messages in queue fail.
- Fallback handler for processing failed messages.
- Message batching. It is used in SQS and IronMQ backends to add/delete messages in batches.
- Automatic message compression using snappy / s2.
Resources:
To get started, see Golang Task Queue documentation.
Producer:
import (
"github.com/vmihailenco/taskq/v3"
"github.com/vmihailenco/taskq/v3/redisq"
)
// Create a queue factory.
var QueueFactory = redisq.NewFactory()
// Create a queue.
var MainQueue = QueueFactory.RegisterQueue(&taskq.QueueOptions{
Name: "api-worker",
Redis: Redis, // go-redis client
})
// Register a task.
var CountTask = taskq.RegisterTask(&taskq.TaskOptions{
Name: "counter",
Handler: func() error {
IncrLocalCounter()
return nil
},
})
ctx := context.Background()
// And start producing.
for {
// Call the task without any args.
err := MainQueue.Add(CountTask.WithArgs(ctx))
if err != nil {
panic(err)
}
time.Sleep(time.Second)
}
Consumer:
// Start consuming the queue.
if err := MainQueue.Start(context.Background()); err != nil {
log.Fatal(err)
}
- Golang ORM for PostgreSQL, MySQL, MSSQL, and SQLite
- Golang PostgreSQL
- Golang HTTP router
- Golang ClickHouse
Thanks to all the people who already contributed!