Skip to content

Latest commit

 

History

History
94 lines (72 loc) · 3.74 KB

README.md

File metadata and controls

94 lines (72 loc) · 3.74 KB
Raw

aws-sqs-filter-redrive

Donate! FOSSA Status

aws-sqs-filter-redrive is a tool to selectively redrive Amazon Simple Queue Service (SQS) messages based on their content. With this tool, you can get and filter messages from a queue based on their attributes or JSON body fields, and then move or delete them to a new queue.

Installation

Download the latest binaries and run

Or install via go install

go install github.com/natenho/aws-sqs-filter-redrive@latest

Usage

aws-sqs-filter-redrive [OPTIONS]

Here are the available options:

  • -source: The URL of the source queue.
  • -target: The URL of the target queue.
  • -count: The number of messages to delete or move.
  • -delete: Delete the messages instead of moving them.
  • -move: Move the messages to the target queue.
  • -dry: Perform a dry run to show which messages would be processed.
  • -workers: The number of workers to use (default: 8).
  • -polling-duration: The duration for polling messages from the source queue (default: 30s).
  • -rate-limit: The maximum number of messages to process per second (default: 10).
  • -before: Get only messages sent before a certain date in format 2006-01-02T15:04:05Z07:00.
  • -after: Get only messages sent after a certain date in format 2006-01-02T15:04:05Z07:00.
  • -attribute-filter: Filter messages by a certain attribute using JQ expression.
  • -body-filter: Filter messages by JSON body field content using JQ expression.
  • -message-attribute-filter: Filter messages by message attribute using JQ expression.

Examples

Move messages that were sent after a date

aws-sqs-filter-redrive \
  -source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
  -target https://sqs.us-east-1.amazonaws.com/123456789012/target-queue \
  -move \
  -after 2022-01-01T00:00:00Z

Delete messages sent before a date

aws-sqs-filter-redrive \
  -source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
  -delete \
  -before 2022-01-01T00:00:00Z

Move messages from one queue to another based on a JSON body field

aws-sqs-filter-redrive \
  -source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
  -target https://sqs.us-east-1.amazonaws.com/123456789012/target-queue \
  -move \
  -body-filter '.field == "value"'

Delete messages from a queue based on an attribute

aws-sqs-filter-redrive \
  -source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
  -delete \
  -attribute-filter '.AttributeName == "AttributeValue"'

Delete messages from a queue based on a message attribute. Message attributes have different value types, so you need to specify the type of the value you are filtering.

aws-sqs-filter-redrive \
  -source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
  -delete \
  -message-attribute-filter '.AttributeName.StringValue == "AttributeValue"' \
  -count 10000 -polling-duration 60s

Contributing

If you find a bug or have a feature request, please open an issue or submit a pull request. Contributions are always welcome!

License

FOSSA Status