Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure consumer tag names/prefixes #265

Merged
merged 3 commits into from
Nov 28, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,11 @@ In order from lowest to highest precedence:

### Generated list of configuration options

Generate with

0. `yard doc lib/hutch/config.rb`
0. Copy the _Configuration_ section from `doc/Hutch/Config.html` here, with the anchor tags stripped.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

♥️ 💛 💚 💙 💜

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This is you coming back to improve the README: 🎩 !)


<table border="1" class="settings" style="overflow:visible;">
<thead>
<tr>
Expand Down Expand Up @@ -595,6 +600,15 @@ In order from lowest to highest precedence:
<td><p>Should Bunny's consumer work pool threads abort on exception.</p>
</td>
</tr>

<tr>
<td><tt>consumer_tag_prefix</tt></td>
<td>hutch</td>
<td>String</td>
<td><tt>HUTCH_CONSUMER_TAG_PREFIX</tt></td>
<td><p>Prefix displayed on the consumers tags.</p>
</td>
</tr>

</tbody>
</table>
Expand Down
3 changes: 3 additions & 0 deletions lib/hutch/config.rb
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,9 @@ def self.boolean_setting(name, default_value)
# The option is ignored on JRuby.
boolean_setting :consumer_pool_abort_on_exception, false

# Prefix displayed on the consumers tags.
string_setting :consumer_tag_prefix, 'hutch'

# Set of all setting keys
ALL_KEYS = @boolean_keys + @number_keys + @string_keys

Expand Down
11 changes: 10 additions & 1 deletion lib/hutch/worker.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
require 'hutch/acknowledgements/nack_on_all_failures'
require 'hutch/waiter'
require 'carrot-top'
require 'securerandom'

module Hutch
class Worker
Expand Down Expand Up @@ -44,7 +45,7 @@ def setup_queue(consumer)
queue = @broker.queue(consumer.get_queue_name, consumer.get_arguments)
@broker.bind_queue(queue, consumer.routing_keys)

queue.subscribe(manual_ack: true) do |*args|
queue.subscribe(consumer_tag: unique_consumer_tag, manual_ack: true) do |*args|
delivery_info, properties, payload = Hutch::Adapter.decode_message(*args)
handle_message(consumer, delivery_info, properties, payload)
end
Expand Down Expand Up @@ -103,5 +104,13 @@ def error_acknowledgements
private

attr_accessor :setup_procs

def unique_consumer_tag
prefix = Hutch::Config[:consumer_tag_prefix]
unique_part = SecureRandom.uuid
"#{prefix}-#{unique_part}".tap do |tag|
raise "Tag must be 255 bytes long at most, current one is #{tag.bytesize} ('#{tag}')" if tag.bytesize > 255
end
end
end
end
20 changes: 19 additions & 1 deletion spec/hutch/worker_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,27 @@
end

it 'sets up a subscription' do
expect(queue).to receive(:subscribe).with(manual_ack: true)
expect(queue).to receive(:subscribe).with(consumer_tag: %r(^hutch\-.{36}$), manual_ack: true)
Copy link
Contributor Author

@sldblog sldblog Nov 24, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I need to double check AMQP/RabbitMQ spec for the maximum available length of the consumer tag name?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be a cool touch.

(Then we could figure out ways to validate configuration options. Example addition to the Config options' DSL: max_length: 34.)

Copy link
Contributor

@olleolleolle olleolleolle Nov 24, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It "depends", it seems: https://www.rabbitmq.com/amqp-0-9-1-reference.html#basic.consume.consumer-tag it is a shortstr, defined as "shortstr shortstr [short string]"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it's 255 bytes max: https://github.com/rabbitmq/rabbitmq-common/blob/rabbitmq_v3_6_0/codegen.py#L178-L179 ... I hope that's the same in 3.4.x as well :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, now we know that the longest prefix that can be used is: 256 - 1 - 36 = 219 bytes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally Hutch should loudly complain about prefixes that are longer, e.g. log an error and trim the prefix or fail to start.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opted for an "on-create" validation as validating the actual prefix length is weird (why 219 bytes?) and coupled to the implementation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the on-create validation sufficient/ok?

worker.setup_queue(consumer)
end

context 'with a configured consumer tag prefix' do
before { Hutch::Config.set(:consumer_tag_prefix, 'appname') }

it 'sets up a subscription with the configured tag prefix' do
expect(queue).to receive(:subscribe).with(consumer_tag: %r(^appname\-.{36}$), manual_ack: true)
worker.setup_queue(consumer)
end
end

context 'with a configured consumer tag prefix that is too long' do
let(:maximum_size) { 255 - SecureRandom.uuid.size - 1 }
before { Hutch::Config.set(:consumer_tag_prefix, 'a'.*(maximum_size + 1)) }

it 'raises an error' do
expect { worker.setup_queue(consumer) }.to raise_error(/Tag must be 255 bytes long at most/)
end
end
end

describe '#handle_message' do
Expand Down