Allow message_attributes to be set from ActiveJob clients

[timbreitkreutz]

We'd like to be able to access the SQS message attributes from the Shoryuken client code and middleware. Previously, a memoized copy of the needed attributes were used which interfered with middleware when jobs were nested (i.e. ActiveJob clients calling other Active Jobs). In addition, we had to work implement workarounds for metrics and tracing attributes at that level.

The suggestion here is to allow these to be passed in as options which then get used as the starting point. The necessary active job attributes are then overwritten on top of any that are passed in to be sure they function the same as before.

We are open to any suggestions on style, thanks!

Co-authored by @tamouse

[timbreitkreutz]

@phstc Here's the nicer version of the PR (#633) we deleted a couple weeks ago. Is the utility function in code climate a blocker? Is there anything else you'd like us to change or adjust on this one?

[timbreitkreutz]

Also how do we get Travis running?

[timbreitkreutz]

Update: We have been running this version in our main app for a couple weeks now and have no reported issues.

[phstc]

Hi @timbreitkreutz

I can't really review this now. But I added a form for "recruiting" Shoryuken maintainers. Let me know if you are interested 4a03b64

[phstc]

@timbreitkreutz Sorry for taking long to get back on this.

Is it being used when enqueuing a job? MyJob.perform_later(params)

I do miss the ability to send Shoryuken specific params (for example, FIFO params) when using Active Job is this change adding this support?

[timbreitkreutz]

Sorry was on vacation. Will try to get an answer soon :)

[timbreitkreutz]

Sadly I don't think there is a way to inject this via perform_later--unless I'm missing something. This PR was actually aimed more at the middleware layer.

[cjlarose]

@phstc I was able to test this locally. The proposed changes do not permit specification of message_attributes when using perform_later (I've documented that as a separate issue in #646).

Instead, the following can be used to enqueue a message with additional message_attributes, without overriding the shoryuken_class attribute.

class AddJob < ActiveJob::Base
  def perform(a, b)
    puts a + b
  end
end

job = AddJob.new 1, 2
AddJob.queue_adapter.enqueue(job, message_attributes: { 'attr' => { string_value: 'myval', data_type: 'String' } })

Everything looks good from a testing perspective, so I think we're good to merge it. Although it doesn't solve #646, it doesn't hinder it and might actually be an important prerequisite.

@timbreitkreutz Thanks so much for your patience on this one!

[cjlarose]

@timbreitkreutz If you could rebase on master, I'd appreciate it. Travis should run correctly after pushing up the rebased version. 🤞

[timbreitkreutz]

Thanks @cjlarose !

[timbreitkreutz]

Fork master has been updated. Merge with squash I believe will result in a single commit to shoryuken repo?

[cjlarose]

Thanks! Tests are green. I'll squash merge.

[cjlarose]

Whoops. I said I'd squash merge and then immediately forgot. The master branch is protected from force-pushes, so I think I'll just leave it. An extra merge commit won't hurt anyone.