:until_executing does not schedule job in Sidekiq 6.0.1 at all

[ArturT]

Describe the bug
:until_executing works with sidekiq 6.0.0 but not in sidekiq 6.0.1. In sidekiq 6.0.1 the job is never added to queue.

Expected behavior
Below is a log from rails console in development.

My sidekiq background process (bin/sidekiq) is not started as I'm running below commands in rails console.

:until_executing flag works correctly for sidekiq 6.0.0 and it won't add another job to queue if the first job was not started yet.

2.6.3 :001 > BuildDistributorWorker.perform_async(123)
 => "387db0680724e21d8bcc203a" <-- job was added, you get job id

2.6.3 :002 > BuildDistributorWorker.perform_async(123)
2019-10-03T18:54:28.379Z pid=69093 tid=owpgpn0yx middleware=SidekiqUniqueJobs::Client::Middleware unique_digest=uniquejobs:44c4cb8c6a188bfb6c3681a48e4d49ed WARN: payload is not unique {"class"=>"BuildDistributorWorker", "args"=>[123], "retry"=>true, "queue"=>"build_distributors", "lock"=>:until_executing, "log_duplicate_payload"=>true, "jid"=>"a029db1954a46b73250f406c", "created_at"=>1570128868.362438, "lock_timeout"=>0, "lock_expiration"=>nil, "unique_prefix"=>"uniquejobs", "unique_args"=>[123], "unique_digest"=>"uniquejobs:44c4cb8c6a188bfb6c3681a48e4d49ed"}
 => nil . <-- job was not created because the job already existed, that's expected. All is good.

Current behavior
In Sidekiq 6.0.1 the behavior is different. The job is never added to the queue even if there are no jobs in the queue.

2.6.3 :002 > BuildDistributorWorker.perform_async(1) <--- I use here ID=1 to ensure there is no job with such ID yet in the queue
 => nil <--- it should retun id of job in sidekiq instead of nil

Worker class

class BuildDistributorWorker
  include Sidekiq::Worker

  sidekiq_options lock: :until_executing, log_duplicate_payload: true, queue: :build_distributors

  def perform(id); end
end

[mhenrixon]

Yup, you can see that exact behavior in tests. Really not sure what they did but I also have completely broken test suite against sidekiq develop so they might be going through some rough times over on sidekiq

[mhenrixon]

@ArturT see sidekiq/sidekiq#4308 for an eventual fix.

[kapso]

So whats the fix for sidekiq-unique-jobs breaking with Sidekiq 6.0.1?

[mhenrixon]

Try v6.0.14

[ArturT]

@mhenrixon Thanks for the fix and new release. I've updated sidekiq-unique-jobs to 6.0.14 but now I see below error when call MyWorker.perform_async:

Errno::ENOENT: No such file or directory @ rb_sysopen - /Users/artur/.rvm/gems/ruby-2.6.5/gems/sidekiq-unique-jobs-6.0.14/redis/convert_legacy_lock.lua

I've checked the directory with installed sidekiq-unique-jobs and it looks like there is missing redis directory.

 ~/.rvm/gems/ruby-2.6.5/gems/sidekiq-unique-jobs-6.0.14 
$ ll
total 168
drwxr-xr-x    7 artur  staff    224 Oct  5 14:13 .
drwxr-xr-x  277 artur  staff   8864 Oct  5 14:13 ..
-rw-r--r--    1 artur  staff  57565 Oct  5 14:13 CHANGELOG.md
-rw-r--r--    1 artur  staff   1084 Oct  5 14:13 LICENSE.txt
-rw-r--r--    1 artur  staff  20046 Oct  5 14:13 README.md
drwxr-xr-x    3 artur  staff     96 Oct  5 14:13 bin
drwxr-xr-x    6 artur  staff    192 Oct  5 14:13 lib

I tried gem uninstall sidekiq-unique-jobs and then again install it with bundle install but the redis directory is still not present. Is it possible that you forgot to include redis directory in the latest release?

[mhenrixon]

Yup, will create a new release immediately! Sorry about that, I don't know an easy way to integration test this part.

[mhenrixon]

Gem yanked and v 6.0.15 released. I built and installed it locally and the directory is there

[ArturT]

@mhenrixon Thank you. It works well now. I've deployed the new version of your gem and new sidekiq to my production API https://knapsackpro.com :)