You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm facing the following issue. Whenever a Sidekiq Worker is picking up a Job from a queue, it take up to 10 seconds until it appears in the "Processing" queue. As the Job gets immediately removed from the queue, the Job basically disappears inbetween.
I noticed, that the Worker reports the status of the jobs with every heartbeat (here). Is there a way to trigger this update immediately?
The disappearance of the job causes several issues (e.g. this).
In order to reproduce the error, I set up a minimal rails app (here). This repo additionally contains sidekiq-unique-jobs, but the described issue appears even without it. To reproduce it, clone the repo and run the following command:
docker compose up app app_worker
Go to http://localhost:3000/sidekiq and then run the following command:
You will see in the console, that the worker immediately picks up the job, but in the Web UI it takes up to 10 seconds. I checked the entries in the redis DB as well (as the Web UI is polling the server as well in an interval of about 5 seconds), and it takes as well about 10 seconds, before the job appears as processing there.
The text was updated successfully, but these errors were encountered:
Is this a performance consideration? I had already several scenarios in which the wrong status info of a job leads to errors -specially if the job disappears completely. Wouldn't it make sense to update the job status in real-time?
Yes, for performance. If you are running 100 jobs/sec, that's 100 round trips to Redis. So the more work you do, the more it slows down. That's poor scalability.
Ruby version: 3.2.2
Rails version: 7.1.2
Sidekiq / Pro / Enterprise version(s): 7.2.0
I'm facing the following issue. Whenever a Sidekiq Worker is picking up a Job from a queue, it take up to 10 seconds until it appears in the "Processing" queue. As the Job gets immediately removed from the queue, the Job basically disappears inbetween.
I noticed, that the Worker reports the status of the jobs with every heartbeat (here). Is there a way to trigger this update immediately?
The disappearance of the job causes several issues (e.g. this).
In order to reproduce the error, I set up a minimal rails app (here). This repo additionally contains
sidekiq-unique-jobs
, but the described issue appears even without it. To reproduce it, clone the repo and run the following command:Go to
http://localhost:3000/sidekiq
and then run the following command:You will see in the console, that the worker immediately picks up the job, but in the Web UI it takes up to 10 seconds. I checked the entries in the redis DB as well (as the Web UI is polling the server as well in an interval of about 5 seconds), and it takes as well about 10 seconds, before the job appears as processing there.
The text was updated successfully, but these errors were encountered: