DEV Community

Discussion on: Rails queueing system, should I use delayed_job or sidekiq?

Collapse
 
rhymes profile image
rhymes • Edited

In a previous gig I switched a system from Delayed job to Resque and then to Sidekiq (we were wary about switcthing to Sidekiq directly because we had some thread safety issues that were resolved).

My personal suggestion: if you use Rails you can distance yourself a bit from the underlying system with ActiveJob.

Regarding charateristics: DJ stores the queue in the DB and to use it or not to use it really depends on which kind of job consuming patterns you have.

We were building a push notification delivery system for enteprise customers which meant that in moments we had 0 jobs on the queues and in others 300 thousand to process really quickly. If your jobs are time sensitive (as push notifications are) DJ is a bad idea.

DEV runs on delayed job even, the reason why it works it's because those jobs don't have to be completed nearly simultaneously.

Resque and Sidekiq (see @molly_struve's post about it) are far better but they enlarge the surface by introducing Redis in your toolbox, which in some cases might be more than you want.

I personally think it's worth learning about Redis (and you can use it as a SaaS) in general.

Sidekiq is also quite faster than Resque (Rails is not great at multi processing concurrency) and faster than DJ.

I remember how happy we were when, after switching to Sidekiq, we went to the PMs and said: "well, now we can send those notifications literally 30 times faster"

So my advise is: start with delayed job and if that's enough for your needs stay there, otherwise migrate to sidekiq

Don't forget to configure Redis for persistency, you don't want to lose your queues in case of restart ;-)