I keep meaning to write up something about find_in_batches for this type of scenario.
I have tested quite a few things in this area, and the one which seems to work best is adding a select and pluck:
Contact.not_opted_out.select(:id).find_in_batches do |batch|
# call Sidekiq with batch.pluck(:id)
This lets you fully leverage ActiveRecord, but minimize an model creation/etc.
Ah, good point! We actually use pluck in a piece of code very similar to this at work. Select and pluck seem like a powerful combo for this kind of thing. If I have time later I’ll add a benchmark for that.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.