Gotcha: Starting Jobs inside Transaction

Starting jobs inside an Active Record Transaction can result in unexpected outputs. I've seen this happen multiple times in my career.

Consider an example where we update a model and then send a third party API the updated model

ActiveRecord::Base.transaction do
   david.withdrawal(100)
   mary.deposit(100)
   NotifyUser.perform_later david.id # Job to notify User
end

Lets assume that NotifyUser sends a user their remaining balance in account.

class NotifyUser < ApplicationJob
   queue_as :default
   def perform(user_id)
     user = User.find(user_id)
     ThirdPartyNotifier.notify(user)
   end
end

So you run this code in your local and it seems to work just fine. Money is withdrawn and the notification sends appropriate balance of the user. But alas, Later you find a bug in production, and weirdly sometimes the notification is correct, sometimes it not.

The reason? The job sometimes runs before the transaction can even complete. Since we queue the job inside the transaction, there is a chance that the Job gets picked up before changes are committed to the database. The job then has old data. If you want to make sure the job send incorrect data every time in local add a sleep(10) after NotifyUser. This will ensure that job gets picked up before the transaction completes.

The fix for this case is quite straightforward. Move the job call outside of the transaction

ActiveRecord::Base.transaction do
   david.withdrawal(100)
   mary.deposit(100)
end
NotifyUser.perform_later david.id # Job to notify User