Making my own “sidekiq” for Rails - part 1.

In web development, when we have a task that takes a long time to process, we typically use a background job processor. This means that the task is delegated to another process, allowing the main application to run without being delayed by the time-consuming task.

In most cases in the Ruby on Rails (RoR) world, Sidekiq plays this role. But how does it work?

This post will demonstrate how to create your own version of 'Sidekiq'.

Load dependencies

The first step is to load all dependencies from the main application. The background processor should be aware of how to initialize the main application. In Ruby on Rails (RoR), you can achieve this by running something like the following using irb.

#!/usr/bin/env ruby
require_relative "application"
require_relative "boot"
require_relative "environment"

If you want to know more details about this process, please refer to the Rails documentation.(here)

Let's start a Rails application and attempt to run it using IRB.

rails new my_person_app
cd my_person_app
rails g controller VeryExpensiveTask index
rails g model FooEntity start_task:datetime end_task:datetime
rake db:migrate

Create the class FooEntity.

class FooEntity < ApplicationRecord
 def verify_expensive_task
  self.save if new_record?

  update_column(:start_task, Time.now)
  sleep rand(10..60)
  update_column(:end_task, Time.now)
 end
end

Run this in IRB.

require_relative "application"
require_relative "boot"
require_relative "environment"

FooEntity.new.verify_expensive_task

In the VeryExpensiveTaskController, let's create an action that calls the verify_expensive_task method.

class VeryExpensiveTaskController < ApplicationController
  def index
    FooEntity.new.verify_expensive_task
  end
end

# route.rb

Rails.application.routes.draw do
  get 'very_expensive_task/index'
end

# index.html.erb

<h1>VeryExpensiveTask#index</h1>
<style>
table, th, td {
  border:1px solid black;
}
</style>
<table>
 <thead>
  <th>ID</th>
  <th>Inicio</th>
  <th>Fim</th>
 </thead>
 <tbody>
  <% FooEntity.all.each do |foo| %>
   <tr>
    <td><%= foo.id %></td>
    <td><%= l foo.start_task, format: :short %></td>
    <td><%= foo.end_task ? l(foo.end_task, format: :short) : "-"%></td>
   </tr>
  <% end %>
 </tbody>
</table>

Initialize the application and access the page localhost:3000/very_expensive_task/index. Check the duration of the page load.

Write the background processing

Now, we need a method for exchanging messages between processes—the main application and the background application.

In this example, we'll employ a straightforward method by opening a socket for the main application to send messages. However, there are various methods available for inter-process communication (IPC). You can explore different IPC methods by searching for 'inter-process communication (IPC) methods' on Google.

Create a file(i give voadora.rb name for my file) in config path of project and than put this code.

require 'socket'

require_relative "application"
require_relative "boot"
require_relative "environment"

# Init a tcp server
server = TCPServer.new 2000 # Server bind to port 2000

puts "Waiting for job..."
# Loop that listener connections
loop do
  client = server.accept    # Wait for a client to connect
  if client
    puts "Job receveid" 
  # Using thread for many connections accept. Try run without 
  # threads =(.
    Thread.new do 
        puts "Init job"
          FooEntity.new.verify_expensive_task 
        puts "Finish job"
    end
  end
  client.puts "Job scheduled!"
  client.close
end

Run the code.

ruby config/voadora.rb 
>>> Waiting for job...

Voala!

Change the controller code for send message to background processor.

...
def index
 client = TCPSocket.open('localhost', 2000)
 client.gets
end
...

Reload the page and you can see backgroud process receive message and process the job.

It's a very simple example to show you how you can write a background process, and we can make many improvements to the code. In the next post, I will add Redis for IPC alternative and encapsulate the job definition in one class.