As with a lot of web applications and APIs, there comes a point where you need to process some tasks in the background. To do this, you'll usually use a background job processor. This helps to make some things asynchronous and speed along the users of your app.
An example of this might be logging when your users login to your site. Maybe you store the IP address they used, when they logged in, how many attempts it took to sign in, and the device they used. If you did all this in-line, then the process would be to track all that info, and the user has to wait while you create this new record in your database before they can actually login. Or, you push that info to a background job, and log the user in. Then you can update that record at your own pace since there's nothing relying on the data.
If you're new to Crystal Lang and/or you're new to Lucky, but you're coming from another language like Rails, you may be familiar with Sidekiq. Now, Sidekiq does exist in the Crystal world which is amazing. It even exists by the same guy that created the Ruby version! But at the time of this writing, I spent nearly an hour trying to figure out how to get it integrated with Lucky, and I was running in to quite a few issues. Thankfully, I found an alternative that I had installed and running locally in maybe 2 minutes.
Take a look at Mosquito. The API for it really isn't too far off from what Sidekiq uses, so for me, the concept was quick and easy. This also uses Redis, and since I was going to use Redis with Sidekiq, there's no additional things I needed to install.
Assuming you have your Lucky app all setup and ready to go, you can start by adding Mosquito in:
dependencies: mosquito: github: robacarp/mosquito
shards install, and you'll be good to go.
Now we just need to setup our mosquito runner. Create a new file
require "./dependencies" require "./workers/*" Mosquito::Runner.start
Setting up workers
Next we need to setup the workers. Create a new folder
./src/workers/, and let's make a
login_worker.cr file in there.
class LoginWorker < Mosquito::QueuedJob params user_id : Int64 params ip : String def perform user = UserQuery.find(user_id) if user # create new Login record from the user end end end
Using the worker
At some point we have to kick off the
LoginWorker we made, so in one of the actions, we can add this:
post "/login" do form = SessionForm.new(params) form.submit do |f, user| if user LoginWorker.new(user_id: user.id, ip: "127.0.0.1").enqueue # redirect logged in else # show your errors end end end
Booting the whole thing
Now that you have mosquito installed, and setup, you'll just need to boot the whole app. Since Mosquito runs in it's own process, you have to run it in a separate process.
Procfile.dev add in the line
worker: crystal src/mosquito.cr.
web: lucky watch --reload-browser assets: yarn watch worker: crystal src/mosquito.cr
Now when you run
lucky dev, it'll boot mosquito in it's on separate process!
- If your app is like mine, and just an api, your
Procfile.devmight look more like
web: crystal src/server.cr worker: crystal src/mosquito.cr
Mosquito needs to know about your stack since it runs in a separate process. If you're trying to slim this down, just keep in mind that whatever you put in your worker file, you'll need to require that separately from your app stack.
By default, Mosquito looks at your
REDIS_URLENV, or pulls just a normal localhost value. Be sure to update that for production, as well as your
Procfilefor your production.
If you would rather build the targets (maybe for docker), and run those separately, you can update your
targets: server: main: src/server.cr mosquito: main: src/mosquito.cr
shards build --production --static --release --no-debug to get your
mosquito binary files. Just run those however you need, and you should be good.
Latest comments (2)
That's a great discovery (Mosquito) and a great combo (with Lucky).
Thank you for this post !
🚀 like it