ActiveStoage is a superb piece of code that makes uploads and file manipulation, resizing, and previews like a breeze on your ruby on rails application.
When you're, like me, hosting your app on Heroku, using s3 or other external storage to persist your file uploads, ActiveStorage makes it as easy as adding some lines on a config file.
Using ActiveStorage was for me a benediction, small efforts to maintain and it works reliably most of the time.
One of the main features of the ActiveStorage is the ability to resize pictures and create variants.
It uses ImagicMagic ou LibVibs libraries on the background to make this happen.
To create a variant we need to write:
# resize the picture to fit 200px by 200px by respecting the ratio. <%= image_tag picture.variant(resize: "200x200", auto_orient: true) %>
When the browser fetches the image for the first time, ActiveStorage will check if it has the asked variant of this picture ready, if not it will call in sync the ImagicMagic process to do the resizing and send it back to the browser.
If the picture is stored to
S3 from aws for example, the process will follow:
- Download the image from S3 into the server
- Call the process to resize the picture
- Upload the variant to S3 and update the database
- Send an HTTP redirect to the new location of the image on S3
This a lot of work and it takes time (some seconds on small dynos).
Meanwhile, the process/thread is too busy and cannot work on other requests.
Imagine, one page contains 5 or 6 pictures, this will make your server busy for resizing all the pictures, may cause a significant slow down for the following requests.
You can tell me, it's not an issue, it's done only once, but when you have an app with uploads occurring all the day, running on a small dyno, this will be a serious issue.
If you care about the user experience and the confidence on your product, you should fix it.
You can throw more money on you Heroku, faster CPU the mitigate the issue, bare with me, I have a solution for you and will cost you only some lines of codes 😇
To make your web server happy, overload all the heavy works with your background workers.
When an upload is done, start a background job to process the variants
class ResizePhotoJob < ApplicationJob queue_as :default def perform(file, resize_cmd:nil) if resize_cmd.nil? file.variant(auto_orient: true).processed else # the `.processed` will force the resizing to be done in sync file.variant(resize: resize_cmd, auto_orient: true).processed end end end # and call it for your different needed variants ResizePhotoJob.perform_later(file) ResizePhotoJob.perform_later(file, resize_cmd: "250x250")
Of course, you need to write some tests for this! I will help you
require 'test_helper' class ProcessPhotosFromAttachmentJobTest < ActiveJob::TestCase test 'resize photos' do profile = profiles(:jamal) profile.avatar.attach(io: File.open(file_fixture("photo_1.jpeg")), filename: 'image') file = profile.avatar key = file.variant(resize: '200x200', auto_orient: true).key refute file.service.exist?(key) ResizePhotoJob.perform_now(profile.avatar, resize_cmd: "200x200") key = file.variant(resize: '200x200', auto_orient: true).key assert file.service.exist?(key) end end
Each variant on the ActiveStorage has a key ( a simple hash on the ImageMagic params )
The variant is stored and retrieved using this key.
# ==> retrieve the variant key file.variant(resize: '200x200', auto_orient: true).key
This test will see if the variant exists before the job executions and its existence after.
That's good, now we process the variants on the background, and hopefully when the user tries to access the picture, he will find the variant processed and ready to be served.
Unfortunately, it's not always the case 😔
Even worse, some pictures will fail to resize, for example on a small Heroku dyno, there is not enough memory allowed to ImageMagic to process some big pictures > 15Mb.
I got this error when resizing an image of 15mb (12000x9000) on a Hobby Dyno on Heroku
convert /tmp/ActiveStorage-36637-20210128-4-18i0017.jpg -auto-orient -resize 250x250 -auto-orient '/tmp/image_processing20210128-4-gnzbsa.jpg' failed with error: convert-im6.q16: DistributedPixelCache '127.0.0.1' @error/distribute-cache.c/ConnectPixelCacheServer/244. convert-im6.q16: cache resources exhausted '/tmp/ActiveStorage-36637-20210128-4-18i0017.jpg' @error/cache.c/OpenPixelCache/3984.
Hopefully, in my case, this is not the regular size of pictures uploaded on my app.
For those cases, our job is useless, each request will start all the optimization task and it will take too much time.
These requests will end up with a
503 timeout and make worse the process/thread availability to process our regular requests.
My solution is dead simple:
When the variant is not available, do not resize and redirect to the original picture.
Okay, the user will download a big picture, that's good enough, the user will get his picture, okay it will take much longer to download but will use the
S3 server resources, not mine 😆.
My app will only redirect the browser to the
S3 servers when the actual download will occur.
We will use the same code used on the tests to check if the variant is present.
<% variant = picture.variant(resize: "200x200" , auto_orient: true) %> <% if picture.service.exist?(variant.key) %> <%= image_tag variant %> <% else %> <%= image_tag picture %> <% end %>
This small line of codes will make stay a little longer on small cheap Heroku dynos, make your app more resilient, and make you save money.
Glad to hear if you have done this kind of Budget optimization in another way.