Migrating attachments to ActiveStorage on the go.

Background

A little while back, the team I was part of was tasked with migrating attachments from carrier_wave to active_storage. Basically the original task was to migrate the storage location of the files for the project that was originally configured to use carrier_wave.

This felt like a good opportunity to also migrate from using carrier_wave to active_storage since our application was already up to date with relatively recent rails version at the time and a solution provided out of the box from the framework makes more sense to use rather than relying on third party dependencies. Not to mention gems like carrierwave and paperclip have been the goto solution and rock solid for years before active_storage was introduced.

Possible Solutions 💡

Looking up online for similar case studies (migrate from carrier_wave/paperclip to active_storage), I found a number of very helpful articles explaining in detail how to achieve this, a common approach I observed was to do the migration in one go via some dedicated rake task or db migration. In our case due to some constraints it wasn't feasible for us to have a single long running task to migrate all existing attachments.

Considering these constraints we eventually decided to go with a different approach, perform the migration on the go. More precisely, migrate an attachment when its referenced from the codebase.

The final flow of events that we came up with looked something like this:

For new uploads:
This case was simple enough, new uploads will be uploaded via active_storage.

For existing uploads
For existing uploads, we devised this sequence of events:

1. Check if the requested attachment exists on active_storage
2. if yes, return the attachment from active_storage
3. if no, enqueue a job to attach the requested attachment to active_storage and in the meantime return the carrierwave attachment

Implementation 💻

Since I worked on this use case approximately a year ago, I will try and reconstruct the bits that I think are relevant and present those in the form of some pseudo implementation just to give an idea of the approach. I assume you will have active storage enabled already in your projects by this point.

However, if you want to ask a question specific to your use case please share those in the comment, I will try my best to guide you.

For the sake of this example lets take a simple use case where we have a Profile model having an attachment by the name of profile_picture.

The model will look something like this:

class Profile < ApplicationRecord
  mount_uploader :profile_picture, ProfilePictureUploader
end

1. Enable ActiveStorage

Firstly, we prepare our model to accept incoming attachments via active_storage.

class Profile < ApplicationRecord
  mount_uploader :profile_picture, ProfilePictureUploader
  has_one_attached :as_profile_picture # temporary alias used which will be removed later
end

This should allow us to enable attaching files to an instance of profile via as_profile_picture using active_storage. The name as_profile_picture is a temporary alias for now to avoid naming conflicts and will be later changed as a part of the clean up.

Note

Something I'd do at this point is also to change any strong params or any code where a profile_picture is assigned to an instance of Profile. That should already allow us to take care of one part of the problem which is to upload new attachments to active_storage.

Also create some sort of helper methods for example one good case to have a helper method would be to return the URL of the attachment for preview purposes based on the attachment type e.g. is_a?(ActiveStorage::Attachment) or is_a?(CarrierWave::Uploader).

2. Handling existing attachments

For existing attachments we'll attach them one at a time asynchronously to active_storage. Having carrier_wave uploader already mounted on the model we have access to the getter profile_picture that we have already been using to access the respective attachment. We can override it slightly to also achieve the second objective which was to migrate existing attachment to active_storage. Theoretically, that would mean doing something like this:

class Profile < ApplicationRecord
  mount_uploader :profile_picture, ProfilePictureUploader
  has_one_attached :as_profile_picture

  def profile_picture
    if as_profile_picture.attached?
      as_profile_picture
    else
      enqueue_attachment_migration("profile_picture")
      super
    end
  end

  private

  def enqueue_attachment_migration(attachment_name)
    MigrateAttachmentToActiveStorageJob.perform_async(id, class.name, attachment_name)
  end
end

As described in the diagram above with the latest changes we are able to now (for existing attachments):

1. Check if they exist on active_storage
2. If yes, just return the attachment from active_storage
3. If no, return the attachment via carrier_wave and enqueue a background job to attach the profile_picture to the instance via active_storage as well.

Background Job

The background job essentially fetches the file currently attached via carrier_wave and attaches it to the as_profile_picture association.

With these changes in place, this would take care of the second part of the use case we are trying to solve i.e. migrate existing attachments on the go asynchronously.

Post Migration Clean Up 🧹

Once we are sure that all the attachments that were meant to be migrated have been successfully migrated we can perform some clean up and change the temporary alias we had declared in the model to track active_storage attachments.

Database Clean Up

Just a heads up before moving forward, I wasn't able to execute this part but will share some ideas about the clean up that can be done after all the attachments have been migrated.

The names given to the attachments in the model, in our case as_profile_picture are tracked in the name column of the active_storage_attachment table. We will need to write a script to update the column values from as_profile_picture to profile_picture.

Model Clean Up

Once we have updated the names in the active_storage_attachments we can clean up some of the code we added earlier in the model to be something like:

class Profile < ApplicationRecord
  has_one_attached :as_profile_picture
end

And if everything works we can even go and remove the Uploader classes and carrier_wave related attributes from the database schema.

Note

At this point, I would also clean up/refactor the helper methods added accordingly e.g. removing any conditional behavior based on whether attachment was previously stored via carrier_wave or active_storage.

Conclusion

To conclude I wanted to sum up what I thought was a newer perspective when it came to data migration, the idea behind writing this post was to present with an alternate idea for anyone looking for options.

The code samples are just to give an idea, I initially felt working with a live project for this post but due to time constraint I wasn't able to. Also, I think active_storage in terms of setup and configuration is well documented on a number of forums.

Hope this was of some help for you, if you have questions or feedback feel free to share those in the comments.