Zero Downtime Database Migrations at Woovi

At Woovi we are moving very fast, and our data modeling needs to evolve as the requirements changes.
For this to happen we need to run data migrations in production with zero downtime to avoid affecting our users.

What is a Migration?

A migration is when you move from one type of code/data to another type of code/data.
This happens because your first assumptions when coding or modeling your database are not valid anymore because of changing business and product needs.
If you don't run a database migration your new code could not work, and your old code can't be removed in the future.
A database migration is a code that updates some old data pattern to a new pattern.

• It can add a new fields to the database.
• It can remove an old field from the database.
• It can transform some old data in new data.

It is a "map" from the old data to the new data that you need.

Migration Principles

We need to ensure our migration follow some principles to avoid production incidents.

Idempotent

Migration needs to be Idempotent.
Idempotent is a function that will always have the same end result no matter how many times you run the function. Migration can break in the middle of execution, so it would be safe to rerun them.

Continue Execution

Migration should continue from where it stopped, and it should finish. The number of items to migrate should decrease as you process.

We want to add the field sender to Transaction model. To fix the past we need a migration for old Transaction items. We search for all Transaction without the sender field, so we can avoid migrating items that do not need to be migrated

const cursor = Transaction.find({
  sender: {
    $exits: false,
  },
});

Database Cursor

The number of items in our collections are getting bigger every day. So we can't bring all the items in the memory. We need to use a cursor to process them in batch.

Tests

Every migration needs a test to cover edge cases. We can't let our database in an inconsistent state.

Tick Tock and Backward Compatible

Our code needs to work with both old and new data model until the migration is done, this ensure the deployment and migration are decoupled.
You can read more on this topic: https://dev.to/woovi/tick-tock-migration-pattern-2gn and https://dev.to/woovi/practical-backward-compatibility-example-2c9n

Running migration at Production

We have a Docker image only to run migration in staging and in production, this ensure we are running migration in the right environment.

The process to run the migration in production follow this:

• make a new release
• run the migration locally
• validate the results
• run the migration at staging
• validate the results
• do a backup
• run the migration at production
• validate the results

Because of the principles above, we can run migration before or after any deployment.

In Conclusion

Running a migration before doing a new deployment does not scale and it is not safe.
Following the principles of this article you can decouple your deployment from your migration process.
You can run your migration safely without rush, and you don't need to block the deployment because of this.
Rethinking existing process is one thing that Woovi is doing to make sure we can move fast without breaking things.

Woovi
Woovi is a Startup that enables shoppers to pay as they like. To make this possible, Woovi provides instant payment solutions for merchants to accept orders.

If you want to work with us, we are hiring!