How easy to setup Master-master replication in CouchDB

Database replication is the process of copying data from a database in one computer or server to a database in another. It is very common if we want to scale our database to increase the availability to access the data. The result is a distributed database in which users can quickly access data relevant to their tasks.

The most common and classic method of database replication is Single-leader based data replication aka Master-slave Replication - one server that receives writes from clients, and replicas draw data from there. It provides read scalability, reduced latency and better availability as compared to no replication. However, the write of data is not scalable as read. We can only write data to the master node, and read from the slave node. This flaw paves way for multi-leader and leaderless replication.

Multi-leader based data replication aka Master-master replication, same as single-leader replication, the only different is it has more than one node to accept writes.

Today I would like to share about how easy to setup Master-master replication in CouchDB. One of CouchDB's strengths is the ability to synchronize two copies of the same database. We can trigger a replication process by sending a JSON object either to the /_replicate api endpoint or storing it as a document into the _replicator database.

Let say we have a database called cars in Server A and we would like to to replicate the cars database to Server B.

POST /_replicate

{
   "create_target": false,
   "continuous": false,
   "source": {
      "url": "http://SERVER_A_HOSTNAME:5984/cars",
      "headers": {
         "Authorization": "YOUR_SERVER_A_CREDENTIAL"
      }
   },
   "target": {
      "url": "http://SERVER_B_HOSTNAME:5984/cars",
      "headers": {
         "Authorization": "YOUR_SERVER_B_CREDENTIAL"
      }
   }
}

I personally recommend document based-replication, by storing document to trigger the replication process as you can review back the replication history and you can stop a running replication process by deleting the document, or by updating it with its cancel property set to true.

For specifying username and password, if you are using CouchDB v3.2.* , there are 3 ways to specify your username and password:

1. In an "auth": { "basic": "..." } object:

{
    "target": {
        "url": "http://someurl.com/mydb",
        "auth": {
            "basic": {
                "username": "$username",
                "password": "$password"
             }
        }
    },
    ...
}

1. In the userinfo part of the endpoint URL:

{
    "target":  "http://user:pass@localhost:5984/bar"
    ...
}

1. In an "Authorization: Basic $b64encoded_username_and_password" header:

{
    "target": {
        "url": "http://someurl.com/mydb",
            "headers": {
                "Authorization": "Basic dXNlcjpwYXNz"
            }
        },
    ...
}

If you are using the CouchDB earlier version < v3.2, you can only use the second and third method to specify the username and password. Second method is not preferred as it prevents using characters like @ and : in usernames or passwords.

After we triggered a replication process, we can inspect CouchDB replication status through the active tasks API /_active_tasks and /_scheduler/jobs. For document based-replications, /_scheduler/docs can be used to get a complete state summary. This API is preferred as it will show the state of the replication document before it becomes a replication job.

If the database doesn't exist in the target CouchDB server, we can set the create_target property to true, it will helps us to create a new database and start replicate the data from the source database. For Master-master replication setup, we would like to setup a long running replication instead of one time off, just set the continuous property to true.

POST /_replicator

{
   "create_target": false,
   "continuous": true,
   "source": {
      "url": "http://SERVER_A_HOSTNAME:5984/cars",
      "headers": {
         "Authorization": "YOUR_SERVER_A_CREDENTIAL"
      }
   },
   "target": {
      "url": "http://SERVER_B_HOSTNAME:5984/cars",
      "headers": {
         "Authorization": "YOUR_SERVER_B_CREDENTIAL"
      }
   }
}

Once we has triggered a continuous database replication from Server A to Server B, then we can trigger one more continuous database replication from Server B to Server A by switching the source and target url.

POST /_replicator

{
   "create_target": false,
   "continuous": true,
   "source": {
      "url": "http://SERVER_B_HOSTNAME:5984/cars",
      "headers": {
         "Authorization": "YOUR_SERVER_A_CREDENTIAL"
      }
   },
   "target": {
      "url": "http://SERVER_A_HOSTNAME:5984/cars",
      "headers": {
         "Authorization": "YOUR_SERVER_B_CREDENTIAL"
      }
   }
}

🎉That's it, now we have both cars databases in Server A and Server B in sync. We can even control which documents to be replicate by using Selector Objects or Filter Functions.
For more detail about CouchDB Replication, please check out the official documentation. Thank you for reading.