You can choose "Multi-AZ" when creating an RDS database. What does it mean in term of availability?
Each database service offers different replication options, which are used to ensure data is stored in multiple locations for redundancy and disaster recovery. With Amazon RDS, techniques like Multi-AZ can be used to quickly restore availability in case of a disaster.
It is crucial to understand the distinction between High Availability and Disaster Recovery. High Availability techniques aim to prevent downtime and provide immediate access to data, while Disaster Recovery techniques focus on restoring availability as rapidly as possible after a failure.
Disaster Recovery
Traditionally, databases are safeguarded against various failures that can cause the database to become unavailable. These failures can include server crashes, storage failures, or network outages. To recover from these failures, data is replicated in multiple ways.
One such way is through the use of Write Ahead Logging (WAL), also known as a redo log or transactional log. Any changes made to the data in memory are written to the WAL file, ensuring that even if the server crashes, the changes can be recovered from the log.
Additionally, the database files on disk, including the data files and WAL files, are copied to backups to protect against disk failures. If the disk fails, the backup can be used to restore the data and WAL files, enabling the database to be brought back online. The backups of datafiles and WAL files are sent to another location so that the database can be restored and recovered even if the primary data center is not accessible.
The technology used for Disaster Recovery is chosen based on the recovery objectives that need to be met. Recovery Time Objective (RTO) measures the amount of time it takes to make the database available again after a failure. To reduce RTO, backups and Write Ahead Logging (WAL) log shipping are used to restore the database to a standby database before the failure occurs. This way, the standby database is ready to be activated as soon as the failure of the primary database is detected.
Recovery Point Objective (RPO) measures the amount of data loss that can occur during recovery. Ideally, RPO is zero, which means there is no data loss. However, achieving a zero RPO requires that the backup location or standby database receive the latest committed transaction before the failure occurs. This can add some latency to the primary database, as it waits for the acknowledgement from the standby database before committing the transaction.
To protect against an AZ failure, a database running in one AZ, like eu-central-1a in Frankfurt, can synchronize its writes to another AZ, like eu-central-1b, with a low Round Trip Time (RTT) to minimize latency. By waiting for the acknowledgment from the second AZ, the database can achieve a zero Recovery Point Objective (RPO) and a standby database can be started with a Recovery Time Objective (RTO) in minutes. This synchronization can be at storage level (block storage replication) or database level (WAL log shipping). This technique only protects against an AZ failure and not a region failure.
To protect against a region failure, the same database can also have all transaction logs sent, asynchronously to a nearby region, such as eu-west-1a in Ireland. However, the increased latency means that waiting for an acknowledgment from this AZ is not feasible for applications with frequent commits. This approach, without synchronous commit, provides a region failure fault tolerance, but may result in some data loss: RPO is in seconds and RTO in minutes or hours because a recovery with data loss involves a manual decision.
To provide additional protection against failures such as the whole Europe being unavailable, backups can also be sent to an S3 bucket in a different region, such as us-east-2 in Ohio. While this approach provides an extra layer of redundancy, it comes with higher RTO and RPO values due to the increased distance and the need to restore files before starting the database. Depending on the specifics of the situation, RTO could be on the order of hours, while RPO could be on the order of minutes to hours.
The Disaster Recovery techniques we've discussed involve some level of downtime, even with the smallest RPO and RTO. This is an outage even for a failover between Availability Zones of a single region. To achieve true High Availability, additional measures must be taken to eliminate single points of failure and ensure continuous availability of services.
High Availability
An application that is highly available is resilient to failure, and this resilience is achieved through redundancy of the infrastructure components it runs on. Specifically, the application instances run in multiple Availability Zones, and the user can connect to any of them via a load balancer. In the event that one instance fails, the application can continue running on the other instances. This concept is different from Disaster Recovery, which focuses on restoring an application to its previous state after a failure has occurred. In the case of High Availability, infrastructure components may fail, such as an instance restarting, but the application remains available because there's no single point of failure. Although there may be some short performance degradations, the application does not encounter any errors. Instead of measuring recovery time objectives (RTO) and recovery point objectives (RPO), High Availability is measured as a percentage of uptime, often referred to as the "nines" of service level agreements (SLA).
It's important to note that High Availability doesn't just refer to unplanned failures but also includes planned maintenance that requires one instance to be stopped. If an application is unavailable during upgrades or maintenance, it cannot be considered High Availability.
Applications that need to store a state can utilize highly available services such as S3 or DynamoDB, which are also resilient to zone failures. This is why they are called Availability Zones in the cloud: they provide High Availability for cloud-native services, those where all components are horizontally scalable. In AWS there are 3 types of services. Global services like IAM, Route53, CloudFront, and regional services, like S3 or DynamoDB, are resilient to one Availability Zone failure and provide High Availability. Services that run on one EC2 instance in one VPC subnet can fail, with downtime, and their High Availability depends on the possibility to run on identical instances in multiple Availability Zones.
SQL databases are more than just instances with state. They process complex transactions with strong consistency guarantees. How does this work across multiple instances?
Relational SQL Databases
The SQL databases available in Amazon RDS are quite different from cloud-native databases. Unlike cloud-native databases, where scaling is achieved through horizontal scaling of components, RDS databases rely on a single server endpoint for DML (Data Manipulation Language), the SQL operations that read and write data. They have no strong consistency and ACID guarantee beyond a single read-write instance.
Even when deployed across multiple Availability Zones, Amazon RDS databases cannot offer High Availability in the same way as cloud-native databases. Unlike cloud-native databases, which can continue to operate even if an instance fails, a failover to another Availability Zone in RDS constitutes an outage. While this outage may be brief with a synchronous standby, it is still a Disaster Recovery scenario. RDS databases may have recovery point objectives (RPO) of zero and recovery time objectives (RTO) of minutes, but they still rely on a single writer node and Disaster Recovery techniques, such as the ones mentioned above, to ensure business continuity.
The limitations of traditional SQL databases in achieving High Availability are not specific to Amazon RDS and apply to all non-cloud-native databases. One exception is Oracle RAC, which can run a database with multiple read-write instances and offers protection against instance failure. However, Oracle RAC is not an option in Amazon RDS. Even in the Oracle Cloud, RAC is used within one Availability Domain. Across Availability domains, Oracle relies on Data Guard, which is a Disaster Recovery technology. Although recovery with Data Guard can be fast, bringing back the database availability in minutes, a failover still constitutes an outage. This is why Oracle has historically addressed High Availability with a different technology, RAC (known as Parallel Server in the past). All other traditional databases offer only Disaster Recovery and not High Availability, due to their monolithic nature.
Another approach that traditional databases use to minimize the impact of outages is to shard the monolithic database into multiple ones. Sharding involves partitioning a database into smaller, self-contained databases, each of which can operate independently. Although sharding is not considered High Availability because it still results in an outage, it can enable some parts of the application to continue running if they are self-contained within one shard isolated from the failing components.
Before looking at Distributed SQL which provides High Availability in addition to Disaster Recovery, I'll detail what means "Multi-AZ" in Amazon RDS because the same term covers different technologies: block device replication, WAL streaming replication, read replicas.
Before delving into distributed SQL databases that provide both High Availability and Disaster Recovery, it is important to understand what is meant by "Multi-AZ" in Amazon RDS. The term "Multi-AZ" can refer to different technologies, such as block device replication, write-ahead log (WAL) streaming replication, and read replicas. Each of these technologies offers good levels of availability even if they don't qualify for High Availability.
Multi-AZ in Amazon RDS
The reason for detailing the different technologies behind "Multi-AZ" in Amazon RDS is to clarify a common confusion between "Multi-AZ" and High Availability. While "Multi-AZ" can provide a level of redundancy and improved durability through various technologies, it is not the same as High Availability. In fact, all Amazon RDS databases are based on the Disaster Recovery techniques described earlier. A failure of the single writer instance in an Amazon RDS database still constitutes an outage, which, even if short, can impact the availability of applications that rely on it.
In short, the term "Multi-AZ" in Amazon RDS simply means that there are instances of the database in multiple availability zones. The specific usage of these instances, to improve availability or scalability, depends on the particular RDS service being used.
RDS Aurora
Aurora databases, whether MySQL or PostgreSQL compatible, and whether "Multi-AZ" is chosen or not, store the database across three availability zones. The single read-write instance sends all changes to the storage servers using write-ahead logging (WAL) shipping, with synchronous commit. This architecture provides the possibility of achieving no-data-loss Disaster Recovery if the writer's availability zone is down, with a recovery point objective (RPO) of zero.
The time to recover (RTO) can be relatively short if there is already a running instance in another availability zone that can read and write to the same storage. This is where Multi-AZ can help improve the RTO: the standby instance can be quickly activated, and the database can be restored with the same performance level as soon as the cache is warmed up.
It is possible to connect to the standby instance and perform reads on it. However, these reads are eventually consistent, since the cache is invalidated asynchronously, even if they can read from a synchronous storage replica. This is where Multi-AZ can also help improve scalability: certain types of read operations, such as reporting, can be offloaded to the standby instance as long as they do not require strong consistency.
RDS MySQL and PostgreSQL
For MySQL and PostgreSQL in Amazon RDS, there are two types of Multi-AZ deployment available:
Multi-AZ DB instance, which is available for all versions, uses synchronous block storage replication. This means that all writes are synchronized to a standby instance in another availability zone, which can be activated in the event of a failure in the primary instance, providing RPO=0 recovery. The standby cannot be used for reads.
Multi-AZ DB cluster, which was added recently, uses the database's write-ahead log (WAL) to stream changes to two standby databases in the other availability zones. The primary database waits for acknowledgement from at least one of the standby databases before committing, allowing for RPO=0 recovery in the event of the primary availability zone going down. However, the WAL changes are not applied immediately to all replicas, which means that the replicas can be used for eventually consistent reads.
The RPO and RTO for Amazon RDS Multi-AZ DB cluster deployments can be unclear due to poor documentation regarding the level of synchronicity between the primary and standby instances. It is not clear if synchronous commits fall back to asynchronous after a certain timeout period, or wait. You can look at the MySQL or PostgreSQL replication parameters. Automatic failover only occurs when there is no data loss.
RDS MariaDB, Oracle
For MariaDB and Oracle databases in Amazon RDS, only the Multi-AZ DB instance option is available: storage replication. This is simply referred to as a "Multi-AZ deployment," as it was the only option before the introduction of the DB cluster option. The Multi-AZ deployment option does not use database-level replication, like Data Guard for Oracle.
SQL Server
SQL Server offers only one type of "Multi-AZ" deployment, which differs from the other options mentioned above. Similar to the Multi-AZ DB instance, it does not provide any read replicas. However, like the Multi-AZ DB cluster, it utilizes built-in database replication, such as SQL Server Database Mirroring (DBM) or Always On Availability Groups (AGs), depending on the version and edition.
High Availability with Distributed SQL
When it comes to traditional monolithic databases, the term "High Availability" is often used incorrectly. With the exception of Oracle RAC with shared storage, which requires a private interconnect network and cannot be easily deployed in common cloud multiple availability zones, the protection of these databases is typically achieved through a Disaster Recovery solution that minimizes outages but does not eliminate them entirely.
The emergence of NewSQL databases has led to improvements in availability, achieved through approaches such as storage distribution, as seen in Aurora, or sharding. While these measures can reduce the scope of an outage, a failure is still considered an outage that impacts a portion of the application for the duration of the recovery process.
Distributed SQL databases provide High Availability by not being limited to a single writer node. Based on the Google Spanner architecture, databases such as Spanner, CockroachDB, TiDB, and YugabyteDB can be deployed with a replication factor of three across three availability zones, making them resilient to a zone failure.
The distributed SQL databases mentioned (Spanner, CockroachDB, TiDB, and YugabyteDB) are not offered as part of the RDS database services on AWS. However, they can be deployed and used on AWS through various means, including using EC2 instances or containers, or by obtaining them through the AWS Marketplace.
If you are looking for a database service similar to RDS, you may be looking for a managed database service with excellent compatibility with other databases, as seen with Aurora, which uses the same API with a different storage layer. Additionally, an open-source database solution may provide the flexibility to migrate to another platform in the future.
Out of the distributed SQL databases mentioned earlier, only YugabyteDB is fully open source and reuses an existing database (PostgreSQL) for its query layer. This allows for greater flexibility and compatibility, as well as the freedom to move to another platform if needed. While other distributed SQL databases offer High Availability and scalability, they have built their SQL features from scratch, limiting the compatibility with existing databases.
YugabyteDB is available on the AWS Marketplace either as a managed service (managed by Yugabyte) or as a custom deployment with Yugabyte support.
By adopting this approach, you can leverage the benefits of AWS infrastructure while ensuring that your SQL database is highly available, thus minimizing application downtime in case of an availability zone failure or maintenance like upgrades or key rotation. Traditional monolithic SQL databases were not built to run on commodity hardware in multiple data centers, whereas distributed SQL databases are cloud-native and provide the same level of High Availability as your application in a cloud environment.
As we have seen above, the same technology cannot be used for High Availability and Disaster Recovery. For the latter, YugabyteDB adds asynchronous replication between two clusters, like between two regions, as I explained in a previous post.
Summary
Deploying a database across multiple instances in multiple availability zones, also known as "Multi-AZ" in RDS, is not enough to achieve High Availability, like it is for an application server. The technologies used for this approach depend on the specific database, and RDS technologies are based on traditional Disaster Recovery mechanisms, such as synchronous standby and eventually consistent read replicas. While they have low RPO and RTO, they still result in an application downtime in case of failure. To achieve true High Availability in a cloud environment, a cloud-native database built for resilience across multiple zones is required.
You don't have to wait for an availability zone failure to recognize that your database is not highly available. Traditional databases experience outages even during planned maintenance. While some operations like scaling up or OS patching may cause brief outages, a major database release upgrade can take longer. Without rolling upgrade capabilities, a database cannot be considered a true high-availability solution. In contrast, distributed SQL databases can take down one node after another without any interruption, providing true High Availability.
Although the examples used were from AWS, the same concept apply to any cloud provider. With the exception of Google Cloud, having Spanner, none of them provide a Distributed SQL database managed by themselves, but all of them can deploy one on compute instances VM, containers or managed kubernetes. The good news is that you have the choice of database technology, all in managed services.
Top comments (0)