How to manage Route53 hosted zones in a multi-account environment

An interesting question came up in a conversation today:

“How should I manage the Route53 DNS records in a multi-account environment?”

Suppose you have configured an AWS Organization with different accounts for dev, staging and production environments. And you have registered the root domain for your application in the master AWS account.

When working with CloudFront or API Gateway, you often need to issue ACM (Amazon Certificate Manager) certificates in order to use custom domain names.

To verify the ACM certificate request, you can add a CNAME record to the Route53 hosted zone. *You can use email verification too, but it doesn’t play as nicely with infrastructure-as-code (IaC) as it introduces an external step that requires human (from the right person!) intervention.

Most IaC tool (CloudFormation/CDK/Terraform/etc.) cannot span across multiple accounts. So in a multi-account environment where the ACM certificate request originates from a different account to where the domain is hosted is problematic.

Instead, you want to arrange your domain names so that each account owns its subdomain and can verify any ACM requests it creates.

Here’s how.

Step 1. host the root domain in the master account.

Step 2. host a subdomain in each environment-specific accounts for dev, test, staging, prod, etc.

Step 3. for each of the subdomains in the corresponding AWS account, note the NS record that Route53 has created automatically.

Step 4. back in the Master account, create a NS record for each of the subdomains and use the NS record values from Step 3.

Essentially, this delegates the ownership of the subdomains to the corresponding AWS account’s Route53 hosted zone. So the dev account now owns the dev.example.com subdomain, and the prod account owns the prod.example.com subdomain and so on.

From here on, any CloudFront distributions or APIs should use subdomains to these account-level subdomains - e.g. images.dev.example.com or user-api.dev.example.com. And since the dev account owns the dev.example.com subdomain, you can verify the ACM certificates for images.dev.example.com by creating a CNAME record inside the dev.example.com hosted zone. No more cross-account dependencies!

But, that still leaves you with the question of “How do I set up these hosted zones in the first place?”.

Infra-as-Code for the Route53 hosted zones

You can of course set them up by hand, which might be the quickest way to get it done and it’s probably gonna be fine if you only have a small number of environments. But it’s not gonna scale if you have a lot of environments, or maybe you need to have team-specific subdomains on top of environment-specific subdomains - e.g. dev.teamA.example.com.

One option is to write custom scripts to stitch multiple CloudFormation templates together and execute them in sequence. For example:

1. Deploy a template in the dev account to provision the dev.example.com hosted zone in Route53.
2. Deploy a template in the master account to add the NS record to the example.com hosted zone in Route53.
3. Rinse and repeat for every other sub-account.

A much better way to do this in a true infrastructure-as-code way is via org-formation. org-formation is an awesome open-source tool that gives you a CloudFormation-like syntax to manage your entire AWS Organization. It lets you create new AWS accounts, assign them to Org Units (OUs), and configure Service Control Policies (SCP) and so on.

Additionally, it lets you create landing zones for these new accounts similar to AWS Control Tower’s account factory. But whereas Control Tower doesn’t let you update the landing zones for existing accounts, org-formation lets you manage and update your landing zone configurations as your environment matures.

org-formation makes cross-account references work like normal references inside a CloudFormation template. So it’s actually trivial to setup the subdomain in the dev account and then reference its NS values in the master account. There’s even an example template that you can use out-of-the-box!

org-formation has been one of the mainstays in my toolbelt and have allowed me to configure complex AWS environments from scratch in a matter of hours. If you’re responsible for managing the AWS environment in your organization then you should definitely check it out!