Build multi-cloud & hybrid networks

Goals

The purpose of this guide is to create four distributed networks and combine them into one highly available logically connected network.

• Build with the top three cloud vendors (aws.amazon.com, azure.microsoft.com, cloud.google.com) and one On-Premise (pfsense.org) network
• Scale Mesh network topology to allow additional point-to-point connections
• Dynamic routing between Autonomous Systems (AS) using Border Gateway Protocol (BGP)
• Encrypt network traversal over Virtual Private Network (VPN) tunnels using Internet Protocol Security (IPSec)

Network

• Final mesh network topology architecture

AWS

Network

• Create a Virtual Private Cloud Network in AWS

Gateway

• Identify the IP address of the ISP
• Point to Point Identification and traffic passthrough

Customer Gateway

Transit Gateway

Route Table

• Update routes to TGW

DNS

• AWS will dedicate a reserved IP address x.x.x.2 for a VPC resolver
• Outbound Endpoints will allow you to forward DNS requests for resolvers on other networks
• Inbound Endpoints will allow resolvers on other networks to forward requests to AWS

Outbound Endpoint

Inbound Endpoint

Site to Site

• Use IPsec tunnels to connect AWS to another datacenter
• Have a failover connection for High availability

Status

• When BGP session is established, the status will go from down to up

SSM

• Using AWS System Manager will allow remote access without opening any ssh ports
• Use to keep your network private
• Use to debug any connectivity issues

IAM Role Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ssm:DescribeAssociation",
                "ssm:GetDeployablePatchSnapshotForInstance",
                "ssm:GetDocument",
                "ssm:DescribeDocument",
                "ssm:GetManifest",
                "ssm:GetParameter",
                "ssm:GetParameters",
                "ssm:ListAssociations",
                "ssm:ListInstanceAssociations",
                "ssm:PutInventory",
                "ssm:PutComplianceItems",
                "ssm:PutConfigurePackageResult",
                "ssm:UpdateAssociationStatus",
                "ssm:UpdateInstanceAssociationStatus",
                "ssm:UpdateInstanceInformation"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "ssmmessages:CreateControlChannel",
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenControlChannel",
                "ssmmessages:OpenDataChannel"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "ec2messages:AcknowledgeMessage",
                "ec2messages:DeleteMessage",
                "ec2messages:FailMessage",
                "ec2messages:GetEndpoint",
                "ec2messages:GetMessages",
                "ec2messages:SendReply"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

Endpoint

SSM Endpoint

SSMMessage Endpoint

EC2Message Endpoint

Azure

VNET

• Create a Virtual Network on Azure

Resource Group

• Resource Group provides a single detailed view of all resources in a groups stack

Virtual Network

• This Iaas will build a virtual network similar to a VPC
• Create 1 network /24 CIDR to create 4 subnets with /26 CIDR

VWAN

• TODO: Azure VWAN
• Have a AWS site-to-site connection config to populate data

Local Network Gateway

• IP of the customer/data center Gateway

Reserved APIPA

Virtual Network Gateway

Private Resolver

• TODO: Azure DNS

Connection

• Use to create an IPsec connection using BGP
• Create a second connection for failover

BGP

• Verify Connection is enabled
• Create second connection for failover

• Verify Route propagation from BGP

Azure VM

• Azure -> AWS

GCP

VPC

• Create a Virtual Private Cloud on Google

Network Connectivity Center

• TODO: GCP Network Connectivity
• Have a AWS site-to-site connection config to populate data

Cloud Router

VPN Gateway

Cloud DNS

• TODO: GCP DNS

Peer VPN

• Set up the infrastructure for GCP VPN

• Repeats these steps on interface 1 (failover)

BGP

• Verify Dynamic Route update

GCP vm

• GCP -> AWS

PFSense

VLAN

Check out this write-up on how to configure VLANs with pfsense

ISP

TODO: Check out this write-up on how to configure a VPN Server with pfsense

PiHole

TODO: Check out this write-up on how to configure a DNS server with PiHole

IPSec

Phase 1

• Start by creating a primary tunnel and repeat the below steps for the failover connection tunnel 2

Phase 2

• Start by creating a primary tunnel and repeat the below steps for the failover connection tunnel 2

Status

• Both primary and failover tunnels connected with IPSec

BGP

FRR Global Settings

FRR Route Maps

FRR BGB

FRR Neighbors

• Start with the primary tunnel and repeat the steps for the failover tunnel

FRR Status

• Verify Dynamic Routes have been updated

• View the BGP Summary

Mac

• Mac -> AWS