Gert Leenders for AWS Community Builders

Posted on Apr 7, 2021 • Originally published at element7.io

Keep up with the times: use AWS PrivateLink

#devops #aws #cloud #architecture

Table of Content

This post starts with a brief explanation of AWS PrivateLink, distilled mostly from the AWS PrivateLink Whitepaper but limited to the very essence.

The second part is a hands-on example describing how to set up a VPC Endpoint Service and a VPC Endpoint connection. It includes a CloudFormation template with all necessary resources.

What is AWS PrivateLink?

In a nutshell, AWS PrivateLink provides secure, private connectivity between Amazon VPCs, AWS services, and on-premises applications on the AWS network. As a result, you can simply and securely access services on AWS using Amazon’s private network.

AWS PrivateLink allows you also to create an application in your Amazon VPC and offers that application as a VPC endpoint service. The VPC endpoint service allows others to access your service using AWS PrivateLink.

AWS Private Link vs VPC Peering

AWS PrivateLink is not a replacement for VPC Peering. In some use cases, VPC Peering is still the better option. On the other hand, cases where access was previously established using VPC Peering, are now better off using AWS PrivateLink.

The benefits of AWS PrivateLink

Compared with VPC Peering AWS PrivateLink has quite some benefits:

PrivateLink uses private IP addresses for traffic: PrivateLink traffic does not traverse the internet. AWS PrivateLink uses private IP addresses and security groups within an Amazon VPC so that services function as though they were hosted directly within an Amazon VPC.
Security: compared to VPC peering, which allows access to all resources, AWS PrivateLink only allows access to a specific service or application.
PrivateLink Supports CIDR overlap
Unidirectional access: Only VPC Endpoints can initiate a connection.
Simplify Network Management: AWS PrivateLink is easy to use and manage because it removes the need to whitelist public IPs and manage internet connectivity with internet gateways, NAT gateways, or firewall proxies. AWS PrivateLink allows for connectivity to services across different accounts and Amazon VPCs with no need for route table modifications. There is no longer a need to configure an internet gateway, VPC peering connection, or Transit VPC to enable connectivity.
Facilitate Your Cloud Migration: AWS PrivateLink gives on-premises networks private access to AWS services via AWS Direct Connect. Customers can more easily migrate traditional on-premises applications to services hosted in the cloud and use cloud services with the confidence that traffic remains private.

AWS PrivateLink use-cases

Private Access to SaaS Applications

AWS PrivateLink enables Software-as-a-Service (SaaS) providers to build highly scalable and secure services on AWS. Using a VPC Endpoint Service, Service providers can privately expose their service to thousands of customers on AWS with ease.

Previously, exposing a service using VPC Peering felt like using a sledgehammer to crack a nut. To me, it felt that way because VPC peering potentially exposes all your resources instead of limiting it to a single service. So, if you have a service you want to expose over the private AWS Network, using a VPC Endpoint Service is a no-brainer.

Shared Services

Shared services like security services, logging, monitoring, DevOps tools, and authentication can be exposed as VPC Endpoint Services. Again this gives you quite some benefits compared to VPC Peering. On top of that, it could help you to overcome the VPC peering limit of 125 peering connections to a single Amazon VPC.

Hybrid Services

The Network Load Balancer sitting behind a VPC Endpoint Service enables the extension of a service architecture to load balance workloads across resources in AWS and on-premises resources and makes it easy to migrate-to-cloud, burst-to-cloud, or failover-to-cloud. As you complete the
migration to the cloud, on-premises targets could be replaced by target instances in AWS.

Microservices

AWS PrivateLink VPC Endpoint Services are well suited for a microservices environment. A service consumer would then request access to the endpoint service and create an
interface VPC endpoint associated with an endpoint service in their Amazon VPC. There isn't much more to tell, it's just a really good fit for Microservices 😄

Deep Dive: Privately exposing your Software-as-a-Service

Recently the requirement to privately expose one of our services came up. Instead of choosing VPC peering, we choose a VPC Endpoint Service to expose our service this time. Let me quickly guide you through the necessary steps to expose your service as VPC Endpoint Service.

Add a Network Load balancer: The creation of a VPC Endpoint Service requires a Network Load Balancer (NLB). To fulfil this I deployed an extra NLB besides my existing Application Load Balancer (ALB). Although an extra load balancer has a slight cost overhead it also separates traffic and responsibilities. In case of troubles, this could help to pinpoint the cause, furthermore, it also limits blast radius in case a load balancer would fail.
Adjust the security group of your targets: in contrast to an ALB, an NLB does not have a security group. If you know the IP ranges of your consumers you can use those. You could also allow all private network ranges like I did in the CloudFormation example below.
Create a VPC Endpoint Service and attach it to the NLB.

This is how this looks translated into CloudFormation:

AWSTemplateFormatVersion: '2010-09-09'
Description: VPC Endpoint service stack.

Parameters:
  Application:
    Type: String
  VpcId:
    Type: AWS::EC2::VPC::Id
  PrivateSubnetIds:
    Type: List<AWS::EC2::Subnet::Id>
  HostedZoneId:
    Description: Hosted zone id
    Type: AWS::Route53::HostedZone::Id
  HostedZoneDomainName:
    Description: Hosted zone domain name
    Type: String
  LogBucketName:
    Type: String

Resources:
  EndpointServiceNetworkLoadBalancerCert:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Sub ${Application}.${HostedZoneDomainName}
      ValidationMethod: DNS
      DomainValidationOptions:
        - DomainName: !Ref HostedZoneDomainName
          HostedZoneId: !Ref HostedZoneId

  EndpointServiceNetworkLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: !Sub endpointsrv-${Application}-nlb-cfn
      Type: network
      Scheme: internal
      Subnets:
        - !Select [ 0, !Ref PrivateSubnetIds ]
        - !Select [ 1, !Ref PrivateSubnetIds ]
        - !Select [ 2, !Ref PrivateSubnetIds ]
      LoadBalancerAttributes:
        - Key: deletion_protection.enabled
          Value: "true"
        - Key: load_balancing.cross_zone.enabled
          Value: "true"
        - Key: access_logs.s3.enabled
          Value: "true"
        - Key: access_logs.s3.bucket
          Value: !Ref LogBucketName
        - Key: access_logs.s3.prefix
          Value: endpointservice-NLB
      Tags:
        - Key: Name
          Value: !Sub endpointsrv-${Application}-nlb-cfn

  EndpointServiceNetworkLoadBalancerListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
          - Type: forward
            TargetGroupArn: !Ref EndpointServiceTargetGroup
      LoadBalancerArn: !Ref EndpointServiceNetworkLoadBalancer
      Port: 443
      Protocol: TLS
      Certificates:
        - CertificateArn: !Ref EndpointServiceNetworkLoadBalancerCert
      SslPolicy: ELBSecurityPolicy-FS-1-2-Res-2019-08

  EndpointServiceTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Sub endpointsrv-${Application}-tg-cf
      Port: 80
      Protocol: TCP
      VpcId: !Ref VpcId
      HealthCheckProtocol : HTTP
      HealthCheckIntervalSeconds: 30
      HealthyThresholdCount: 3
      UnhealthyThresholdCount: 3
      TargetGroupAttributes:
        - Key: deregistration_delay.timeout_seconds
          Value: "90"
      Tags:
        - Key: Name
          Value: !Sub endpointtsrv-${Application}-tg-cf

  EndpointService:
    Type: AWS::EC2::VPCEndpointService
    Properties:
      AcceptanceRequired: True
      NetworkLoadBalancerArns:
        - !Ref EndpointServiceNetworkLoadBalancer

  EndpointServicePermissions:
    Type: AWS::EC2::VPCEndpointServicePermissions
    Properties:
      AllowedPrincipals:
        - arn:aws:iam::123456789012:root #changeme
      ServiceId: !Ref EndpointService

# Other necessary resources ##############################

  EC2AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      ...
      MaxInstanceLifetime: 1209600 # 14 days
      LaunchTemplate:
        LaunchTemplateId: !Ref Ec2ServiceLaunchTemplate
        Version: !GetAtt Ec2ServiceLaunchTemplate.LatestVersionNumber
      TargetGroupARNs:
        - !Ref EndpointServiceTargetGroup
      ...

  Ec2ServiceLaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
      ...
      LaunchTemplateData:
        MetadataOptions:
          HttpTokens: required
        SecurityGroupIds:
            - !Ref Ec2ServiceSecurityGroup

 Ec2ServiceSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${Application}-sg-cfn
      GroupDescription: Enable HTTP and Endpoint Access
      SecurityGroupIngress:
        ...
        - CidrIp: 10.0.0.0/8
          FromPort: 80
          IpProtocol: tcp
          ToPort: 80
          Description: Allow traffic from the NLB (which has no security group and is a pass-through)
        - CidrIp: 172.16.0.0/12
          FromPort: 80
          IpProtocol: tcp
          ToPort: 80
          Description: Allow traffic from the NLB (which has no security group and is a pass-through)
        - CidrIp: 192.168.0.0/16
          FromPort: 80
          IpProtocol: tcp
          ToPort: 80
          Description: Allow traffic from the NLB (which has no security group and is a pass-through)
      VpcId: !Ref VpcId
      Tags:
        - Key: Name
          Value: !Sub ${Application}-sg-cfn

A few Caveats

Tags or not supported for the CloudFormation VPCEndpointService Resource. 😟
To allow a VPC Endpoint connection from another account change AllowedPrincipals under the EndpointServicePermissions CloudFormation Resource.
CloudFormation doesn't support private DNS names for an Endpoint Service. You need to add the private DNS manually in the Web Console (under VPC Endpoint Service) after the endpoint is created. This name must match with the Certificate Domain name attached to the NLB Listener. The DNS name also needs to be verified using TXT records.
It's NOT possible to test a VPC Endpoint Service from within the same account. You can only test a VPC Endpoint service by creating a VPC Endpoint connection in another account.

Create a VPC Endpoint Connection to a VPC Endpoint Service (Web Console)

In another account create a VPC Endpoint using 'Find Service by name'. Specify the Endpoint Service Name and the VPC to deploy the Endpoints.
Accept the connection in the account hosting the VPC Endpoint service.
Back in the Customer's account, after the connection is accepted, on the VPC Endpoint choose 'Modify Private DNS names' and tick 'Enable Private DNS Name'. Ensure 'Enable DNS hostnames' and 'Enable DNS Support' are set to 'true' for the VPC.
In the VPC Endpoint Service account, for a second time accept the connection under the VPC Endpoint service. The connection is now ready and the Endpoint is reachable using the Private DNS name.

A VPC Endpoints connection can also easily be created using CloudFormation. See the example below.

Bonus: Other useful VPC Endpoint connection

Besides VPC Endpoint connections to custom SAAS Endpoint Services, the following VPC Endpoints are very handy or even highly recommended to have. These endpoints will prevent traffic to other AWS Services to leave the AWS private network and to go over a public network. Besides being more secure this is also more efficient.

VPC Interface Endpoint for EC2. Security Hub finding EC2.10 even states you should have this configured
VPC Interface Endpoints for ECR
VPC Gateway Endpoint for S3

A CloudFormation example of an VPC Interface Endpoint for EC2:

  EC2VpcEndpointSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub vpc-endpoint-ec2-${EnvironmentName}-sg-cfn
      GroupDescription: Allow EC2 VPC Endpoint outbound traffic
      VpcId: !Ref VPC
      SecurityGroupIngress:
        - CidrIp: !Ref VPCCidr
          FromPort: 443
          IpProtocol: tcp
          ToPort: 443
          Description: Allow traffic from within the VPC to the EC2 Endpoint

  EC2VpcEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcEndpointType: Interface
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ec2
      VpcId: !Ref VPC
      SubnetIds:
        - !Ref PrivateSubnetAZA
        - !Ref PrivateSubnetAZB
        - !Ref PrivateSubnetAZC
      PrivateDnsEnabled: True
      SecurityGroupIds:
         - !Ref EC2VpcEndpointSG
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal: '*'
            Action: '*'
            Resource: '*'

Enjoy and until next time!

References:

DEV Community