Faster, Cheaper - AWS Graviton 2

Graviton: Hypothetical quantum of gravity, a Marvel character, and now an ARM based CPU from AWS

There have been multiple attempts to bring ARM based CPUs into the data center in recent years due to their perceived efficiency compared to x86_64 designs. Generally they have been seen as lower performance designs suitable for running webservers and other light workloads.

In 2018, AWS released Graviton 1 CPUs that powered the A1 instance family. These were cheaper than Intel/AMD instance types, but also lower performing. I think I launched one to take a look, but never really considered using them again.

In 2019, AWS released Graviton 2 CPUs, and these are a very different beast. AWS claims that they are up to 40% faster than the equivalent Intel instance types, and prices them 20% less. Instead of being limited to one instance family, they are available in a whole range (T4g, C6g, M6g, R6g, X2gd) and they're no longer limited to EC2 instances.

In many ways you can think of what AWS is doing with Graviton to be similar to what Apple is doing with their M1 line of chips. Both are using custom ARM designs developed in house (by companies they bought) and are redefining what we expect from ARM based CPUs.

Supported Services

Some of the services (the list is growing over time, and with re:Invent on the horizon this could well grow in the next couple of weeks) that support Graviton 2:

• EC2 - Run servers with a range of instance types
• Lambda - Run serverless applications in a range of languages
• ECS/EKS - ARM based EC2 AMIs are available to work with both container platforms in AWS, and it's hopefully just a matter of time before Fargate support appears

Update: 11/23 - Graviton 2 support for AWS Fargate announced before re:Invent! Currently this does not include Fargate Spot.

• RDS - Supports MySQL, Postgres and Aurora
• ElastiCache - Redis and Memcached
• OpenSearch - AWS's fork of ElasticSearch
• EMR - Elastic Map Reduce
• CodeBuild - Build code to run on ARM on Graviton instances

With a lot of these services you will need to be on a recent version to be able to use Graviton instances. For example you need to use a recent version of MySQL 8 to select Graviton instance types in RDS, and the very latest version to use the burstable T4g instance family.

Where to start using Graviton 2?

If you are starting a new project in AWS, and you're using some of the services listed above, I would recommend starting with Graviton 2 instances where possible. That will give you cost savings and performance benefits from day 1. In some cases AWS is already defaulting the instance type to a Graviton 2 processor in the AWS console.

This does make some assumptions:

• You're using services/applications that can run on Linux
• The runtimes and tools you need are available for ARM (true of pretty much all major languages and Open Source tools)
• You're willing to accept there may be some extra work in compiling/packaging applications to run on a different architecture than you're developing on (unless you're on a new Mac)

APIs are (generally) easy

For services where you interact with an API and don't run your own code (RDS, ElastiCache, OpenSearch), consumers of the API don't care whether they're calling a service running on x86_64 or ARM, so these are often the easiest to switch.

If you're already running some of these services, you may find that you need to upgrade the version and this can be disruptive depending upon how far from the latest version you are and the type of service. But perhaps this is the time to do the upgrade you've been putting off for the last 6 months.

Open Source applications

Linux has supported ARM for a long time (doesn't everyone have a stack of Raspberry Pis sitting around?), and if you can install a package on an x86_64 machine you can almost certainly install it on Graviton 2 instances.

Nginx example

This snippet of CloudFormation launches an EC2 instance running Amazon Linux 2, installs Nginx then starts the service:

  NginxInstance:
    Type: AWS::EC2::Instance
    Properties:
      IamInstanceProfile: !Ref InstanceProfile
      ImageId: !Ref AmiParameter
      InstanceType: !Ref InstanceType
      SecurityGroups:
        - !Ref SecurityGroup
      Tags:
        - Key: Name
          Value: !Ref 'AWS::StackName'
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -xe
          amazon-linux-extras install -y nginx1
          systemctl start nginx

This works identically whether the AMI/Instance Type are for x86_64 or for Graviton.

Custom Applications

For your own applications, whether running on EC2 or Lambda, how much work it is to run on Graviton 2 will depend upon the language:

Languages: No changes necessary

• Java
• .NET Core, .NET 5 or 6
• NodeJS

All of these run on top of a runtime and your code is either not compiled (NodeJS) or compiled into an intermediary form that works across platforms.

Java in particular has always had the concept of build once, run anywhere, and it's over 20 years ago that I first compiled Java code on an Intel/Windows machine and ran it on a Sun/Solaris server.

Languages: Repackage (sometimes)

• Python
• Ruby

These languages can run on multiple platforms, often without changing, but may make use of some packages that use native code. This means that you may need to re-package your application for ARM, and some packages may need to be compiled if they're not available pre-compiled in the package manager.

I'm not that familiar with PHP, but I think it may fall into this list as well. This blog post talks about some of the work AWS has done with the PHP community to improve the performance on ARM64 CPUs like Graviton 2.

Languages: Recompile

• Golang
• Rust
• C, C++

These languages will need to be recompiled to run on a Graviton instance, using an ARM64 server to compile (e.g. in CodeBuild), or cross-compiling from another machine.

Containers

Although containers are considered portable (run the same container many different places), they are architecture specific and you cannot run the same container on x86_64 and ARM architectures. Even if your application code is in a language like Java, it's going to reply on a JVM being installed in the container that uses native code.

Containers can either be built on the target architecture, or there are options for building multiple architectures using emulation.

AWS ECR has added support for multi-architecture container images. This allows you to store different architectures in the same repository and when you pull a multi-architecture tag it picks to correct container to download based upon the client architecture.

At the moment the UI for ECR doesn't do a good job of displaying architectures, and I'm not sure how lifecycle policies are affected.

Lambda functions

Depending upon the language you're using, it may be as easy as changing the Architecture value to switch from x86_64 to ARM64 (i.e. Graviton 2). As discussed above, if you're using native packages or code you will need to repackage or recompile to switch.

One particularly nice feature is the ability to gradually shift traffic from one architecture to another by using weights to distribute traffic between two versions of the function. This would allow you to send a small percentage of your traffic to the Graviton 2 version of the function until you are satisfied that there are no errors and the performance is satisfactory (hopefully better).

Graviton Challenge

AWS is running the Graviton Challenge which walks you through migrating an application to Graviton 2 over 4 days. Some parts of the challenge have finished, but you can still run a single T4g.micro instance for free until 12/31.

Looking forward to re:invent

The re:invent catalog contains a number of sessions related to Graviton and custom silicon:

• CMP301: The journey of silicon innovation at AWS
• CMP213: Lessons learned from customers who have adopted AWS Graviton
• AMZ301: How Amazon migrated a large ecommerce platform to AWS Graviton
• CMP214: Run containerized workloads on AWS Graviton2 for better price-performance
• DEM002-S: Workload analysis on Arm-based AWS Graviton2 processors

I'd also be very surprised if there aren't some more announcements in this area.