How to Provision and Manage Amazon EKS with Ease

AWS is the unquestioned leader of the $180-billion cloud market today, with a 33% overall market share according to Synergy Research Group. Their dominance also extends to the managed Kubernetes space. Both the Cloud Native Computing Foundation 2019 survey and a more recent Logicata Kubernetes poll results show EKS with the lead in terms of popularity amongst its competition (e.g. GKE, AKS, etc).

Source: CNCF Survey 2019

However, as more applications are onboarded onto EKS, managing multiple clusters and workloads remain a challenge. In this post, we'll discuss a few ways to provision an EKS cluster and using KubeSphere as the platform layer to securely deploy and maintain containerized applications on Kubernetes.

eksctl

eksctl is an open-source tool jointly developed by the AWS and Weaveworks to create and manage EKS clusters. Behind the scenes, eksctl creates a CloudFormation stack to provision and update AWS artifacts.

After installing eksctl, a cluster can be bootstrapped imperatively with command line flags or via a config file declaratively:

via command-line:

eksctl create cluster --name=cluster-1 --nodes=4

or via config file:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: basic-cluster
  region: us-east-2

nodeGroups:
  - name: ng-1
    instanceType: m5.large
    desiredCapacity: 10
    volumeSize: 80
    ssh:
      allow: true # will use ~/.ssh/id_rsa.pub as the default ssh key
  - name: ng-2
    instanceType: m5.xlarge
    desiredCapacity: 2
    volumeSize: 100
    ssh:
      publicKeyPath: ~/.ssh/ec2_id_rsa.pub

Similar to kubectl commands, config file changes can be applied via the -f flag: eksctl create cluster -f <file-name.yaml>.

eksctl maintains a comprehensive documentation website with various configuration parameters like enabling CloudWatch, Fargate nodegroups, EKS addons (e.g. CNI, EBS driver, core-dns), as well as nice integrations for IRSA support. EKS workshop also hosts a step-by-step guide with a video to bootstrap an EKS cluster.

If you are familiar with CloudFormation or would like to use a nice wrapper that is jointly maintained by the AWS team, eksctl is a great choice to use to maintain EKS clusters.

Terraform EKS Module

Another popular option is to use Terraform to provision EKS cluster either with the official EKS module or use submodules for VPC, EKS, and/or nodegroups separately. The complete example for the EKS module will bootstrap an EKS cluster with self-managed and AWS managed nodes with KMS encryption enabled by default.

Since Terraform is one of the more popular IaC providers, the advantage of using Terraform to manage EKS is the ability to keep everything with the same tech stack. If you are also using multi-cloud or have plans to branch out into AKS or GKE in the future, using Terraform would be more desirable than eksctl as well.

Finally, for a deep-dive into designing and provisioning a production ready EKS cluster, you can check out some tips in this article.

Installing KubeSphere on Amazon EKS

Now that we have a functional EKS cluster, we can install KubeSphere using kubectl.

First, we need to update the kubeconfig to match our newly created cluster:

aws eks --region <my-region> update-kubeconfig --name <my-cluster-name>

Then we can apply the kubectl manifests to install:

kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/kubesphere-installer.yaml

kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/cluster-configuration.yaml

Once the installation is complete, you should see:

#####################################################
###              Welcome to KubeSphere!           ###
#####################################################
Account: admin
Password: P@88w0rd
NOTES：
1. After logging into the console, please check the
   monitoring status of service components in
   the "Cluster Management". If any service is not
   ready, please wait patiently until all components
   are ready.
2. Please modify the default password after login.
#####################################################
https://kubesphere.io             2020-xx-xx xx:xx:xx

Alternatively, KubeSphere has partnered with AWS to easily install KubeSphere as an AWS Quick Start. This will use a CloudFormation template to deploy an EKS cluster and install KubeSphere automatically. You can edit the CloudFormation template to remove VPC and EKS creation and only trigger KubeSphere installation with an existing cluster.

Managing Amazon EKS with KubeSphere

Although Amazon provides some add-on services such as VPC CNI, CoreDNS, EBS CSI, and kube-proxy to the core EKS offering, it is pretty barebones in terms of the extra tooling needed for a production-ready Kubernetes platform. It does not come with any ingress controllers (e.g. nginx, traefik, etc), autoscalers (e.g. karpenter, cluster autoscaler), logging and monitoring agents, or common tools like external dns or cert-manager. If you also plan to support multitenancy on EKS, then you are also responsible for configuring namespaces and necessary RBAC components yourself.

This is where KubeSphere can help ease the burden by providing a platform layer on top. KubeSphere comes prepackaged with integrations with Jenkins, logging/monitoring, service mesh, ingress controllers, and more deliver a complete application management experience. User management is also built in via workspaces and projects, which will assign users and RBAC roles to namespaces corresponding to each project. The main dashboard can also be used to deploy new microservices with Jenkins pipelines or utilize the App Store to deploy popular Helm charts like etcd, redis, tomcat, postgresql, etc.

KubeSphere also shines when there are multiple clusters (e.g. multi-region or multi-environment) involved. KubeSphere follows the federation model where the KubeSphere running in the host cluster can control downstream member clusters. Using KubeSphere, platform teams can consistently install, upgrade, and manage not only the infrastructure components but also their application across multiple clusters.

Image Credit: New Stack

Conclusion

The learning curve for mastering Kubernetes is high. But with Amazon EKS, users can offload the management of the master plane and core addon components to Amazon. With eksctl and Terraform, teams can easily provision many Kubernetes clusters at scale. To go a step further, utilize the rich ecosystem of integrations that KubeSphere provides to further ease the burden of having to manage Kubernetes clusters and the applications in a cloud-native way.