EKS is a managed kubernetes service from Amazon AWS. There is nothing difficult about it(Sorry for the misleading title!). That being said, there are still a lot of moving pieces in the backend, to make this peachy platform work.
So instead of going the universally acclaimed easiest route of deploying though EKSCTL, I will be going into the individual components that we need to make this cluster work, there by understanding more about them instead of just spinning up through a cli command.
Assuming you have a fresh AWS account with just a root user to go in, we will do this setup in 3 phases.
1. Create a three Subnet highly available Network Stack on AWS.
Please read my blog to understand and create it. It will be just a single Cloudformation stack to get this setup. We need this to deploy our worker nodes of EKS cluster into.
2. Create an admin user to deploy and administer the EKS cluster.
It is always a best practice to not use the root account to use the AWS. In this blog I explain how to create a user with another cloudformation stack. The blog explains to create two users, but we will be using the admin user for this tutorial for simplicity. Also, how to configure the user access from your local desktop. We will need this access after we have the cluster to deploy apps into EKS. The famous kubectl will have this as a prerequisite to work. Going forward with the assumption that you have this step completed and you have a working AWS CLI with a admin profile.
3. Deploy the cluster
This section consist of creating the IAM resources for the cluster and then the compute resources. This is the meat and potato of this blog so lets explain them in detail.
EKS IAM Resources
The EKs cluster consists of two defined sections, the EKS service that AWS maintains and the worker node cluster that you are responsible for providing the details and security for.
- Both these sections have two different IAM roles with a defined sets of Managed policies to attach to them. These can be defined as :
###### EKS Service Role ######
EksClusterServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- eks.amazonaws.com
Action:
- 'sts:AssumeRole'
Description: EksClusterServiceRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
- arn:aws:iam::aws:policy/AmazonEKSServicePolicy
MaxSessionDuration: 3600
Path: /
RoleName: !Ref EksClusterServiceRoleName
Tags:
- Key: Environment
Value: !Ref Environment
###### EKS Node Role ######
EksClusterNodeRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
Description: EksClusterNodeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
- arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
- arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
MaxSessionDuration: 3600
Path: /
RoleName: !Ref EksClusterNodeRoleName
Tags:
- Key: Environment
Value: !Ref Environment
- Both these compute sections need their own security groups. To design these groups we have to think beside accessing each other for the functioning of the cluster(port 1025-65535), who accesses these groups. The Service section is being accessed by outside when we run Kubectl from our desktop. We can clear a definite IP, a known CIDR range if we know what that IP might be. In this case for simplicity, I have opened it to the whole world. The worker nodes need to be accesed by other services accessing the apps running inside it. That will again need futher scrutiny based on what accesses your apps. For example, If you plan to deploy web applications and expose them to only a internal load-balancer, you can keep a defined port open to that load balancer and that will make it secure. For the purpose of simplicity, I have kept the https port 443 open to the whole world. These two groups can be defined as follows:
## Service Security Group ##
EksServiceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: EksServiceSecurityGroup
GroupName: !Ref EksServiceSecurityGroupName
Tags:
- Key: Environment
Value: !Ref Environment
VpcId: !Ref Vpc
## Node Security Group ##
EksNodeSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn: EksServiceSecurityGroup
Properties:
GroupDescription: EksNodeSecurityGroup
GroupName: !Ref EksNodeSecurityGroupName
Tags:
- Key: Environment
Value: !Ref Environment
VpcId: !Ref Vpc
### Service Security group Ingress ###
EksServiceSecurityGroupIngress1:
Type: AWS::EC2::SecurityGroupIngress
DependsOn: EksNodeSecurityGroup
Properties:
GroupId: !Ref EksServiceSecurityGroup
IpProtocol: tcp
FromPort: 443
ToPort: 443
SourceSecurityGroupId: !Ref EksNodeSecurityGroup
### Service Security group Egress ###
EksServiceSecurityGroupEgress1:
Type: AWS::EC2::SecurityGroupEgress
DependsOn: EksNodeSecurityGroup
Properties:
DestinationSecurityGroupId: !Ref EksNodeSecurityGroup
FromPort: 1025
ToPort: 65535
IpProtocol: tcp
GroupId: !Ref EksServiceSecurityGroup
EksServiceSecurityGroupEgress2:
Type: AWS::EC2::SecurityGroupEgress
DependsOn: EksNodeSecurityGroup
Properties:
DestinationSecurityGroupId: !Ref EksNodeSecurityGroup
FromPort: 443
ToPort: 443
IpProtocol: tcp
GroupId: !Ref EksServiceSecurityGroup
### Node Security group Ingress ###
# Open every port to itself
EksNodeSecurityGroupIngress1:
Type: AWS::EC2::SecurityGroupIngress
DependsOn: EksNodeSecurityGroup
Properties:
GroupId: !Ref EksNodeSecurityGroup
IpProtocol: -1
FromPort: -1
ToPort: -1
SourceSecurityGroupId: !Ref EksNodeSecurityGroup
# open 1024-65535 to service SG
EksNodeSecurityGroupIngress2:
Type: AWS::EC2::SecurityGroupIngress
DependsOn: EksNodeSecurityGroup
Properties:
IpProtocol: tcp
FromPort: 1025
ToPort: 65535
GroupId: !Ref EksNodeSecurityGroup
SourceSecurityGroupId: !Ref EksServiceSecurityGroup
# open 443 to Service SG
EksNodeSecurityGroupIngress3:
Type: AWS::EC2::SecurityGroupIngress
DependsOn: EksNodeSecurityGroup
Properties:
IpProtocol: tcp
FromPort: 443
ToPort: 443
GroupId: !Ref EksNodeSecurityGroup
SourceSecurityGroupId: !Ref EksServiceSecurityGroup
### Node Security group Egress ###
# All open
EksNodeSecurityGroupEgress1:
Type: AWS::EC2::SecurityGroupEgress
DependsOn: EksNodeSecurityGroup
Properties:
FromPort: -1
ToPort: -1
GroupId: !Ref EksNodeSecurityGroup
IpProtocol: -1
CidrIp: 0.0.0.0/0
- The last resource we need is a SSH key the cluster will use to encrypt data at rest. We will use AWS SSM to create store and manage accessibility to this key. This key will be used by the EKs cluster to encrypt the secrets and configurations in it. The key has access defnied on the admin user and root. We can add more users to this key based on the use. Example, we have a user on AWS Lambda who might query the EKS cluster for some data, that user will need access to this key on certain access (List,Decrypt,etc) to be able to read the data from the EKS cluster.
## SSH Key for cluster secret encryption
EksKMS:
Type: AWS::KMS::Key
Properties:
Description: Key for EKS cluster to use when encrypting your Kubernetes secrets
KeyPolicy:
Version: '2012-10-17'
Id: EKS-key
Statement:
- Sid: Enable IAM User Permissions
Effect: Allow
Principal:
AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
Action: kms:*
Resource: '*'
- Sid: Allow administration of the key
Effect: Allow
Principal:
AWS: !Sub
- 'arn:aws:iam::${AWS::AccountId}:user/${AWSaccountAdminUserName}'
- {AWSaccountAdminUserName: !Ref AWSaccountAdminUserName}
Action:
- kms:Create*
- kms:Describe*
- kms:Enable*
- kms:List*
- kms:Put*
- kms:Update*
- kms:Revoke*
- kms:Disable*
- kms:Get*
- kms:Delete*
- kms:ScheduleKeyDeletion
- kms:CancelKeyDeletion
Resource: '*'
EksKMSAlias:
Type: AWS::KMS::Alias
Properties:
AliasName: !Ref EksKMSAliasName
TargetKeyId: !GetAtt EksKMS.Arn
This complete set of resource will spin up as a single Cloudformation stack . Please fork my github file here.
EKS Compute Resource
This section deploys the actual bare-metals of your cluster(hah Gotcha! there is no bare metal, its AWS, its upon some cloud in the sky).
We deploy the service section of EKS and after that the worker node cluster of EKS. Every parameter that we use here comes from the last two templates.
It is worth mentioning that the smallest t-shirt size of EC2 that you can use is t2.small and the smallest disk that can be attached to it is of 4GB .
## EKS Build ##
EKS:
Type: AWS::EKS::Cluster
Properties:
Name: !Ref EKSClusterName
ResourcesVpcConfig:
SecurityGroupIds: !Ref EksServiceSecurityGroupID
SubnetIds: !Ref SubnetIds
RoleArn: !Ref EksClusterServiceRole
Version: !Ref KubernetesVersion
EKSNodegroup:
Type: AWS::EKS::Nodegroup
DependsOn: EKS
Properties:
AmiType: !Ref AmiType
ClusterName: !Ref EKS
DiskSize: !Ref DiskSize
ForceUpdateEnabled: !Ref ForceUpdateEnabled
InstanceTypes:
- !Ref InstanceTypes
NodegroupName: !Ref EKSManagedNodeGroupName
NodeRole: !Ref EksClusterNodeRoleIP
ScalingConfig:
MinSize: !Ref NodeAutoScalingGroupMinSize
DesiredSize: !Ref NodeAutoScalingGroupDesiredSize
MaxSize: !Ref NodeAutoScalingGroupMaxSize
Subnets: !Ref SubnetIds
RemoteAccess:
Ec2SshKey: !Ref Ec2SshKey
This complete set of resource will spin up as a single Cloudformation stack . Please fork my github file here.
The cluster takes about 10-15 mins to be ready. To access the cluster, we will need to install kubectl on our local system. Kubectl is a command line tool for controlling Kubernetes clusters, allowing you to run commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs.
We need to configure the kubectl to talk to our EKS cluster. For this we need to go and edit the file ~./kube/config with the details of the cluster we find on the EKS tab of the AWS console. The format of the file will be as such
clusters:
- cluster:
certificate-authority-data: << TOKEN >>
server: << ENDPOINT URL >>
name: << ARN OF THE CLUSTER >>
contexts:
- context:
cluster: << ARN OF THE CLUSTER >>
user: << ARN OF THE CLUSTER >>
name: << CLUSTER ID >>
current-context: << CLUSTER ID >>
kind: Config
preferences: {}
users:
- name: << CLUSTER ID >>
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- token
- -i
- << CLUSTER ID >>
- -r
- << ADMIN USER ROLE ARN >>
command: aws-iam-authenticator
env:
- name: AWS_PROFILE
value: << AWS PROFILE >>
NOTE: This is error prone and a much easier way of doing it will be leveragin the aws cli and its admin role profile to modify this file for you by executing the following :
aws eks update-kubeconfig --name my-eks-cluster --profile <ADMIN-PROFILE> --region <EKS-Cluster-Region>
If everything is good till this point, you should be good to hit a sane kubectl command to see whats the fuss it all about.
kubectl get namespace
WARNING:
Enjoy your K8s responsibly.
Please dont keep this setup running. Cloudformations are super easy to recreate, so take this down once you are done for the day. Mr.Bezos has enough $$$s as we speak.
Top comments (0)