Versions

⚠️ Warning! Use this fork: https://github.com/kubernetes-incubator/kubespray/pull/3486.

Motivation

This post serves as documentation for creating a kubernetes cluster from nothing using kubespray on aws with ubuntu images behind a bastion host. I could not find all the documentation that puts all of this together and wanted to write it down for myself the next time I need to do this.

Steps

1. Clone kubespray and set up some default files

git clone https://github.com/bartlaarhoven/kubespray
cd kubespray
virtualenv ks && . ks/bin/activate
pip install -r requirements.txt
cp -Rp inventory/sample/ inventory/mycluster

1. Create an IAM user with admin privileges in some account (TODO probably scope this down?)
2. Create an EC2 key pair
3. Copy the terraform environment file to credentials.tfvars and modify it with the user's key and secret along with the ssh key pair name and the region you'd like the infrastructure to exist in.
4. Customize the terraform file with the architecture you'd like, I used 1 master, 1 worker, 1 etcd and left bastions as default. Also modify the inventory file to be ../../../inventory/mycluster/hosts.ini.
5. Modify the variables.tf to be

    data "aws_ami" "distro" {
      most_recent = true

      filter {
        name = "name"

        values = ["ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-*"]
      }

      filter {
        name = "virtualization-type"

        values = ["hvm"]
      }

      owners = ["099720109477"]
    }

1. Run terraform

    terraform apply --var-file credentials.tfvars

1. Modify the ansible.cfg file to use a bastion host by changing the ssh_args value to

    ssh_args = -F ssh-bastion.conf

1. Modify the hosts.ini file. Use the internal DNS names as the ansible_host for each of the instances in the private subnet and include the ansible_user to be ubuntu, for example:

    kubernetes-devtest-master0 ansible_host=ip-10-250-205-127.us-west-2.compute.internal ansible_user=ubuntu

1. Modify the bastion lines to be the public DNS names as the ansible_host and include the ansible_user to be ubuntu, for example:

    bastion-0 ansible_host=ec2-22-222-22-22.us-east-2.compute.amazonaws.com ansible_user=ubuntu

1. Run ansible-playbook

    ansible-playbook -i ./inventory/mycluster/hosts.ini ./cluster.yml -b --become-user=root --flush-cache

If you'd like more logs add -v or -vv up to -vvvvv. I also like to pipe this to tee and write the logs to disk or inspection later in case of failure.

Enjoy your new cluster~!

P.S. if anyone knows how to get the code as part of the item so the bulleted stuff works please comment, I'd love to fix the numbering.