DEV Community

loading...

Kubernetes - quickly set up your own cluster

Darius Juodokas
Performance Engineer | Java developer | Linux/Unix administrator | (dev|net|sys|NO)Ops
・6 min read

Want to quickly spin up your own Kubernetes cluster? Here's how.

  • install docker
  • install kubeadm, kubectl
  • disable swap
  • kubeadm init
  • kubeadm join
  • kubectl get nodes
  • install network fabric, e.g. weave
  • install nginx controller
  • get ingress controller's Endpoint
  • install helm

This is what it looks like


                                      [ CLIENT ]
                                          ↓
                                 [ Ingress Endpoint ] NodePort :80/tcp, :443/tcp
                                          ↓
                                [ Ingress Controller ] (nginx pod)
            ______________________________|______________________________
            ↓                             ↓                             ↓
     [ ingress-app1 ]  /app1/*     [ ingress-app2 ] /app2/*      [ ingress-app3 ] /app3/*
            ↓                             ↓                             ↓
     [ endpoint-app1 ] :8080/tcp   [ endpoint-app2 ] :8080/tcp   [ endpoint-app3 ] :8080/tcp
            ↓                             ↓                             ↓
       [ svc-app1 ]                  [ svc-app2 ]                  [ svc-app3 ]
    ________|________             ________|________             ________|________
    ↓       ↓       ↓             ↓       ↓       ↓             ↓       ↓       ↓
  [pod1]  [pod2]  [pod3]        [pod1]  [pod2]  [pod3]        [pod1]  [pod2]  [pod3]
    ↓       ↓       ↓             ↓       ↓       ↓             ↓       ↓       ↓
 [cont0] [cont0] [cont0]       [cont0] [cont0] [cont0]       [cont0]  [cont0] [cont0]

Enter fullscreen mode Exit fullscreen mode

Install Docker

Ubuntu


sudo apt-get update
sudo apt-get install docker-ce
sudo systemctl status docker

Enter fullscreen mode Exit fullscreen mode

Install kubeadm and kubectl

kubectl

kubernetes.io link

generic linux

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo mv kubectl /usr/bin/kubectl
sudo chown root:root /usr/bin/kubectl
sudo chmod 0755 /usr/bin/kubectl
kubectl version --client

Enter fullscreen mode Exit fullscreen mode

ubuntu

sudo apt-get update && sudo apt-get install -y apt-transport-https gnupg2 curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl
kubectl version --client

Enter fullscreen mode Exit fullscreen mode

RH/CentOS

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
sudo yum install -y kubectl
kubectl version --client

Enter fullscreen mode Exit fullscreen mode

kubeadm

kubernetes.io link

generic linux

Install CNI plugins (required for most pod network):


CNI_VERSION="v0.8.2"
sudo mkdir -p /opt/cni/bin
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-amd64-${CNI_VERSION}.tgz" | sudo tar -C /opt/cni/bin -xz

Enter fullscreen mode Exit fullscreen mode

Define the directory to download command files

Note: The DOWNLOAD_DIR variable must be set to a writable directory. If you are running Flatcar Container Linux, set DOWNLOAD_DIR=/opt/bin.


DOWNLOAD_DIR=/usr/local/bin
sudo mkdir -p $DOWNLOAD_DIR

Enter fullscreen mode Exit fullscreen mode

Install crictl (required for kubeadm / Kubelet Container Runtime Interface (CRI))


CRICTL_VERSION="v1.17.0"
curl -L "https://github.com/kubernetes-sigs/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-amd64.tar.gz" | sudo tar -C $DOWNLOAD_DIR -xz

Enter fullscreen mode Exit fullscreen mode

Install kubeadm, kubelet, kubectl and add a kubelet systemd service:


RELEASE="$(curl -sSL https://dl.k8s.io/release/stable.txt)"
cd $DOWNLOAD_DIR
sudo curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
sudo chmod +x {kubeadm,kubelet,kubectl}

RELEASE_VERSION="v0.4.0"
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service
sudo mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

Enter fullscreen mode Exit fullscreen mode

Enable and start kubelet:


sudo systemctl enable --now kubelet

Enter fullscreen mode Exit fullscreen mode

Ubuntu

sudo apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Enter fullscreen mode Exit fullscreen mode

The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do.


RH/CentOS

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

# Set SELinux in permissive mode (effectively disabling it)
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet

Enter fullscreen mode Exit fullscreen mode

Notes:

  • Setting SELinux in permissive mode by running setenforce 0 and sed ... effectively disables it. This is required to allow containers to access the host filesystem, which is needed by pod networks for example. You have to do this until SELinux support is improved in the kubelet.

  • You can leave SELinux enabled if you know how to configure it but it may require settings that are not supported by kubeadm.

The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do.


Disable SWAP


sudo swapoff -a
sudo sed -i.bak -r 's/(.+ swap .+)/#\1/' /etc/fstab

Enter fullscreen mode Exit fullscreen mode

Initialize control plane (master node)


sudo kubeadm init

Enter fullscreen mode Exit fullscreen mode

workaround the problems (if any logged)

note down the kubeadm join <...> command in the output


mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Enter fullscreen mode Exit fullscreen mode

P.S. Should you need to undo any effects of kubeadm, run


sudo kubeadm reset

Enter fullscreen mode Exit fullscreen mode

Join cluster nodes


ssh remote_server -l username
## install kubeadm
## disable swap
## run the complete `kubeadm join <...>` command acquired from the `kubeadm init` output in the control plane

Enter fullscreen mode Exit fullscreen mode

If any errors logged - resolve them as you go

Verify nodes

In control plane


kubectl get nodes

Enter fullscreen mode Exit fullscreen mode

all nodes should be Ready.

Install network fabric

There are multiple network implementations: flannel, weave, calico, etc.

Flannel or weave are both perfectly fine for a local setup

Flannel

link

For Kubernetes v1.17+


kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Enter fullscreen mode Exit fullscreen mode

Weave

link

Kubernetes versions 1.6 and above:


kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

Enter fullscreen mode Exit fullscreen mode

Kubernetes versions up to 1.5:


kubectl apply -f https://git.io/weave-kube

Enter fullscreen mode Exit fullscreen mode

Calico

link

  1. Install the Tigera Calico operator and custom resource definitions.

    
    kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml
    
    
  2. Install Calico by creating the necessary custom resource. For more information on configuration options available in this manifest, see the installation reference.

    
    kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml
    
    

    Note: Before creating this manifest, read its contents and make sure its settings are correct for your environment. For example, you may need to change the default IP pool CIDR to match your pod network CIDR.

  3. Confirm that all of the pods are running with the following command.

    
    watch kubectl get pods -n calico-system
    
    

    Wait until each pod has the STATUS of Running.

    Note: The Tigera operator installs resources in the calico-system namespace. Other install methods may use the kube-system namespace instead.

  4. Remove the taints on the master so that you can schedule pods on it.

    
    kubectl taint nodes --all node-role.kubernetes.io/master-
    
    

    It should return the following.

    
    node/<your-hostname> untainted
    
    
  5. Confirm that you now have a node in your cluster with the following command.

    
    kubectl get nodes -o wide
    
    

    It should return something like the following.

    
    NAME              STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION    CONTAINER-RUNTIME
    <your-hostname>   Ready    master   52m   v1.12.2   10.128.0.28   <none>        Ubuntu 18.04.1 LTS   4.15.0-1023-gcp   docker://18.6.1
    
    

Install Ingress Controller (nginx)

link

Bare-metal

Using NodePort:


kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/baremetal/deploy.yaml

Enter fullscreen mode Exit fullscreen mode

Using HELM


helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx

Enter fullscreen mode Exit fullscreen mode

Get ingess controller's Endpoint


kubectl -n ingress-nginx get svc | grep NodePort

Enter fullscreen mode Exit fullscreen mode

Use this IP to query your ingresses

Install helm

link

generic linux

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

Enter fullscreen mode Exit fullscreen mode

ubuntu

curl https://baltocdn.com/helm/signing.asc | sudo apt-key add -
sudo apt-get install apt-transport-https --yes
echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm

Enter fullscreen mode Exit fullscreen mode

Done

Now deploy your pods, services and ingresses and access them through the NodePort endpoint's IP:port.

Written with StackEdit.

Discussion (0)