DEV Community

Darius Juodokas
Darius Juodokas

Posted on

Kubernetes - quickly set up your own cluster

#kubernetes #k8s #howto #cheatsheet

Want to quickly spin up your own Kubernetes cluster? Here's how.

  • install docker
  • install kubeadm, kubectl
  • disable swap
  • kubeadm init
  • kubeadm join
  • kubectl get nodes
  • install network fabric, e.g. weave
  • install nginx controller
  • get ingress controller's Endpoint
  • install helm

This is what it looks like


                                      [ CLIENT ]
                                          ↓
                                 [ Ingress Endpoint ] NodePort :80/tcp, :443/tcp
                                          ↓
                                [ Ingress Controller ] (nginx pod)
            ______________________________|______________________________
            ↓                             ↓                             ↓
     [ ingress-app1 ]  /app1/*     [ ingress-app2 ] /app2/*      [ ingress-app3 ] /app3/*
            ↓                             ↓                             ↓
     [ endpoint-app1 ] :8080/tcp   [ endpoint-app2 ] :8080/tcp   [ endpoint-app3 ] :8080/tcp
            ↓                             ↓                             ↓
       [ svc-app1 ]                  [ svc-app2 ]                  [ svc-app3 ]
    ________|________             ________|________             ________|________
    ↓       ↓       ↓             ↓       ↓       ↓             ↓       ↓       ↓
  [pod1]  [pod2]  [pod3]        [pod1]  [pod2]  [pod3]        [pod1]  [pod2]  [pod3]
    ↓       ↓       ↓             ↓       ↓       ↓             ↓       ↓       ↓
 [cont0] [cont0] [cont0]       [cont0] [cont0] [cont0]       [cont0]  [cont0] [cont0]
Enter fullscreen mode Exit fullscreen mode

Install Docker

Ubuntu 


sudo apt-get update
sudo apt-get install docker-ce
sudo systemctl status docker
Enter fullscreen mode Exit fullscreen mode

Install kubeadm and kubectl

kubectl

kubernetes.io link

generic linux 

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo mv kubectl /usr/bin/kubectl
sudo chown root:root /usr/bin/kubectl
sudo chmod 0755 /usr/bin/kubectl
kubectl version --client
Enter fullscreen mode Exit fullscreen mode
ubuntu 

sudo apt-get update && sudo apt-get install -y apt-transport-https gnupg2 curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl
kubectl version --client
Enter fullscreen mode Exit fullscreen mode
RH/CentOS 

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
sudo yum install -y kubectl
kubectl version --client
Enter fullscreen mode Exit fullscreen mode

kubeadm

kubernetes.io link

generic linux

Install CNI plugins (required for most pod network):


CNI_VERSION="v0.8.2"
sudo mkdir -p /opt/cni/bin
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-amd64-${CNI_VERSION}.tgz" | sudo tar -C /opt/cni/bin -xz
Enter fullscreen mode Exit fullscreen mode

Define the directory to download command files

Note: The DOWNLOAD_DIR variable must be set to a writable directory. If you are running Flatcar Container Linux, set DOWNLOAD_DIR=/opt/bin.


DOWNLOAD_DIR=/usr/local/bin
sudo mkdir -p $DOWNLOAD_DIR
Enter fullscreen mode Exit fullscreen mode

Install crictl (required for kubeadm / Kubelet Container Runtime Interface (CRI))


CRICTL_VERSION="v1.17.0"
curl -L "https://github.com/kubernetes-sigs/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-amd64.tar.gz" | sudo tar -C $DOWNLOAD_DIR -xz
Enter fullscreen mode Exit fullscreen mode

Install kubeadm, kubelet, kubectl and add a kubelet systemd service:


RELEASE="$(curl -sSL https://dl.k8s.io/release/stable.txt)"
cd $DOWNLOAD_DIR
sudo curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
sudo chmod +x {kubeadm,kubelet,kubectl}

RELEASE_VERSION="v0.4.0"
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service
sudo mkdir -p /etc/systemd/system/kubelet.service.d
curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
Enter fullscreen mode Exit fullscreen mode

Enable and start kubelet:


sudo systemctl enable --now kubelet
Enter fullscreen mode Exit fullscreen mode
Ubuntu 

sudo apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
Enter fullscreen mode Exit fullscreen mode

The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do.

RH/CentOS 

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

# Set SELinux in permissive mode (effectively disabling it)
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet
Enter fullscreen mode Exit fullscreen mode

Notes:

  • Setting SELinux in permissive mode by running setenforce 0 and sed ... effectively disables it. This is required to allow containers to access the host filesystem, which is needed by pod networks for example. You have to do this until SELinux support is improved in the kubelet.

  • You can leave SELinux enabled if you know how to configure it but it may require settings that are not supported by kubeadm.

The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do.

Disable SWAP 


sudo swapoff -a
sudo sed -i.bak -r 's/(.+ swap .+)/#\1/' /etc/fstab
Enter fullscreen mode Exit fullscreen mode

Initialize control plane (master node) 


sudo kubeadm init
Enter fullscreen mode Exit fullscreen mode

workaround the problems (if any logged)

note down the kubeadm join <...> command in the output


mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Enter fullscreen mode Exit fullscreen mode

P.S. Should you need to undo any effects of kubeadm, run


sudo kubeadm reset
Enter fullscreen mode Exit fullscreen mode

Join cluster nodes 


ssh remote_server -l username
## install kubeadm
## disable swap
## run the complete `kubeadm join <...>` command acquired from the `kubeadm init` output in the control plane
Enter fullscreen mode Exit fullscreen mode

If any errors logged - resolve them as you go

Verify nodes

In control plane


kubectl get nodes
Enter fullscreen mode Exit fullscreen mode

all nodes should be Ready.

Install network fabric

There are multiple network implementations: flannel, weave, calico, etc.

Flannel or weave are both perfectly fine for a local setup

Flannel

link

For Kubernetes v1.17+


kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Enter fullscreen mode Exit fullscreen mode

Weave

link

Kubernetes versions 1.6 and above:


kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
Enter fullscreen mode Exit fullscreen mode

Kubernetes versions up to 1.5:


kubectl apply -f https://git.io/weave-kube
Enter fullscreen mode Exit fullscreen mode

Calico

link

  1. Install the Tigera Calico operator and custom resource definitions.

    
kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml

  2. Install Calico by creating the necessary custom resource. For more information on configuration options available in this manifest, see the installation reference.

    
kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml

    Note: Before creating this manifest, read its contents and make sure its settings are correct for your environment. For example, you may need to change the default IP pool CIDR to match your pod network CIDR.

  3. Confirm that all of the pods are running with the following command.

    
watch kubectl get pods -n calico-system

    Wait until each pod has the STATUS of Running.

    Note: The Tigera operator installs resources in the calico-system namespace. Other install methods may use the kube-system namespace instead.

  4. Remove the taints on the master so that you can schedule pods on it.

    
kubectl taint nodes --all node-role.kubernetes.io/master-

    It should return the following.

    
node/<your-hostname> untainted

  5. Confirm that you now have a node in your cluster with the following command.

    
kubectl get nodes -o wide

    It should return something like the following.

    
NAME              STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION    CONTAINER-RUNTIME
<your-hostname>   Ready    master   52m   v1.12.2   10.128.0.28   <none>        Ubuntu 18.04.1 LTS   4.15.0-1023-gcp   docker://18.6.1

Install Ingress Controller (nginx)

link

Bare-metal

Using NodePort:


kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/baremetal/deploy.yaml
Enter fullscreen mode Exit fullscreen mode

Using HELM


helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx
Enter fullscreen mode Exit fullscreen mode

Get ingess controller's Endpoint 


kubectl -n ingress-nginx get svc | grep NodePort
Enter fullscreen mode Exit fullscreen mode

Use this IP to query your ingresses

Install helm

link

generic linux 

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
Enter fullscreen mode Exit fullscreen mode
ubuntu 

curl https://baltocdn.com/helm/signing.asc | sudo apt-key add -
sudo apt-get install apt-transport-https --yes
echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm
Enter fullscreen mode Exit fullscreen mode

Done

Now deploy your pods, services and ingresses and access them through the NodePort endpoint's IP:port.

Written with StackEdit.

Top comments (0)

Read next

thangchung profile image

SpinKube - the first look at WebAssembly/WASI application (SpinApp) on Kubernetes

Thang Chung -

ken_mwaura1 profile image

Troubleshooting Common Issues in Kubernetes Deployments

Zoo Codes -

willvelida profile image

Managing Secrets in Azure Container Apps

Will Velida -

cicubeteam profile image

How to Fix K8S Exit Code 137?

CiCube Team -