Building a Kubernetes cluster at home, part One.
This chapter will focus on the preparation work. I will wire all computers together and I configure one of my laptops to serve as an access point to the cluster and also it will serve as a gateway from the cluster to the outside world. I already gathered about 5 old computers and a simple non-managed Ethernet switch with 5 ports. I will start with the simplest possible solutions and will add complexity as goes.
Plugin the first laptop. This one will serve as a gateway as it has two network interfaces. Wifi will interface with my router network and the Ethernet will interface private network. When you connect your laptop to your home WIFI, it usually will obtain the dynamic IP address from a router, something like 192.168.0.15. The router is set as the default gateway for all internet traffic. When you plug in the computer with an Ethernet cable, it does the same (minus the WIFI security bit). The operating system usually activates only one network interface (Ethernet takes precedes if both are present). In our case, we need to enable both so the machine can communicate with the cluster and with me.
mnode@mnode:~$ ip address show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000 link/ether 49:ba:4e:4a:eb:c5 brd ff:ff:ff:ff:ff:ff altname enp1s0 inet 192.168.3.34/24 brd 192.168.3.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fc80::4aba:4eff:fe39:ebc5/64 scope link valid_lft forever preferred_lft forever 3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether da:64:6a:9f:cd:1f brd ff:ff:ff:ff:ff:ff altname wlp2s0 inet 192.168.1.5/24 metric 600 brd 192.168.1.255 scope global dynamic wlan0 valid_lft 86379sec preferred_lft 86379sec inet6 fe80::d66a:6aff:fe9c:cd1f/64 scope link valid_lft forever preferred_lft forever
The above is the list of all available network interfaces. Those interesting for us are
eth0 which is the Ethernet interface and
wlan0 which is WIFI. from the above list we can see that Ethernet is in the state DOWN. For comparison the wireless LAN is wlan0 and the state is UP.
Running the following command will turn on the networking interface:
sudo ip link set eth0 up Running
ip address show again will show that the
eth0 interface is now with the flag
UP, so the operation worked. However, doing this on the Ubuntu desktop might result in the operating system (Ubuntu) reporting an error message activation
of network connection failed Ubuntu
This is because Ubuntu Desktop uses NetworkManager to manage all the network interfaces by default. It is the application responsible for network management via UI. This is not the case in Ubuntu Server which I will be using. The documentation can be found here NetworkManager It can be completely turned off by choosing a different renderer or a specific networking interface can be excluded from under the management of NetworkManager. The configuration is in the file
/etc/NetworkManager/NetworkManager.conf. On UI-less versions like Ubuntu Server, there is no NetworkManager installed. The default network renderer is systemd-networkd. In both flavors of Ubuntu, the networking configuration is done via Netplan and the configuration for Netplan is stored in yaml file in the location
/etc/netplan/*.yaml. There could be more than one configuration file. More information on this are to be found in the provided link.
The default configuration on the Ubuntu desktop might look like this and it basically tells the system that NetworkManager manages all networking devices:
network: version: 2 renderer: NetworkManager
If there is no renderer specified, Ubuntu by default uses systemd-networkd as the network renderer. We want to customize values for the gateway and the static IP address as we need to manage two separate network interfaces and we need to have them
UP at the same time.
network: wifis: wlan0: dhcp4: true dhcp6: true access-points: "WIFI-Name": password: "wifi_password" ethernets: eth0: dhcp4: false addresses: [192.168.3.34/24] routes: - to: default - via: 192.168.1.1
The above configuration tells the system that both interfaces are managed (by default) by systemd-networkd specifies the access point for WIFI and password. It specifies that WIFI takes the IP address from the router for both ipv4 and ipv6 protocols. It specifies that the Ethernet interface does not try to obtain IP addresses from DHCP server and takes the hard-coded one. The format 192.168.3.34/24 specifies that the IP address in the domain 192.168.3.xx and the address 192.168.3.34 within the domain. There is a lot that can be done configuring the netplan.
NOTE: for the desktop version turning off the NetworkManager results in an inability to select the network from the dropdown menu in the taskbar.
An important part is that the Wifi subnet is not clashing with the ethernet subnet. In my case wifi runs on
192.168.1.0/24 and I set up the ethernet IP to
After this netplan's new configuration needs to be applied bash sudo
netplan apply In case this is done on the Ubuntu desktop, there might be a message
systemd-networkd not running, output will be incomplete. this indicates the network manager is not managing the network anymore, but the default networkd is not running yet. it needs to be started
bash systemctl start systemd-networkd I have to note also that to setup the Ubuntu server is much simpler as there is no NetworkManager and the systemd-networkd is a default network renderer.
Step 2, adding another compouter to cluster
This should be fairly simple. For me the second guinea pig is raspberry pi3, running Ubuntu for raspberry, not too dissimilar to the Ubuntu server. The only configuration which I need to do here is to assign a static IP address and point to the default gateway:
yaml network: version: 2 ethernets: eth0: dhcp4: false addresses: [192.168.3.40/24] after
bash sudo netplan apply I can reach the Raspberry node from the gateway laptop and vice versa. I also can reach the gateway laptop on the Wifi domain. I should be good to go to plug in the rest of the computers and do the same to them After that I could start installing k3s and build the Kubernetes cluster.
It is also worth noting - when you are building the cluster out of old laptops - it is handy to prevent them to go sleep or hibernate when the lid is closed. That way is is possible to stack them on each other to save the space.
On Ubuntu server as per documentation this needs to be added to
/etc/systemd/sleep.conf or preferably new file in the location
/etc/systemd/sleep.conf.d/00-sleep.conf to be created as it takes precedence over default configuration in the former file.
[Sleep] AllowSuspend=no AllowHibernation=no AllowSuspendThenHibernate=no AllowHybridSleep=no
Step 3, optional - internet access
However, there is one more thing I wanted to do. It is not essential and perhaps even not recommended, but for my convenience want to have a connection to the internet from the individual machines in the cluster. This will help me with the installation of the required software on individual nodes. This is not a production environment so I can easily get away with it. The same scenario would apply if you had a desktop machine without WIFI. You could use a laptop to connect to WIFI and use the laptop's Ethernet interface to connect with your desktop and to serve as a router.
To do this we need to add a few rules in IP tables. We need to forward all internet traffic from ethernet to wifi interface and back. We also want to set up network address translation(NAT). For Linux operating system to enable IP forwarding. This can typically be done by editing the
/etc/sysctl.conf file and adding the following line:
net.ipv4.ip_forward = 1
sysctl -w net.ipv4.ip_forward=1 We also need to add the following rules in the ip tables:
iptables -A FORWARD -i eth0 -o wlan0 -j ACCEPT iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE iptables -A FORWARD -i wlan0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
The first rule forwards all the traffic coming from the individual machines on our subnet to out via wireless LAN.
the second adds a record to NAT table to masquerade packets coming from the individual machines from the private subnet via the gateway laptop as packets coming from the gateway laptop itself (the home router does the same thing again when you are reaching out to the internet)
The third rule forwards all the traffic originating from the individual machines back to them
Save your IP tables rules and make sure they are automatically loaded at startup. This can typically be done by using the
iptables-restore commands. For example:
iptables-save > /etc/iptables.rules
Edit the /etc/rc.local file and add the following line before the exit 0 line:
iptables-restore < /etc/iptables.rules
Once this is done it should be possible to access the internet from a machine connected to an Ethernet switch provided the configuration in step 2 is followed.
After the above steps, I do have a cluster of computers wired together via an Ethernet switch. The computers share a private network and can communicate with each other. The private network is connected to the router via a wireless connection and the internet is also reachable from inside the private network.
Latest comments (0)