DEV Community

Cover image for Turing Pi 2 Home cluster
Tomas Sirio
Tomas Sirio

Posted on

Turing Pi 2 Home cluster

Table of Contents

The Story

In June 2022 I came across a Kickstarter about a board capable of having up to 4 Raspberry Pi 4 Compute Modules at once.
Without much deliberation, I decided to back the project and purchase the board.

Fast-forward two years, and my Turing Pi 2 was still sitting unopened in its box on a shelf
Life had gotten in the way, and I was unsure why I had bought it in the first place.

Turing Pi Board

However, I finally decided to give it a try. I wanted to learn more about clustering and had never built a complete Kubernetes cluster from scratch before. So, I went into spending mode and acquired three Raspberry Pi 4s (8GB RAM, 8GB internal storage) and one Nvidia Jetson Nano (4GB).

Raspberry Pi 4 CMs

Given the board's versatility, I could mix different Compute Modules. I decided to include a Jetson Nano, thinking it might allow me to experiment with CUDA drivers in the future and delve into machine learning. Who knows? I might even end up hosting my own GPT assistant on this Kubernetes cluster.

Jetson Nano
(Spoilers: It didn't happen)

The Plan

My initial plan included the 3 Pi 4 CM and the Jetson Nano hosted on the board. Also, I planned to use a 1TB SSD drive for storage and a Wi-Fi card for internet access.
However, after encountering numerous difficulties with the Jetson Nano's setup process and its poor documentation, I decided to return it. Instead, I opted for a fourth Raspberry Pi 4.

Additionally, I had an old Raspberry Pi 4 with 4GB of RAM lying around, so I decided to incorporate it as a fifth node.

Old Raspberry

Turing Pi 2

The Turing Pi 2 is a Mini ITX form factor board that can accommodate up to four Raspberry Pi Compute Modules (also compatible with Jetson Nanos and a Turing Compute Module). It features a PCI Express port, two NVME ports, two SATA ports, and a USB port for flashing the Compute Modules.

Plan 0

  • Node 1:
    • USB 2.0 port (For flashing the Compute Modules)
    • HDMI port (For debugging)
    • PCI Express port (For the Wi-Fi card)
  • Node 2:
    • I would use this one for NVME storage, but it's not compatible with Raspberry Pi 4s
  • Node 3:
    • The SATA ports, however, can be used here. So this node will have the NFS shared drive
  • Node 4:
    • USB 3.0 ports (If I ever need them)
  • My Old Raspberry:
    • Would be the Kubernetes master node. There is no special reason, I can think better about my setup this way

Plan 2

Ultimately, the idea is to host a Media Server with some add-ons

Flow Chart

The Set-up

It's been a while since I've put together a computer, and it was the first time I played around with Compute Modules and their adapters, so that was plenty of fun for the weekend.
Since my wallet was hot but still not burning, I thought, why the hell not add a nice case for it?

Unboxing 1

Given the Mini ITX form factor of the board, I could fit it into whatever fancy ITX case I could find on Amazon. The Qube 500 got me through and through. I was already making a DIY cluster, the best case for such a thing, was a DIY as well.

Qube 500

I also added a 650W power supply (total overkill), one small Wi-Fi Mini PCI Express card, and a 1TB SATA SSD.

Putting up the 'thing' together was fairly simple. A bit of thermal paste between the Compute Modules and their heat sinks and clutching them together with their Adapters before setting them up in order in the Turing Board;

Turing Pi Board 1

I mentioned the order because it was a significant part of the project. The Turing Pi 2 offers management of its ports distributed throughout the compute modules. In this case, the PCI Express 1 was managed by the First node while the SSD drive was managed by the 3rd one. The 2nd could handle the NVME port and the 4th one the other SSD IIRC but I had no use for them right now.

The Flashing

I've installed Raspberry Pis before, but never Compute Modules. The Turing Pi 2 has a USB port in the back which is used for flashing the Compute Modules.
Unfortunately, I tried using a USB A to USB A cable that was not a data transfer cable, so while I waited for Amazon to deliver a cable, I found another way of flashing the Compute Modules.

The Turing Pi 2 has a CLI tool that can be used not only to flash the Compute Modules but also to manage their power, reset them, check some stats, and so on.

The command used for installing the Compute Modules was:

tpi flash -i /path/to/image -n {nodes 1 to 4}
Enter fullscreen mode Exit fullscreen mode

Pretty straightforward process I thought to myself before realizing that the Raspbian image doesn't come with SSH enabled by default.
This, of course, is not Turing's responsibility. I should have waited for that cable, but oh well.

To fix this, I had to mount the image on my local machine and add an empty file named ssh in the boot partition. This would enable SSH by default.

sudo mkdir /mnt/pi-boot
sudo mount /dev/sdX1 /mnt/pi-boot

sudo touch /mnt/pi-boot/ssh
sudo umount /mnt/pi-boot
Enter fullscreen mode Exit fullscreen mode

Now all my pis were ready to be used. I connected them to the network and started configuring them. There was little to be configured since I was going to use them as Kubernetes nodes.
But things like vim and updating the system were necessary.

Turing 1

This also gave me the chance to learn how to use Tmux. Best tool I've learned in a while.

The Storage

If you recall a few paragraphs above, I mentioned that the 3rd node would be used for the NFS shared drive. I had the 1TB SSD drive that I was going to use for this purpose. I had to format it and mount it in the 3rd node.
But I also needed to install the NFS server in this node and configure it in the other nodes. Is this recommended for a production environment? Hell no, but it's a home cluster, so I'm not too worried about it.

Tmux1

Here are the steps I took to configure the NFS server:

pi@turing-03:/mnt/ssd/data $ lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda            8:0    0 953.9G  0 disk /mnt/ssd
mmcblk0      179:0    0   7.3G  0 disk
├─mmcblk0p1  179:1    0   512M  0 part /boot/firmware
└─mmcblk0p2  179:2    0   6.8G  0 part /
mmcblk0boot0 179:32   0     4M  1 disk
mmcblk0boot1 179:64   0     4M  1 disk
Enter fullscreen mode Exit fullscreen mode

First I checked the drive was mounted correctly. Then I installed the NFS server:

sudo mkdir /mnt/nfs_share
sudo mount /dev/sda /mnt/nfs_share
Enter fullscreen mode Exit fullscreen mode

Added it to the fstab file to make sure it was mounted on boot:

(adding the following line to the /etc/fstab file)

echo '/dev/sda /mnt/ssd ext4 defaults 0 0' | sudo tee -a /etc/fstab
Enter fullscreen mode Exit fullscreen mode

Now installing nfs-kernel-server:

sudo apt update
sudo apt install nfs-kernel-server
Enter fullscreen mode Exit fullscreen mode

And adding my drive to the /etc/exports file:

echo '/mnt/ssd *(rw,sync,no_subtree_check,no_root_squash)' | sudo tee -a /etc/exports
Enter fullscreen mode Exit fullscreen mode

Now on the other nodes I had to install nfs-common:

sudo apt update
sudo apt install nfs-common
Enter fullscreen mode Exit fullscreen mode

And hooking the drive to each node:

sudo mount -t nfs {IP-for-the-drives-node}:/mnt/ssd /mnt
Enter fullscreen mode Exit fullscreen mode

Tmux 0

Neofetch installed in all nodes because I'm fancy

The Kubernetes Cluster

I had never set up a Kubernetes cluster from scratch before, but I've been watching a lot of Jeff Geerling's videos on the subject... This is experience enough, right?

Jeff led me to K3s using Ansible, a lightweight Kubernetes distribution that is perfect for my home cluster and a pre-defined way of installing it because I don't have pre-requirements nor the idea on how to set it up otherwise.

The installation was pretty straightforward. I had to install it in all nodes, but I had to make sure the master node was the first one to be installed.
So first I cloned the k3s-ansible repository:

git clone https://github.com/k3s-io/k3s-ansible.git
Enter fullscreen mode Exit fullscreen mode

Then I had to configure the inventory file. My master node, as I mentioned before, was my old Raspberry Pi 4. So I had to make sure it was the first one in the inventory file. I also had to make sure the other nodes were in the correct groups.:

k3s_cluster:
  children:
    server:
      hosts:
        192.168.2.105:
    agent:
      hosts:
        192.168.2.101:
        192.168.2.102:
        192.168.2.103:
        192.168.2.104:
Enter fullscreen mode Exit fullscreen mode

In that same file, I had to set up an encryption token. The file indicates how to do this so I won't go into details here.

Then I had to run the playbook:

cd k3s-ansible
ansible-playbook playbooks/site.yml -i inventory.yml
Enter fullscreen mode Exit fullscreen mode

That's it. As far as the installation goes, I had a Kubernetes cluster up and running. I had to install K9s in my local machine to manage the cluster and bind the cluster to the ./kube/config file.

The Applications

Lastly, I had to install the applications I wanted to run in the cluster. I had some ideas on what I wanted.

  • I wanted to have both a Media Server with scheduled downloads
  • A Pi-hole instance to my network's DNS and block all ads in all devices at home
  • A Retroarch instance to play some old games and share the save games across my home's network (looking at you Megaman Battle Network 6 in all my devices)

That's where my repository comes in.

Repository 0

For the Media Server, I decided to use:

As an example, I'll show you how I installed Sonarr using kubectl. The other applications were installed in a similar fashion.

For each Application, I created 3 files:

  • deployment.yaml is the configuration for each of the pods running the application
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sonarr
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sonarr
  template:
    metadata:
      labels:
        app: sonarr
    spec:
      containers:
      - name: sonarr
        image: linuxserver/sonarr
        ports:
        - containerPort: 8989
        env:
        - name: PUID
          value: "911"
        - name: PGID
          value: "911"
        - name: TZ
          value: "Europe/Amsterdam"
        volumeMounts:
        - mountPath: /data
          name: data
        - name: config
          mountPath: /config
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: nfs-pvc
      - name: config
        persistentVolumeClaim:
          claimName: nfs-config-pvc
Enter fullscreen mode Exit fullscreen mode
  • service.yaml is the configuration for the service that will expose the application to the cluster
apiVersion: v1
kind: Service
metadata:
  name: sonarr
spec:
  selector:
    app: sonarr
  ports:
    - port: 80
      targetPort: 8989
  type: ClusterIP
Enter fullscreen mode Exit fullscreen mode
  • ingress.yaml and this is the configuration for the ingress that will expose the application to my network
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sonarr
spec:
  ingressClassName: nginx 
  rules:
  - host: sonarr.localhome.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: sonarr
            port: 
              number: 80
Enter fullscreen mode Exit fullscreen mode

Then we deploy all of them using kubectl:

kubectl apply -f sonarr/deployment.yaml
kubectl apply -f sonarr/service.yaml
kubectl apply -f sonarr/ingress.yaml
Enter fullscreen mode Exit fullscreen mode

As you can see, I'm using NFS-backed persistent storage for the data and the configuration of the applications.
In the repository, you can find the nfs-pv.yaml and nfs-pvc.yaml files that I used to create the NFS storage.
Additionally, I created another persistent volume claim for the configuration of the applications.

K9s 0

The final build

Even though the case looks amazing, it's a bit too big for a Raspberry Pi Cluster. A Mini ITX case would have suited my needs as well, but I have to admit, I'm a sucker for the DIY stuff.

Full Build 0

Also, a Sucker for LEDs in general. I didn't add any more lights to the case, I think that the board does a nice job already.
Unfortunately, the fan pins were not compatible with the board and I didn't buy a fan controller or a pin for the motherboard. I might in the future.

Working 0

Sometimes you just have to sit back and enjoy the view.

Full Build 1

And finally, the Turing Pi 2 Home Cluster is up and running and my house is not a mess anymore.

Full build 2

The Future

Only time will tell what I'll do with this cluster.
However, I've been thinking of adding Prometheus and Grafana to have some metrics and nice graphs to check on the cluster.

Migrating all my Kubernetes files to Helm would be a good idea as well.

Lastly, the Retroarch instance is still in the works. Maybe in the works is a bit too optimistic given that the pod lives in CrashLoopBackOff state. But I'll get there.

The End

If you've reached the end of this post, I thank you for your time. I hope you've enjoyed it as much as I did both putting up the cluster together and writing about it.

Top comments (7)

Collapse
 
terry_gilligan_e4bf4680b8 profile image
Terry Gilligan

Thank you for your detailed posting about your Pi cluster. I have routed out 4x Pi 3b that were gathering dust in a drawer and am inspired to build my own cluster. Thank you again for your article. Live long and prosper. :))

Collapse
 
benborla profile image
Ben Borla

Nicely done!

Collapse
 
frickingruvin profile image
Doug Wilson

Really, really interesting. Thank you for sharing your journey and lessons learned!

Collapse
 
tomassirio profile image
Tomas Sirio

Thank you for reading!

Collapse
 
chris_magee_1989 profile image
Chris Magee

Awesome work. Nicely done..

Collapse
 
juststevemcd profile image
Steve McDougall

Amazing write up, it's really making me jealous! I've always wanted a Pi cluster for K3s! I may have to use this as a guide!

Collapse
 
tomassirio profile image
Tomas Sirio

Give it a try! If you are on a low budget, you can start with two Raspberry Pis. You don't need the case, the board, or anything else; those were just gimmicks I bought compulsively.

I added most of the links to the stuff I used here. However, the most remarkable one is still Jeff Geerlin's YouTube channel. That guy is not only wholesome but quite knowledgeable, he will be able to guide you better than I can.

Have fun!