For whatever reason, in my head, running software on a raspberry pi is a lot cooler than running it on my macbook. I have one running pi-hole and a couple others just lying around, so I thought of something to use them for. I have a couple web scrapers running for whenarethefights.com on lambda and thought it would be nice to move those to a pi-cluster.
I already had 3 pis so here was my shopping list:
So after I got everything I just need to flash each sd card with an OS. A list of them can be found here. I chose Ubuntu Server for no reason other than I have used it before on Digital Ocean. To flash each card I used Etcher. Its pretty straight forward and keeps me from having to use dd. I knew I wanted to run these in a headless mode so I need to place an empty file named
ssh on each drive after they were done flashing. To do this you will need to re-mount the drive after Etcher ejects it; The easiest way to do that on Mac is to use the disk utility.
Once mounted, you can open up the terminal to place the empty file.
cd /Volumes # find the drive with the rpi OS cd <OS Drive> touch ssh cd
Unmount the drive and repeat for each drive you have
I like to set one pi up at a time because each pi sets its hostname as ubuntu and I don't want to arp for the ip of each one and guess. My plan was to name each pi green-pi, white-pi, etc according to whatever color ethernet cable is plugged into it. Load all the sd cards but only plug one of the pi in and wait a min. You should now be able to ssh into the first pi.
ssh ubuntu@ubuntu # When asked for a password, enter 'ubuntu'. You will be asked to change it to a new one.
Now we need to change the hostname and restart so our dnsserver picks up the new name.
hostnamectl set-hostname green-pi sudo reboot
If you try to ssh to
ubuntu again it won't let you since the ssh keys changed, so you will need to delete
ubuntu from known users. Open up
~/.ssh/known_hosts and delete the line with
ubuntu on it.
vim ~/.ssh/known_hosts ubuntu,192.168.0.88 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGRsJ...
Repeat for each pi.
So now that they are all set up, we need to log into each one so we know we can. Since the hostname was updated we should be able to use the new hostname.
ssh ubuntu@green-pi # Enter the new password you created
Once we know that works, let's copy our keys over so we don't need to enter a password anymore.
ssh-copy-id ubuntu@green-pi # Enter the password again. When it's successful it will log you out and tell you to login again. ssh ubuntu@green-pi
You should be able to login without a password now.
Less typing is always better for me so the final step is to set our ssh config so I don't have to type the username anymore. Open up ~/.ssh/config and place the following:
Host green-pi HostName green-pi User ubuntu Host white-pi HostName white-pi User ubuntu Host blue-pi HostName blue-pi User ubuntu
After adding that you can just ssh with the hostname and be done.
Now that the pi cluster is set up it can be used to do anything. We can do all the buzzword things like Blockchain, Kubernetes, crypto mining.
Just kidding, we're just gonna put some web scrapers on them.
Current word count: ~1681 / 250,000
Daily average: ~560