This is great! I've been thinking about how to do something similar to make a "distribute cluster" with some friends.
You wrote this back in March. Is it still working well? Have you made any changes?
I have a few more thoughts:
What if you setup wireguard on each client physical server, but then setup a KVM (or other) virtual machine to act as the node in k8s cluster and assigned a bridge interface as the VMs only network interface? I'm thinking you might avoid some of the pain and custom config because from within the VM the k8s software will only see a single interface.
Have you thought about how to enable a different topology? If all the nodes have to go through the single server, it becomes a bottleneck. It would be amazing to make a fully connected mesh where every node can send traffic directly to every other node over VPN. However, I haven't figured out how to do this yet. An ideas?
Hi!
I didn't have to re-setup my cluster as it's still working as intended. I can only assume the same steps would work today as they did last year.
Yes, that would probably solve the configuration headache. However, that's at the price of more moving parts to maintain and bullet-proof (VM and hypervisor - both could be attacked) AND more resources wasted. Where on the other hand you could simply spend more time at configurations and make it work closer to the bare metal.
That would be neat! And there is a very easy way to get that mesh going. Connect the nodes over internet, rather than VPN :) Ofc that'd require every node to expose their kubernetes port (6443) to the internet. Yes, you could netfilter who can connect to that port and so, but my the goal of my setup was the exact opposite -- to hide the whole cluster behind a single machine.
I think you could work-around that SPoF by using another VPN server - one that supports MESH-like functionality. However, you'll most likely have lower baseline performance, as I doubt there's anything much faster than Wireguard.
FTR I'm using my setup for a non-prod cluster, so I'm not that worried about network performance.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
This is great! I've been thinking about how to do something similar to make a "distribute cluster" with some friends.
You wrote this back in March. Is it still working well? Have you made any changes?
I have a few more thoughts:
What if you setup wireguard on each client physical server, but then setup a KVM (or other) virtual machine to act as the node in k8s cluster and assigned a bridge interface as the VMs only network interface? I'm thinking you might avoid some of the pain and custom config because from within the VM the k8s software will only see a single interface.
Have you thought about how to enable a different topology? If all the nodes have to go through the single server, it becomes a bottleneck. It would be amazing to make a fully connected mesh where every node can send traffic directly to every other node over VPN. However, I haven't figured out how to do this yet. An ideas?
Hi!
I didn't have to re-setup my cluster as it's still working as intended. I can only assume the same steps would work today as they did last year.
I think you could work-around that SPoF by using another VPN server - one that supports MESH-like functionality. However, you'll most likely have lower baseline performance, as I doubt there's anything much faster than Wireguard.
FTR I'm using my setup for a non-prod cluster, so I'm not that worried about network performance.