After a few hours of work, I have high-availability storage on my #k3s #Kubernetes cluster.

Running on bare Ubuntu VMs, each of the three servers has 48GB of RAM and 720GB of SSD storage. The provider I'm using doesn't supply extra SAN storage, so the on-VM storage is all I have, and any redundancy I have to handle myself.

Enter Longhorn. Longhorn is a FOSS project from Rancher that allows you to use local storage inside your Kubernetes cluster, and keeps replicas available on other nodes in case one of your servers is unavailable. The system is trivial to set up and highly efficient, and acts as a StorageClass that you can use when requesting storage for a pod. It can also schedule snapshots and backups to an offsite S3 instance for additional safety. It even has experimental support for shared volumes via NFS!

For object storage I've configured a modern Minio cluster. Minio is an FOSS S3-compatible server that also uses local storage, keeping multiple instances around for high availability. It's also quite easy to configure and use, with an incredibly rich feature set and a lovely UI. It doesn't have its own backups, but it's easy to replicate with a simple cron job.

I'm slowly moving workloads over to the new pod, and will migrate the Diaspora pod in a few days (expect an hour or so of downtime during the migration). The new cluster is more secure, more stable, and is much less likely to go down than the old one was.