K3s Cluster Setup on Refurbished Mini-PCs

Three mini-PCs sit on a shelf in my house. Together they run my GitLab, my monitoring stack, home automation, file sync, password manager, and a pile of other things I refuse to hand to someone else’s computer. The whole setup cost less than a single month of the equivalent managed Kubernetes bill, and I understand every layer of it because I built it myself.

You can have the same thing. No cloud account, no rack, no five-figure budget. You need three second-hand machines and an afternoon where nobody bothers you. This is the cluster I keep coming back to when I talk about sovereign infrastructure, and here is exactly how it goes together.

Start with the smallest thing that works

Before any of the hardware talk, here is the part that surprises people: a working Kubernetes cluster is one command.

curl -sfL https://get.k3s.io | sh -

That gives you a single-node K3s install. Real Kubernetes API, real kubectl, real workloads. If all you want is to poke at manifests on a spare laptop, you can stop reading right here and you already have something useful.

K3s is the reason this is so painless. It is the full Kubernetes API packed into a single ~70MB binary, with the bits most homelabs never use stripped out and sensible defaults wired in for the rest. It runs on a Raspberry Pi, and it runs beautifully on a mini-PC. People sometimes call it “Kubernetes lite” and that undersells it. The workloads you deploy are identical to what you would run on a big managed cluster. You just skip the enterprise overhead you were never going to touch.

The rest of this post is about turning that one command into something you would trust with data you care about: three nodes, embedded etcd, a virtual IP that survives a node dying, and the storage and ingress to actually host things.

The hardware: why refurbished mini-PCs

You can build this on anything, but I keep recommending the same thing because it keeps being right. Refurbished business mini-PCs hit a sweet spot that nothing else matches for the money.

What	Why
HP EliteDesk 800 G3	Reliable, cheap, available everywhere
Lenovo ThinkCentre M720q	Great thermals, quiet
Dell OptiPlex 3060 Micro	Good expansion options

Aim for roughly this per node:

CPU: Intel i5 6th-8th gen (4-6 cores)
RAM: 16-32GB (DDR4, upgradeable)
Storage: 256GB+ NVMe/SSD
Network: Gigabit Ethernet

Expect to pay around €100-200 per node, so €300-600 for three.

People always ask about a Raspberry Pi cluster instead, and Pis do work. I ran one for a while. But you fight the platform: ARM images that don’t exist yet for the thing you want, SD cards that corrupt themselves when a node loses power, 8GB of RAM as a hard ceiling on a Pi 4, and USB-attached storage that flakes out at the worst moment. A refurbished mini-PC hands you x86, an NVMe slot, proper RAM, and a machine that was built to run business workloads for years without complaint. The reliability difference is not subtle.

My own cluster is three of the same box, which makes life easier:

Node 1: HP EliteDesk 800 G3 Mini
        i5-7500T, 32GB RAM, 512GB NVMe
        Role: Control plane + Worker

Node 2: HP EliteDesk 800 G3 Mini
        i5-7500T, 32GB RAM, 512GB NVMe
        Role: Control plane + Worker

Node 3: HP EliteDesk 800 G3 Mini
        i5-7500T, 16GB RAM, 256GB NVMe
        Role: Control plane + Worker

That is about €450 for 12 cores, 80GB of RAM, and 1.2TB of storage that lives on my shelf and answers to nobody.

Getting the nodes ready

Two things to sort before K3s: addressing and the OS.

Predictable addresses

Every node needs an IP that doesn’t move. Either set static IPs or, easier, hand out DHCP reservations from your router.

192.168.1.10  k3s-node-1
192.168.1.11  k3s-node-2
192.168.1.12  k3s-node-3
192.168.1.100 k3s-api      # VIP for API server

That .100 is a virtual IP we will set up later so the API has one stable address even when a node dies. Give the nodes DNS names too, in your local resolver or /etc/hosts if you’re in a hurry:

192.168.1.10  k3s-node-1.home.lan
192.168.1.11  k3s-node-2.home.lan
192.168.1.12  k3s-node-3.home.lan
192.168.1.100 k3s.home.lan

The OS

For getting started, Ubuntu Server 22.04 LTS minimal is the path of least resistance. Install it, update, and disable swap (Kubernetes wants swap off):

# After install, update
sudo apt update && sudo apt upgrade -y

# Install useful tools
sudo apt install -y curl wget vim htop

# Disable swap (required for Kubernetes)
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

If you want the immutable, locked-down version of this, Talos Linux is where I’d send you next. It’s a better foundation and a worse starting point, so I’d save it for round two. Ubuntu gets you a cluster today.

Layer one: a real HA cluster

This is the step that takes you from “Kubernetes on a laptop” to “Kubernetes I’d trust.” The key flag is --cluster-init, which turns on embedded etcd so the control plane survives a node failure.

On node 1:

# On node 1
curl -sfL https://get.k3s.io | sh -s - server \
  --cluster-init \
  --tls-san k3s.home.lan \
  --tls-san 192.168.1.100 \
  --disable traefik \
  --disable servicelb

What those flags are doing:

--cluster-init: enable embedded etcd for HA
--tls-san: add SANs so the API cert is valid for the names and VIP we’ll use
--disable traefik: skip the bundled ingress, we’ll bring our own
--disable servicelb: skip the bundled load balancer, MetalLB does this better

Grab the join token:

sudo cat /var/lib/rancher/k3s/server/node-token

Then bring up nodes 2 and 3, pointing them at node 1 and reusing the same flags:

# On node 2 and 3
curl -sfL https://get.k3s.io | sh -s - server \
  --server https://192.168.1.10:6443 \
  --token <TOKEN_FROM_NODE_1> \
  --tls-san k3s.home.lan \
  --tls-san 192.168.1.100 \
  --disable traefik \
  --disable servicelb

Pull the kubeconfig, point it at your workstation, and check that all three nodes joined:

# Get kubeconfig
sudo cat /etc/rancher/k3s/k3s.yaml

# Copy to your workstation, update server URL
export KUBECONFIG=~/.kube/k3s-config

# Check nodes
kubectl get nodes
NAME         STATUS   ROLES                       AGE   VERSION
k3s-node-1   Ready    control-plane,etcd,master   5m    v1.28.5+k3s1
k3s-node-2   Ready    control-plane,etcd,master   3m    v1.28.5+k3s1
k3s-node-3   Ready    control-plane,etcd,master   2m    v1.28.5+k3s1

Three control-plane nodes, each carrying an etcd member. Now any one of them can fall over and the cluster keeps running. That is the whole point of doing this instead of running a single node.

One address that survives a dead node

There is a gap left over. Your kubeconfig points at one node’s IP, and if that node is the one that dies, your kubectl stops working even though the cluster is fine. You need a single API address that floats across whichever control-plane nodes are alive.

My pick is kube-vip. It runs as a DaemonSet on the control-plane nodes and answers for the VIP (192.168.1.100) using ARP, moving it to a healthy node automatically.

# On each control plane node
kubectl apply -f https://kube-vip.io/manifests/rbac.yaml

# Create kube-vip manifest
cat <<EOF | sudo tee /var/lib/rancher/k3s/server/manifests/kube-vip.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-vip
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: kube-vip
  template:
    metadata:
      labels:
        app: kube-vip
    spec:
      hostNetwork: true
      tolerations:
        - key: node-role.kubernetes.io/control-plane
          operator: Exists
          effect: NoSchedule
      containers:
        - name: kube-vip
          image: ghcr.io/kube-vip/kube-vip:latest
          args:
            - manager
          env:
            - name: vip_interface
              value: eth0
            - name: vip_address
              value: 192.168.1.100
            - name: vip_arp
              value: "true"
            - name: lb_enable
              value: "true"
EOF

If you’d rather keep the VIP logic off the cluster, and your router runs HAProxy, you can balance the API there instead:

frontend k3s-api
    bind *:6443
    default_backend k3s-servers

backend k3s-servers
    balance roundrobin
    server node1 192.168.1.10:6443 check
    server node2 192.168.1.11:6443 check
    server node3 192.168.1.12:6443 check

Both work. kube-vip keeps everything inside the cluster, which I prefer because there’s one less box in the failure path. HAProxy on the router is simpler to reason about if you already trust your router. Pick the one that matches what you already run.

Layer two: making it useful

A bare cluster doesn’t host anything yet. Three pieces turn it into a platform you can actually deploy to.

LoadBalancer services with MetalLB

We disabled the bundled servicelb, so LoadBalancer services have nothing to answer them. MetalLB fills that gap by handing out IPs from a pool on your LAN.

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.3/config/manifests/metallb-native.yaml

Give it a range to hand out, well clear of your node and VIP addresses:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default
  namespace: metallb-system
spec:
  addresses:
    - 192.168.1.200-192.168.1.250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default
  namespace: metallb-system

Ingress

We dropped the bundled Traefik so we could install it on our own terms. Here it is via Helm, taking a LoadBalancer IP from the MetalLB pool we just defined:

# Traefik (via Helm)
helm repo add traefik https://traefik.github.io/charts
helm install traefik traefik/traefik \
  --namespace traefik \
  --create-namespace \
  --set service.type=LoadBalancer

Storage with Longhorn

Pods that need to keep data need real persistent volumes. Longhorn gives you replicated block storage across the three nodes, so a volume survives a node going down:

helm repo add longhorn https://charts.longhorn.io
helm install longhorn longhorn/longhorn \
  --namespace longhorn-system \
  --create-namespace

If you’re weighing storage options, I put Longhorn head to head with the heavier alternative in Longhorn vs Rook-Ceph.

Talking to the cluster from your desk

Last bit of setup: get a kubeconfig onto your workstation so you stop SSHing into node 1 for everything. The one K3s generates points at 127.0.0.1, so swap that for your VIP DNS name and the kubeconfig keeps working no matter which node answers.

# On node 1
sudo cat /etc/rancher/k3s/k3s.yaml

# Save locally, edit server URL
# Change: server: https://127.0.0.1:6443
# To:     server: https://k3s.home.lan:6443

# Set permissions
chmod 600 ~/.kube/k3s-config
export KUBECONFIG=~/.kube/k3s-config

Keeping it alive

A homelab cluster you never maintain becomes a liability the day it breaks. A few habits keep it boring, which is exactly what you want from infrastructure.

Upgrades

Upgrade one node at a time by rerunning the install with the same flags, or let the system-upgrade-controller orchestrate it for you:

# On each node, one at a time
curl -sfL https://get.k3s.io | sh -s - server \
  # same flags as initial install

kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/system-upgrade-controller.yaml

Backups

etcd is where your whole cluster state lives. Snapshot it. K3s can do this on a schedule, and you should let it:

# Manual snapshot
k3s etcd-snapshot save --name manual-backup

# Scheduled backups (add to k3s config)
# /etc/rancher/k3s/config.yaml
etcd-snapshot-schedule-cron: "0 */6 * * *"
etcd-snapshot-retention: 5

Pulling a node for maintenance

Need to open a box up or reboot it? Drain it first so workloads reschedule cleanly, then bring it back:

# Drain node before maintenance
kubectl drain k3s-node-2 --ignore-daemonsets --delete-emptydir-data

# Do maintenance...

# Uncordon when done
kubectl uncordon k3s-node-2

When something’s broken

A few failures show up often enough to memorise. If kubectl can’t reach the server, check that K3s is actually running, that the firewall isn’t eating port 6443, and that your kubeconfig points where you think it does:

sudo systemctl status k3s
sudo ufw status

A node stuck NotReady usually means the kubelet, the container runtime, or the disk is unhappy:

sudo journalctl -u k3s -f
sudo crictl ps
df -h

And if etcd itself gets corrupted, you’ll be reaching for snapshots. The --cluster-reset path exists, but treat it as a last resort because it tears the cluster down to a single member:

# Check etcd health
sudo k3s etcd-snapshot list

# Force new cluster if etcd corrupted (DANGEROUS)
# sudo k3s server --cluster-reset

The full picture: what it’s actually for

Remember the one-line install at the top? You’ve come a long way from there. You now have three nodes sharing an HA control plane, a VIP that shrugs off a dead node, load balancing, ingress, replicated storage, and a backup schedule. That is a platform.

Here is what I run on mine:

GitLab for code hosting
Prometheus and Thanos for monitoring
ArgoCD for GitOps
Home Assistant for automation
Nextcloud for file sync
Vaultwarden for passwords

All of it self-hosted, all of it answering only to me.

The money case is almost unfair. Running the equivalent in the cloud costs me €150-300 a month. The shelf costs maybe €10-20 of electricity:

	Cloud (3 nodes)	Homelab (3 nodes)
Monthly	€150-300	€10-20 (electricity)
Annual	€1800-3600	€120-240
3 year	€5400-10800	€360-720 + €450 hardware

The hardware pays for itself in three to six months. After that it’s basically free compute that I own outright.

But the money was never the real reason for me. Running your own cluster forces you to learn how Kubernetes actually behaves, not just which command makes the dashboard go green. You find out what really happens when a node dies, because at some point a node will die and you’ll be the one watching the VIP fail over. You learn capacity planning with your own RAM on the line. That understanding is worth more than the euros saved, and it’s the kind of understanding you can’t buy from a control panel.

And you get sovereignty over the whole stack. Your data, your rules, your responsibility. The cloud is fine for work that belongs to someone else. The things that matter to me live on a shelf in my house, where I can reach out and touch the hardware, and where the best way to understand it is to break it on purpose and watch how it heals.

Start with the smallest thing that works#

The hardware: why refurbished mini-PCs#

Getting the nodes ready#

Predictable addresses#

The OS#

Layer one: a real HA cluster#

One address that survives a dead node#

Layer two: making it useful#

LoadBalancer services with MetalLB#

Ingress#

Storage with Longhorn#

Talking to the cluster from your desk#

Keeping it alive#

Upgrades#

Backups#

Pulling a node for maintenance#

When something’s broken#

The full picture: what it’s actually for#