Backup Strategy for Your Homelab: The 3-2-1 Rule in Practice

TL;DR Three copies, two kinds of media, one of them offsite. Restic for files, Velero for the cluster, a Backblaze bucket for the offsite copy, and a restore test on a schedule. Mine costs about €5 a month, and the restore test is the part I still have to make myself do.

Picture the disk in your homelab dying tonight. Not corrupting, not throwing SMART warnings for a week first, just gone. The git server with every repo you care about, the password vault, fifteen years of family photos, the home automation that runs the house. All of it on a drive that’s decided it’s done.

Can you say what happens next without your stomach dropping? If not, you don’t have backups. You have hope, and I ran on hope for longer than I’d like to admit.

The good news is that you don’t need anything clever. The 3-2-1 rule has been around for decades and it’s almost boringly simple. This post takes you from “I think I have something” to a setup you’ve tested. I’ll start with the version you can run today and layer the rest on top.

The 3-2-1 rule

Three copies, two different media, one offsite. That’s the whole rule. The rest of this post is filling in those three numbers with real tools.

Why three copies

• Copy 1: Your live data (original)
• Copy 2: Local backup (fast restore)
• Copy 3: Offsite backup (disaster recovery)

One copy is your data waiting to fail. Two copies feel safer, right up until one event takes out both. A fire does that. So does a ransomware payload that walks every mount it can reach. Three copies with real distance between them is where I’d start calling it resilience.

Why two media types

Different hardware fails in different ways, and you want your copies spread across those failure modes so one bad day can’t reach all of them.

• SSDs can fail silently through bit rot, with data quietly going bad before you notice
• HDDs die mechanically, often with enough warning to react if you’re watching
• RAID is not a backup. It protects against a drive dropping out, and does nothing about corruption, deletion, or a bad write that gets faithfully mirrored across every disk

Put your copies on an SSD, a spinning disk and a remote bucket, and whatever kills one rarely kills the others.

Why one offsite

Your house can burn down. A power surge can cook everything on the same circuit. Local copies are fast and convenient and worth nothing against anything that hits the whole building. The offsite copy is just the one that sits in a different building.

What to back up

Before you back up anything, sort your data into three buckets. Most people back up far too much, or they quietly skip the one thing they can’t rebuild. A few minutes of sorting fixes both.

Critical, backed up daily

Important, backed up weekly

Rebuildable, not backed up

• Base OS (reinstall from ISO)
• Downloaded packages (re-download)
• Cached data (regenerates)
• Temporary files

If you can get it back with a command or a download, leave it out. Backup space costs money and backup time costs time, and spending either on the base OS is a waste.

Backup tools

Right, that’s the theory. Here’s where it gets practical. I’ll start with the simplest tool that gives you a real second copy, then build up to the Kubernetes-aware stuff.

Restic for file-level backups

If you do nothing else after reading this, set up Restic. It’s what I use for file backups. It’s fast, it encrypts by default, it deduplicates so repeat backups stay small, and it talks to a local disk or an S3 bucket without you changing anything. This is the minimum viable backup, and you can have it running in five minutes.

# Initialize repository
restic init --repo /mnt/backup/restic

# Or with S3 backend
restic init --repo s3:s3.amazonaws.com/my-bucket

# Backup a directory
restic backup /home/user/documents

# Backup with exclusions
restic backup /data \
  --exclude="*.tmp" \
  --exclude=".cache" \
  --exclude="node_modules"

# List snapshots
restic snapshots

# Restore
restic restore latest --target /restore/location

Automated Restic backups

A backup you have to remember to run is one you’ll forget. Wrap the commands in a script, point it at your offsite bucket, and let it prune so the repo doesn’t grow forever.

#!/bin/bash
# /usr/local/bin/backup.sh

export RESTIC_REPOSITORY="s3:s3.eu-west-1.amazonaws.com/homelab-backups"
export RESTIC_PASSWORD_FILE="/etc/restic/password"
export AWS_ACCESS_KEY_ID="your-key"
export AWS_SECRET_ACCESS_KEY="your-secret"

# Backup
restic backup /data/important \
  --exclude-caches \
  --tag homelab \
  --tag daily

# Prune old snapshots (keep 7 daily, 4 weekly, 12 monthly)
restic forget \
  --keep-daily 7 \
  --keep-weekly 4 \
  --keep-monthly 12 \
  --prune

# Check repository integrity
restic check

That restic check at the end matters more than it looks. It reads the repository back and verifies it. Without it you’ve got a folder full of bytes and no idea whether they’re readable.

Cron job:

# /etc/cron.d/restic-backup
0 3 * * * root /usr/local/bin/backup.sh >> /var/log/restic-backup.log 2>&1

Velero for Kubernetes

Once you run a cluster, file-level backups stop being enough. You also want the cluster’s own state, the Deployments and ConfigMaps and PersistentVolumeClaims. Velero backs up Kubernetes resources and their persistent volumes together, so one restore brings back the workloads and their data.

# Install Velero with S3 backend
velero install \
  --provider aws \
  --plugins velero/velero-plugin-for-aws:v1.8.0 \
  --bucket velero-backups \
  --backup-location-config region=eu-west-1 \
  --secret-file ./credentials-velero \
  --use-volume-snapshots=true \
  --snapshot-location-config region=eu-west-1

Scheduled backups

apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: daily-backup
  namespace: velero
spec:
  schedule: "0 3 * * *"
  template:
    includedNamespaces:
      - production
      - gitlab
      - monitoring
    excludedResources:
      - events
      - pods
    ttl: 720h  # Keep for 30 days
    storageLocation: default
    volumeSnapshotLocations:
      - default

Restore from Velero

# List backups
velero backup get

# Describe a backup
velero backup describe daily-backup-20260518030000

# Restore entire backup
velero restore create --from-backup daily-backup-20260518030000

# Restore specific namespace
velero restore create --from-backup daily-backup-20260518030000 \
  --include-namespaces gitlab

Longhorn backups

If you run Longhorn for storage, you already have a backup engine under your volumes, and it talks straight to S3:

# Configure backup target
apiVersion: longhorn.io/v1beta1
kind: Setting
metadata:
  name: backup-target
  namespace: longhorn-system
value: "s3://longhorn-backups@eu-west-1/"
---
apiVersion: longhorn.io/v1beta1
kind: Setting
metadata:
  name: backup-target-credential-secret
  namespace: longhorn-system
value: "longhorn-s3-secret"

Schedule recurring backups:

apiVersion: longhorn.io/v1beta1
kind: RecurringJob
metadata:
  name: daily-backup
  namespace: longhorn-system
spec:
  cron: "0 3 * * *"
  task: backup
  groups:
    - default
  retain: 7
  concurrency: 2

Offsite options

The “1” in 3-2-1 is the part people skip, usually because it sounds like the expensive, complicated one. It isn’t. You’ve got three honest options. They trade cost against control, and how far down that list you go says a lot about how much sovereignty you want.

Cloud storage

The cheapest way to get offsite. Pick a provider, point Restic or Velero at it, done.

Want the offsite copy inside the EU? The Hetzner Storage Box is hard to beat on price. Backblaze is the easy default when you just want it working. Mine is at Backblaze, US-based, so I don’t get to be smug about the EU option.

Second location

Got a friend who also runs a homelab? Back each other up. They host an encrypted copy of your data, you host one of theirs, and neither of you can read what you’re storing.

This costs you some disk and some upload bandwidth, and your offsite copy never touches a cloud provider.

Self-hosted cloud

The full sovereignty version. Run your own S3-compatible storage at a second physical location and back up to that.

# MinIO at remote location
apiVersion: apps/v1
kind: Deployment
metadata:
  name: minio
spec:
  template:
    spec:
      containers:
        - name: minio
          image: minio/minio
          args:
            - server
            - /data
          env:
            - name: MINIO_ROOT_USER
              valueFrom:
                secretKeyRef:
                  name: minio-credentials
                  key: user
            - name: MINIO_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: minio-credentials
                  key: password

Reach it over Tailscale so the storage never has to sit on the public internet.

Database backups

Databases need their own treatment. Copy the files out from under a running database and you get a snapshot that may be mid-write, and a mid-write snapshot may not restore. Dump them properly instead.

PostgreSQL

#!/bin/bash
# Kubernetes PostgreSQL backup

NAMESPACE="gitlab"
POD=$(kubectl get pod -n $NAMESPACE -l app=postgresql -o jsonpath='{.items[0].metadata.name}')
DATE=$(date +%Y%m%d_%H%M%S)

# Dump all databases
kubectl exec -n $NAMESPACE $POD -- \
  pg_dumpall -U postgres | \
  gzip > /backup/postgres_${DATE}.sql.gz

# Upload to S3
restic backup /backup/postgres_${DATE}.sql.gz --tag postgres --tag daily

Vault backup

Vault is the one I lose sleep over, because losing it means losing access to everything else. Take a raft snapshot, encrypt it before it leaves the machine, and only then ship it off.

# Export Vault data (requires root token)
vault operator raft snapshot save /backup/vault_$(date +%Y%m%d).snap

# Encrypt and upload
gpg --encrypt --recipient backup@example.com /backup/vault_$(date +%Y%m%d).snap
restic backup /backup/vault_$(date +%Y%m%d).snap.gpg --tag vault

Testing restores

Here’s the part everyone nods along to and almost nobody does. I’ve nodded along too. A backup you haven’t tested is a guess. The first time you run a restore shouldn’t be the time you need it, with production down, and that’s when you find out the dumps have been empty for three months.

Monthly restore test

So test on a schedule. Restore into a throwaway namespace, check the thing comes up, tear it down.

#!/bin/bash
# test-restore.sh

# Create test namespace
kubectl create namespace restore-test

# Restore from Velero
velero restore create test-restore \
  --from-backup $(velero backup get -o json | jq -r '.items[0].metadata.name') \
  --include-namespaces gitlab \
  --namespace-mappings gitlab:restore-test

# Wait for restore
velero restore wait test-restore

# Verify pods are running
kubectl get pods -n restore-test

# Test application (example: GitLab)
kubectl port-forward -n restore-test svc/gitlab 8080:80 &
curl -s http://localhost:8080/health | grep "ok"

# Cleanup
kubectl delete namespace restore-test

Document recovery procedures

When something is on fire you don’t want to reverse-engineer your own restore process from memory. Write the runbook down for each critical system, with how long it took and when you last proved it works.

# GitLab Recovery Procedure

## Prerequisites
- Access to Velero backups
- Access to PostgreSQL backups
- GitLab Helm values

## Steps
1. Restore PostgreSQL from backup
2. Restore GitLab PVCs with Velero
3. Deploy GitLab with same Helm values
4. Verify user login works
5. Verify repositories are accessible

## Estimated Time: 45 minutes
## Last Tested: 2026-05-01

That “Last Tested” line keeps the test honest. If the date is a year old, nobody knows whether the procedure still works, and that includes you.

Monitoring backups

A backup job that silently stopped two weeks ago is worse than no backup, because you think you’re covered. Wire the jobs into your monitoring so a failure shouts at you.

Prometheus alerts

groups:
  - name: backup-alerts
    rules:
      - alert: BackupFailed
        expr: restic_backup_last_successful_timestamp < (time() - 86400)
        for: 1h
        labels:
          severity: critical
        annotations:
          summary: "Backup hasn't succeeded in 24 hours"

      - alert: BackupStorageLow
        expr: restic_repository_size_bytes / restic_repository_max_bytes > 0.9
        for: 1h
        labels:
          severity: warning
        annotations:
          summary: "Backup storage over 90% full"

Backup dashboard

Track in Grafana:

• Last successful backup time
• Backup duration trend
• Storage usage
• Restore test results

My backup setup

Here’s how it fits together in my own homelab, which is the layers above stacked into one flow. Local copies for fast restores, an offsite bucket for the disaster case, and Git for anything that’s already text.

Schedule

Costs

• Backblaze B2: ~€5/month for 200GB
• NAS storage: Already owned
• Total: ~€5/month for peace of mind

Five euros a month, and the NAS was already there. The photos are the reason I pay it.

Common mistakes

I’ve made most of these myself, or watched friends make them. They all sound reasonable until the day they cost you something.

“RAID is my backup”

RAID protects against a drive failing. It does nothing about:

• Accidental deletion
• Ransomware
• Software bugs corrupting data
• Fire/flood/theft

An rm -rf in the wrong directory propagates across every disk in the array instantly. RAID keeps the array up when a drive dies. Nothing on that list is a drive dying.

“I’ll restore when I need to”

If you’ve never restored, you don’t know whether your backups work, and the day you need them is the day you’ll find out. Test quarterly at the absolute minimum.

“I back up everything”

Backing up 10TB of films you can re-download burns money and time you could spend protecting the data you can’t replace. Sort first, then back up the irreplaceable stuff.

“My backup is in the same room”

One fire takes out the server and the backup drive next to it. Offsite is the part that makes the rest of the rule worth doing.

Why this matters

Data loss isn’t a maybe. Drives carry a 3-5% annual failure rate, people run the wrong command, software corrupts databases in ways that pass every health check, and houses do occasionally catch fire. Run a homelab long enough and you’ll meet at least one of these.

What changes the outcome is whether you did the boring work ahead of time. It’s the same thing I keep coming back to on this blog, the difference between a system you understand and control and one you’re hoping holds together. With a tested backup, losing a disk costs you an afternoon. Without one it costs you the photos.

So spend the five euros and the afternoon. Then put the restore test in your calendar, because that’s the step I keep having to push myself to do.

• Resilience
• Self-Hosting
• Server