Backup Strategy for Your Homelab: The 3-2-1 Rule in Practice

Picture your homelab disk dying tonight. Not corrupting, not throwing SMART warnings for a week first, just gone. The GitLab that holds every repo you care about, the password vault, fifteen years of family photos, the home automation that runs your house. All of it on a drive that has decided it’s done.

Can you answer what happens next without your stomach dropping? If not, you don’t have backups. You have hope, and hope is not a strategy you want to discover the limits of at 2am.

Here’s the good news. You don’t need anything clever. The 3-2-1 rule has been around for decades, it’s almost boringly simple, and the whole point of this post is to take you from “I think I have something” to a setup you’ve actually tested. We’ll start with the simplest version you can run today and layer on the rest from there.

The 3-2-1 Rule Explained

Three copies, two different media, one offsite. That’s the entire rule. Everything else in this post is just filling in those three numbers with real tools.

flowchart TD
    subgraph rule["3-2-1 Backup Rule"]
        Data["Original Data"]

        subgraph three["3 Copies"]
            C1["Copy 1<br/>(Original)"]
            C2["Copy 2<br/>(Local Backup)"]
            C3["Copy 3<br/>(Offsite)"]
        end

        subgraph two["2 Media Types"]
            M1["NVMe/SSD"]
            M2["HDD/NAS"]
        end

        subgraph one["1 Offsite"]
            Off["Cloud/Remote"]
        end
    end

    Data --> C1
    Data --> C2
    Data --> C3
    C1 --> M1
    C2 --> M2
    C3 --> Off

Why Three Copies?

• Copy 1: Your live data (original)
• Copy 2: Local backup (fast restore)
• Copy 3: Offsite backup (disaster recovery)

One copy is just your data sitting there waiting to fail. Two copies feel safer right up until the same event takes out both at once, and plenty of events do exactly that: a fire, a flood, a ransomware payload that walks every mount it can reach. Three copies with real separation between them is the first point where you’ve got actual resilience instead of a comforting feeling.

Why Two Media Types?

Different hardware fails in different ways, and you want your copies spread across those failure modes so one bad day can’t take all of them.

• SSDs can fail silently through bit rot, with data quietly going bad before you notice
• HDDs die mechanically, often with enough warning to react if you’re watching
• RAID is not a backup. It protects against a drive dropping out, and does nothing about corruption, deletion, or a bad write that gets faithfully mirrored across every disk

Put your copies on an SSD and a spinning disk and a remote bucket, and the scenario that kills one is rarely the scenario that kills the others.

Why One Offsite?

Your house can burn down. Your street can flood. A power surge can cook everything plugged into the same circuit. Local copies are fast and convenient and completely worthless against anything that hits the whole building at once. Offsite is the copy that survives when the room your server lives in does not.

What to Back Up

Before you back up anything, sort your data into three buckets. Most people either back up far too much or quietly skip the one thing they can’t rebuild, and a few minutes of sorting fixes both.

Critical (Daily Backup)

Important (Weekly Backup)

Rebuildable (Don’t Backup)

• Base OS (reinstall from ISO)
• Downloaded packages (re-download)
• Cached data (regenerates)
• Temporary files

If you can recreate it from a command or a download, leave it out. Backup space and backup time both cost something, and spending either on data you can summon back at will is wasted.

Backup Tools

Right, that’s the theory. Here’s where it gets practical. I’ll start with the simplest tool that gets you a real second copy, then build up to the Kubernetes-aware stuff.

Restic: File-Level Backups

If you do nothing else after reading this, set up Restic. It’s my go-to for file backups: fast, encrypted by default, deduplicated so repeat backups stay small, and happy to talk to a local disk or an S3 bucket without changing your workflow. This is the minimum viable version of a backup strategy, and you can have it running in five minutes.

# Initialize repository
restic init --repo /mnt/backup/restic

# Or with S3 backend
restic init --repo s3:s3.amazonaws.com/my-bucket

# Backup a directory
restic backup /home/user/documents

# Backup with exclusions
restic backup /data \
  --exclude="*.tmp" \
  --exclude=".cache" \
  --exclude="node_modules"

# List snapshots
restic snapshots

# Restore
restic restore latest --target /restore/location

Automated Restic Backups

A backup you have to remember to run is a backup that won’t happen. Wrap the commands in a script, point it at your offsite bucket, and let it handle pruning so the repo doesn’t grow forever.

#!/bin/bash
# /usr/local/bin/backup.sh

export RESTIC_REPOSITORY="s3:s3.eu-west-1.amazonaws.com/homelab-backups"
export RESTIC_PASSWORD_FILE="/etc/restic/password"
export AWS_ACCESS_KEY_ID="your-key"
export AWS_SECRET_ACCESS_KEY="your-secret"

# Backup
restic backup /data/important \
  --exclude-caches \
  --tag homelab \
  --tag daily

# Prune old snapshots (keep 7 daily, 4 weekly, 12 monthly)
restic forget \
  --keep-daily 7 \
  --keep-weekly 4 \
  --keep-monthly 12 \
  --prune

# Check repository integrity
restic check

That restic check at the end matters more than it looks. It verifies the repository can actually be read back, which is the difference between a backup and a folder full of bytes you’re hoping are intact.

Cron job:

# /etc/cron.d/restic-backup
0 3 * * * root /usr/local/bin/backup.sh >> /var/log/restic-backup.log 2>&1

Velero: Kubernetes Backups

Once you’re running a cluster, file-level backups stop being enough. You also want the cluster’s own state: the Deployments, the ConfigMaps, the PersistentVolumeClaims, the shape of the thing. Velero backs up Kubernetes resources and their persistent volumes together, so a restore brings back the workloads and their data in one move.

# Install Velero with S3 backend
velero install \
  --provider aws \
  --plugins velero/velero-plugin-for-aws:v1.8.0 \
  --bucket velero-backups \
  --backup-location-config region=eu-west-1 \
  --secret-file ./credentials-velero \
  --use-volume-snapshots=true \
  --snapshot-location-config region=eu-west-1

Scheduled Backups

apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: daily-backup
  namespace: velero
spec:
  schedule: "0 3 * * *"
  template:
    includedNamespaces:
      - production
      - gitlab
      - monitoring
    excludedResources:
      - events
      - pods
    ttl: 720h  # Keep for 30 days
    storageLocation: default
    volumeSnapshotLocations:
      - default

Restore from Velero

# List backups
velero backup get

# Describe a backup
velero backup describe daily-backup-20260518030000

# Restore entire backup
velero restore create --from-backup daily-backup-20260518030000

# Restore specific namespace
velero restore create --from-backup daily-backup-20260518030000 \
  --include-namespaces gitlab

Longhorn Backups

If you run Longhorn for storage, you’ve already got a backup engine sitting under your volumes, and it talks straight to S3:

# Configure backup target
apiVersion: longhorn.io/v1beta1
kind: Setting
metadata:
  name: backup-target
  namespace: longhorn-system
value: "s3://longhorn-backups@eu-west-1/"
---
apiVersion: longhorn.io/v1beta1
kind: Setting
metadata:
  name: backup-target-credential-secret
  namespace: longhorn-system
value: "longhorn-s3-secret"

Schedule recurring backups:

apiVersion: longhorn.io/v1beta1
kind: RecurringJob
metadata:
  name: daily-backup
  namespace: longhorn-system
spec:
  cron: "0 3 * * *"
  task: backup
  groups:
    - default
  retain: 7
  concurrency: 2

Offsite Options

The “1” in 3-2-1 is the part people skip, usually because it feels like the expensive, complicated one. It isn’t. You’ve got three honest options here, and they trade off cost against control in ways that map neatly onto how much you care about sovereignty.

Cloud Storage

The cheapest way to get offsite. Pick a provider, point Restic or Velero at it, done.

If you’d rather your offsite copy live inside the EU, the Hetzner Storage Box is hard to beat on price, and Backblaze is the easy default when you just want it working.

Second Location

Got a friend or family member who also runs a homelab? Back each other up. They host an encrypted copy of your data, you host an encrypted copy of theirs, and nobody can read what they’re storing.

flowchart LR
    subgraph your["Your Home"]
        YourData["Your Data"]
        YourBackup["Their Backup<br/>(encrypted)"]
    end

    subgraph their["Their Home"]
        TheirData["Their Data"]
        TheirBackup["Your Backup<br/>(encrypted)"]
    end

    YourData -->|Encrypted| TheirBackup
    TheirData -->|Encrypted| YourBackup

This costs you nothing but some disk and a bit of upload bandwidth, and it keeps your offsite copy entirely out of any cloud provider’s hands.

Self-Hosted Cloud

The full sovereignty version: run your own S3-compatible storage at a second physical location and back up to that.

# MinIO at remote location
apiVersion: apps/v1
kind: Deployment
metadata:
  name: minio
spec:
  template:
    spec:
      containers:
        - name: minio
          image: minio/minio
          args:
            - server
            - /data
          env:
            - name: MINIO_ROOT_USER
              valueFrom:
                secretKeyRef:
                  name: minio-credentials
                  key: user
            - name: MINIO_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: minio-credentials
                  key: password

Reach it over Tailscale so the storage never has to sit on the public internet.

Database Backups

Databases need their own treatment, because copying the files out from under a running database gives you a snapshot that may be mid-write and unrestorable. Dump them properly instead.

PostgreSQL

#!/bin/bash
# Kubernetes PostgreSQL backup

NAMESPACE="gitlab"
POD=$(kubectl get pod -n $NAMESPACE -l app=postgresql -o jsonpath='{.items[0].metadata.name}')
DATE=$(date +%Y%m%d_%H%M%S)

# Dump all databases
kubectl exec -n $NAMESPACE $POD -- \
  pg_dumpall -U postgres | \
  gzip > /backup/postgres_${DATE}.sql.gz

# Upload to S3
restic backup /backup/postgres_${DATE}.sql.gz --tag postgres --tag daily

Vault Backup

Vault is the one I lose sleep over, because losing it means losing access to everything else. Take a raft snapshot, encrypt it before it leaves the machine, and only then ship it off.

# Export Vault data (requires root token)
vault operator raft snapshot save /backup/vault_$(date +%Y%m%d).snap

# Encrypt and upload
gpg --encrypt --recipient backup@example.com /backup/vault_$(date +%Y%m%d).snap
restic backup /backup/vault_$(date +%Y%m%d).snap.gpg --tag vault

Testing Restores

Here’s the part everyone nods along to and almost nobody does. A backup you haven’t tested is not a backup. It’s a guess. The first time you run a restore should never be the time you actually need it, when you’re stressed and the production system is down and you’re finding out your dumps were empty for the last three months.

Monthly Restore Test

So test it on a schedule. Restore into a throwaway namespace, check the thing actually comes up, tear it down.

#!/bin/bash
# test-restore.sh

# Create test namespace
kubectl create namespace restore-test

# Restore from Velero
velero restore create test-restore \
  --from-backup $(velero backup get -o json | jq -r '.items[0].metadata.name') \
  --include-namespaces gitlab \
  --namespace-mappings gitlab:restore-test

# Wait for restore
velero restore wait test-restore

# Verify pods are running
kubectl get pods -n restore-test

# Test application (example: GitLab)
kubectl port-forward -n restore-test svc/gitlab 8080:80 &
curl -s http://localhost:8080/health | grep "ok"

# Cleanup
kubectl delete namespace restore-test

Document Recovery Procedures

When something is actually on fire, you don’t want to be reverse-engineering your own restore process from memory. Write the runbook down for each critical system, including how long it took and when you last proved it works.

# GitLab Recovery Procedure

## Prerequisites
- Access to Velero backups
- Access to PostgreSQL backups
- GitLab Helm values

## Steps
1. Restore PostgreSQL from backup
2. Restore GitLab PVCs with Velero
3. Deploy GitLab with same Helm values
4. Verify user login works
5. Verify repositories are accessible

## Estimated Time: 45 minutes
## Last Tested: 2026-05-01

That “Last Tested” line is a quiet pressure to keep the test honest. If the date is a year old, you don’t have a backup strategy, you have a backup wish.

Monitoring Backups

A backup job that silently stopped running two weeks ago is worse than no backup, because you think you’re covered. Wire the jobs into your monitoring so a failure shouts at you instead of waiting to surprise you.

Prometheus Alerts

groups:
  - name: backup-alerts
    rules:
      - alert: BackupFailed
        expr: restic_backup_last_successful_timestamp < (time() - 86400)
        for: 1h
        labels:
          severity: critical
        annotations:
          summary: "Backup hasn't succeeded in 24 hours"

      - alert: BackupStorageLow
        expr: restic_repository_size_bytes / restic_repository_max_bytes > 0.9
        for: 1h
        labels:
          severity: warning
        annotations:
          summary: "Backup storage over 90% full"

Backup Dashboard

Track in Grafana:

• Last successful backup time
• Backup duration trend
• Storage usage
• Restore test results

My Backup Setup

Here’s how all of this fits together in my own homelab, which is really just the layers above stacked into one flow. Local copies for fast restores, an offsite bucket for the disaster case, and Git for anything that’s already text.

flowchart TD
    subgraph homelab["Homelab (K3s)"]
        PV["Persistent Volumes"]
        DB["Databases"]
        Config["Configs (Git)"]
    end

    subgraph local["Local Backup (NAS)"]
        Longhorn["Longhorn Snapshots"]
        Restic1["Restic Repository"]
    end

    subgraph offsite["Offsite (Backblaze B2)"]
        Velero["Velero Backups"]
        Restic2["Restic Offsite"]
    end

    PV --> Longhorn
    PV --> Velero
    DB --> Restic1
    Restic1 --> Restic2
    Config --> Git["GitLab (self-hosted)"]
    Git --> GitMirror["GitHub Mirror"]

Schedule

Costs

• Backblaze B2: ~€5/month for 200GB
• NAS storage: Already owned
• Total: ~€5/month for peace of mind

Five euros a month. That’s the whole bill for not lying awake wondering whether the photos are gone.

Common Mistakes

I’ve made most of these myself, or watched friends make them. They all sound reasonable right up to the moment they cost you everything.

“RAID is my backup”

RAID protects against a drive failing. It does nothing about:

• Accidental deletion
• Ransomware
• Software bugs corrupting data
• Fire/flood/theft

A rm -rf in the wrong directory propagates across every disk in the array instantly. RAID is uptime, not safety.

“I’ll restore when I need to”

If you’ve never restored, you don’t know whether your backups work, and the failure case is exactly when you find out they don’t. Test quarterly at the absolute minimum.

“I backup everything”

Backing up 10TB of films you can re-download burns money and time you could spend protecting the data you can’t replace. Sort first, then back up the irreplaceable stuff.

“My backup is in the same room”

One fire takes out the server and the backup drive sitting next to it. Offsite is the part that makes the whole rule worth doing.

Why This Matters

Data loss isn’t a maybe. Drives carry a 3-5% annual failure rate, humans run the wrong command, software corrupts databases in ways that pass every health check, and houses occasionally catch fire. Run a homelab long enough and you’ll meet at least one of these.

What changes the outcome is whether you did the boring work ahead of time. The same gap I keep coming back to on this blog shows up here too: the difference between systems you actually understand and control, and systems you’re just hoping hold together. A tested backup strategy is what turns “I lost everything” into “I restored from last night and lost an afternoon.”

Your homelab holds the things you’d most hate to lose. Spend the five euros and the afternoon, and protect them properly.

The best time to set up backups was before you needed them. The second best time is now.