NixOS vs Talos for Kubernetes Nodes: Two Flavors of Immutable Infrastructure

TL;DR Talos and NixOS are both declarative and both can be immutable, and that’s about where the resemblance stops. Talos runs Kubernetes and nothing else, with no shell to drift in. NixOS is a full Linux where the whole machine is one Nix config. I run Talos on my cluster nodes and NixOS on the machines around them, and this post is how I got to that split.

I’ve written about Talos Linux as the immutable Kubernetes OS, and I’ve compared Arch vs NixOS for workstations. After both, the same question kept arriving in my inbox. Why not NixOS on the Kubernetes nodes too?

Fair question. On paper the two look like siblings. NixOS and Talos are both declarative. Both can be immutable. Both put the configuration of a machine under version control. So why would you pick one over the other for a cluster?

My bias first. I run a homelab where I want to understand and own every layer, and I’ve got a low tolerance for drift. I’ve run both of these in production, and I like Nix as a language, which not everyone does. So I come to this with a preference already formed. I’ve tried to give both a fair run below anyway.

I’ll compare them on the things that moved my own decision: how you configure them, what an attacker finds on the box, what else the node can run, how upgrades and rollbacks go, how Kubernetes gets onto the node, and what debugging looks like when something breaks. The feature lists are on both project homepages already.

Where the two disagree on purpose

Talos Linux runs Kubernetes and nothing else. Every design decision in it bends toward that one job.

NixOS is a general-purpose Linux that happens to be good at Kubernetes. It can do anything a traditional distribution does, except that the whole machine is described in one declarative config.

Every trade-off below comes from that one difference in intent.

Configuration

Talos: YAML

Talos configuration is plain YAML:

machine:
  type: controlplane
  network:
    hostname: node-01
    interfaces:
      - interface: eth0
        dhcp: true
  install:
    disk: /dev/sda
cluster:
  clusterName: production
  controlPlane:
    endpoint: https://k8s.example.com:6443

You apply it with talosctl apply-config and the node converges to that state. There’s nothing else to learn. The schema is documented and the fields do what they say.

NixOS: the Nix language

NixOS configuration is written in Nix, which is a functional language with ideas of its own:

{ config, pkgs, ... }:

{
  networking.hostName = "node-01";

  services.k3s = {
    enable = true;
    role = "server";
    extraFlags = toString [
      "--cluster-init"
      "--disable traefik"
    ];
  };

  # You can do anything here
  services.prometheus.exporters.node.enable = true;
  virtualisation.docker.enable = true;

  system.stateVersion = "24.05";
}

You apply it with nixos-rebuild switch and the system converges to what you declared. Same idea, different vocabulary.

Talos’s YAML is limited and you’ll learn it in an afternoon. Nix is a lot more powerful, and it took me weeks before it stopped feeling alien. I say that as someone who ended up liking it.

So the question is scope. If the node only needs Kubernetes, the YAML is enough and the Nix learning curve buys you nothing. If the node needs Kubernetes plus a pile of other things, that same curve is what pays for them.

Security

Talos: nothing to find

Talos gets its security by deleting things:

• No SSH daemon
• No shell
• No package manager
• Read-only root filesystem
• Minimal userspace (just containerd , kubelet, and essentials)

An attacker who compromises a container and escapes to the host finds almost nothing there. No tools to pivot with, no obvious way to persist, no package manager to pull malware down with. That’s blast radius reduction as an OS design, and I care about it more than I care about most features.

NixOS: hardened when you say so

NixOS can be hardened a long way, but every step is opt-in:

{
  # Disable SSH password auth
  services.openssh = {
    enable = true;
    settings.PasswordAuthentication = false;
  };

  # Firewall
  networking.firewall = {
    enable = true;
    allowedTCPPorts = [ 6443 ];
  };

  # Read-only store (always true)
  # But /etc, /var, /home are still mutable
}

The /nix/store is always read-only, which is good. /etc, /var and /home stay mutable unless you do something about them. You can, and it’s a thing you have to remember to do.

So Talos is hardened on delivery and NixOS is hardened after you’ve put the work in. For a node whose only job is Kubernetes, the paranoid defaults are the ones I want, and I’d rather not even have the option to skip them.

What else the node can run

This is where the two pull apart hardest.

NixOS: anything

A NixOS node can run Kubernetes and, next to it, anything else you’d put on a Linux box. The Prometheus node exporter, a log shipper or monitoring agents as systemd services instead of a pod. A bare-metal database or a legacy app that has no business in a container. Any of the 80,000+ packages in nixpkgs. The debugging tools you want on the node itself, one config line away.

Here’s a PostgreSQL next to Kubernetes, for testing:

{
  services.k3s.enable = true;

  services.postgresql = {
    enable = true;
    package = pkgs.postgresql_15;
  };
}

Talos: nothing else, on purpose

Talos’s restrictions are the product. You can’t make an undocumented change, because there’s no shell to make it in. Every change goes through the config, then the API, then apply. The node is its config and there’s no second place to look. And what doesn’t exist on the box can’t be exploited.

If your nodes only ever run Kubernetes workloads, those walls stop you making mistakes. If your nodes need to do more, the same walls are a cage, and NixOS is the way out of it.

Upgrades and rollback

Talos: two partitions

talosctl upgrade --nodes 192.168.1.10 \
  --image ghcr.io/siderolabs/installer:v1.7.0

Talos writes the new OS to an alternate partition and reboots into it. If the new image doesn’t boot, it falls back to the previous one by itself. The upgrade either lands completely or it doesn’t land, and you’re not watching a console either way.

NixOS: generations

nixos-rebuild switch --flake .#node-01

Every configuration change gives you a new generation, and each generation is bootable from GRUB. Going back is picking an earlier one:

nixos-rebuild switch --rollback
# Or select from boot menu

NixOS keeps several generations by default, so you can drop back to the configuration from weeks ago.

Both rollback stories are good, which is rarer than it sounds. Talos does it for you when a boot fails. NixOS keeps as many generations as you like and lets you pick.

Getting Kubernetes onto the node

Talos: it’s already there

Talos ships with everything a cluster needs. The Kubernetes components are built in, the CNI config lives in the machine config, etcd is managed for you, certificates rotate automatically, and an upgrade drains the node, upgrades it and uncordons it again. You don’t install Kubernetes on Talos. It comes with the OS.

NixOS: pick one

NixOS gives you options:

# Option 1: K3s (lightweight)
services.k3s = {
  enable = true;
  role = "server";
};

# Option 2: Full Kubernetes via kubeadm
services.kubernetes = {
  roles = ["master" "node"];
  masterAddress = "k8s.example.com";
};

# Option 3: Just the kubelet
services.kubernetes.kubelet.enable = true;

You decide how Kubernetes gets onto the box. More options, and every option is a decision you now own.

Talos means fewer decisions and defaults that were chosen for Kubernetes specifically. NixOS lets you wire it up however suits you, and then you’re the one who has to stand behind the wiring.

Debugging

Talos: through the API

# View logs
talosctl logs kubelet --nodes 192.168.1.10

# Get system info
talosctl get members

# Read files
talosctl read /etc/hosts

# Dashboard (read-only TUI)
talosctl dashboard

Everything goes through the API. No SSH, no shell. There’s a debug container for emergencies, and it’s inconvenient on purpose so you don’t reach for it out of habit.

NixOS: SSH in

# SSH in
ssh root@192.168.1.10

# Use any tool
htop
journalctl -u k3s
kubectl get nodes
vim /etc/nixos/configuration.nix

It’s a Linux box. Every tool you know already works. That’s comfortable, and it’s also how drift starts, because every SSH session is a chance to fix something that never makes it back into Git. I’ve done it.

Talos makes you debug through the API, the logs and the metrics. NixOS lets you debug however you like, including the ways that leave the node different from its config. Whether that freedom helps or hurts depends on how disciplined you are on a bad day.

What I’d choose

After running both, this is how I decide.

Choose Talos when:

• Kubernetes is the only workload on these nodes
• you want the smallest attack surface you can get
• preventing drift matters more to you than flexibility
• you’re fine managing machines through an API
• the team already works declaratively

Choose NixOS when:

• the same nodes need non-Kubernetes workloads
• you want a full Linux with the declarative benefits
• you need packages that don’t exist as Talos extensions
• the team is already invested in Nix
• you want to customise past what Talos allows

My setup

I run both, in different roles. The homelab Kubernetes nodes are Talos. They only run Kubernetes, and I want the security and the simplicity of a cluster with nowhere to hide. The auxiliary servers are NixOS. They run Postgres, backup services and the monitoring infrastructure I keep outside Kubernetes on purpose.

That gives me cluster nodes that can’t drift, and NixOS machines for everything that doesn’t belong in a pod. Each one sits where its limits are useful.

The hybrid approach

Some teams run NixOS on the control plane nodes, where they want debugging access, and Talos on the workers, where they want the lockdown. It works. It also means maintaining two shapes of node for as long as the cluster exists.

There’s another path I’ve seen work. Start on NixOS because it’s familiar, then move to Talos once the team is used to declarative operations. NixOS teaches the mental model gently. Talos enforces it.

The KubeVirt option

There’s a third approach worth putting on the table. KubeVirt, with NixOS running as VMs inside Kubernetes. Talos, or any other minimal OS, goes on the bare metal, and the NixOS workloads become VMs that Kubernetes manages.

The stack looks like this:

Why this works

Your VM definitions become Kubernetes manifests:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: nixos-workload-01
spec:
  running: true
  template:
    spec:
      domain:
        resources:
          requests:
            memory: 4Gi
            cpu: 2
        devices:
          disks:
            - name: nixos-disk
              disk:
                bus: virtio
      volumes:
        - name: nixos-disk
          persistentVolumeClaim:
            claimName: nixos-workload-01-pvc

With ArgoCD driving it, this VM definition lives in Git. The NixOS config inside the VM lives in Git too. Top to bottom, the whole thing is declarative and has a history.

The full declarative stack

Combine that with:

• Ansible for the bare metal provisioning, installing Talos or K3s on the physical nodes
• ArgoCD for everything in Kubernetes, KubeVirt VMs included
• NixOS flakes for the VM configurations, pulled from Git when the VM boots

Then your entire infrastructure, from bare metal to VMs to Kubernetes workloads, is defined in Git. Change a VM’s NixOS config, commit, ArgoCD syncs, the VM rebuilds. Full audit trail, full reproducibility, and exactly the sovereignty story I keep coming back to.

When this makes sense

This earns its complexity when:

• you have mixed workloads, some containerised and some that need a full VM
• you want one control plane, Kubernetes, for all of it
• you have workloads that can’t be containerised but still want them under GitOps
• you’re already running KubeVirt for other reasons

The cost is real. You’re now running a virtualisation layer inside Kubernetes, with everything that implies. But if you need VMs anyway, managing them through the same GitOps pipeline as the rest of your estate is a strong place to be.

Example: a database on a NixOS VM

Say you want a PostgreSQL server with kernel tuning that won’t fit in a container. Instead of maintaining a separate NixOS server by hand:

1. Define the VM in a KubeVirt manifest, in Git
2. The VM boots NixOS and pulls its config from a flake, in Git
3. ArgoCD manages the VM lifecycle
4. Backups, monitoring and networking all run through Kubernetes

The database gets NixOS’s flexibility, you get Kubernetes’s scheduling, and everything is in Git.

Which constraints do you want

NixOS and Talos both point at the same kind of infrastructure: declarative, reproducible, version-controlled. What differs is which constraints they put on you.

Talos boxes you into Kubernetes-only, API-only, minimal-surface operations. That’s what you want when you’d rather the system enforce the discipline than rely on yours.

NixOS holds you to a declarative configuration and leaves you the full run of a Linux box. That’s what you want when you need the range and you trust the team to use it without drifting.

For pure Kubernetes nodes I run Talos. For everything around them I run NixOS. Both projects ship regularly and both communities are active, so if you’re picking today, pick by what else the node has to run. That was the question that settled it for me.

• NixOS
• Talos
• Linux
• Infrastructure