TL;DR KVM and QEMU are the engine inside Proxmox and most of the cloud you rent. On NixOS, with libvirt on top, that engine is one block of config, and live migration, memory ballooning, GPU passthrough, SR-IOV and CPU pinning are each a command or an XML snippet away. What Proxmox adds is the dashboard and the delegation, and I’m honest below about how much that’s worth.
Ask around how to run a “real” hypervisor at home and you get the same shortlist. VMware, Hyper-V, or at minimum Proxmox. Something with a web UI, a clustering tab and a marketing page full of enterprise features. The shortlist is so well worn that running VMs without one of those products feels like cutting corners. I don’t blame anyone for that. The marketing is good.
Here’s what the marketing leaves out. The part doing the work in every one of those products is a Linux kernel module that’s been production-grade for over a decade. KVM with libvirt does live migration, memory ballooning, CPU pinning, GPU passthrough, SR-IOV and nested virtualization. The features on the glossy page are kernel features, and the web UI sits in front of them.
I run NixOS as my hypervisor. No Proxmox, no web UI, a declarative Nix config and virsh. This post walks through what that gives me, feature by feature, and then the part where I’d be lying if I said the CLI was always enough.
KVM, QEMU and libvirt
Three names come up in every command below, so a short round of introductions. KVM is the kernel module. It turns the Linux kernel into a type-1 hypervisor with hardware acceleration, so a guest runs at close to native speed. QEMU is the userspace program that gives the guest its virtual hardware, the disks and the network cards. libvirt is the management layer over both, with a stable API across virtualization backends and the virsh CLI on top.
It’s the same stack from the VMs on your laptop up to the cloud. AWS’s Nitro hypervisor is built on KVM. Google Cloud runs KVM. Most of the cloud you pay for is this stack with a billing system bolted on.
The NixOS config
Here’s my hypervisor config, the whole thing:
{ config, pkgs, ... }:
{
# Enable virtualization
virtualisation.libvirtd = {
enable = true;
qemu = {
package = pkgs.qemu_kvm;
runAsRoot = true;
swtpm.enable = true; # TPM emulation
ovmf = {
enable = true; # UEFI support
packages = [ pkgs.OVMFFull.fd ];
};
};
};
# Add yourself to the libvirt group
users.users.tom = {
extraGroups = [ "libvirtd" ];
};
# Useful tools
environment.systemPackages = with pkgs; [
virt-manager # GUI if you want it
virt-viewer # For console access
libguestfs # VM image manipulation
win-virtio # Windows VirtIO drivers
];
# Bridge networking (optional, for bridged VMs)
networking.bridges.br0.interfaces = [ "enp1s0" ];
networking.interfaces.br0.useDHCP = true;
}
Apply it and the machine is a hypervisor. There’s no installer, no setup wizard and no account to create on first boot, because the config is the install. Remove the block and rebuild, and the hypervisor is gone the same way it arrived.
Creating VMs with virsh
Let’s make one. First a disk:
# Create a 50GB qcow2 disk
qemu-img create -f qcow2 /var/lib/libvirt/images/ubuntu-server.qcow2 50G
Then define the VM. You can write the libvirt XML by hand, or let virt-install write it for you:
virt-install \
--name ubuntu-server \
--memory 4096 \
--vcpus 2 \
--disk /var/lib/libvirt/images/ubuntu-server.qcow2 \
--cdrom /var/lib/libvirt/images/ubuntu-24.04-server.iso \
--os-variant ubuntu24.04 \
--network bridge=br0 \
--graphics vnc
That’s a running VM. The day-to-day is five commands:
virsh list --all # List all VMs
virsh start ubuntu-server # Start VM
virsh shutdown ubuntu-server # Graceful shutdown
virsh destroy ubuntu-server # Force stop
virsh console ubuntu-server # Serial console
Live migration
This is the feature people are most sure needs a commercial product. Moving a running VM to another host with no downtime sounds like something with a licence key attached.
It’s a virsh subcommand. Live migration copies the VM’s memory to the target host while the guest keeps running, then does the final switchover in milliseconds. The guest doesn’t notice.
You need three things in place:
- Shared storage (NFS, Ceph , GlusterFS) or the same disk path on both hosts
- The same CPU architecture, or a compatible CPU model
- libvirt running on both hosts
# From the source host
virsh migrate --live ubuntu-server qemu+ssh://target-host/system
# With options for better performance
virsh migrate --live --persistent --undefinesource \
--copy-storage-all \
ubuntu-server qemu+ssh://target-host/system
One command, and the VM is running on the other host. If you don’t have shared storage, --copy-storage-all moves the disk along with the memory.
If you want the target host picked for you, that’s a loop around the same command:
#!/bin/bash
# Simple load balancer - migrate VM to least loaded host
HOSTS=("host1" "host2" "host3")
VM="ubuntu-server"
# Find host with lowest load
TARGET=$(for h in "${HOSTS[@]}"; do
echo "$(ssh $h 'cat /proc/loadavg | cut -d" " -f1') $h"
done | sort -n | head -1 | cut -d" " -f2)
virsh migrate --live $VM qemu+ssh://$TARGET/system
That’s a load balancer in a few lines of bash. Proxmox’s HA tab drives the same QEMU migration.
Memory ballooning
Ballooning lets you change a VM’s memory while it runs. The guest has a balloon driver that hands pages back to the host or asks for more, no reboot needed.
Enable it in the VM definition:
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</memballoon>
Then resize at runtime:
# Set current memory to 2GB (VM has 4GB max)
virsh setmem ubuntu-server 2G --live
# Give it back 4GB
virsh setmem ubuntu-server 4G --live
This is what makes overcommitting safe. Hand out 64GB across VMs on a 32GB host and let the balloon drivers sort out who needs what.
GPU passthrough
The one homelab people obsess over. Hand a physical GPU to a VM and the guest gets near-native graphics, which is what a gaming VM, a machine learning box or a hardware transcoder wants.
First IOMMU, in the NixOS config:
{
boot.kernelParams = [
"intel_iommu=on" # or amd_iommu=on for AMD
"iommu=pt"
];
# Isolate the GPU from the host
boot.extraModprobeConfig = ''
options vfio-pci ids=10de:2204,10de:1aef
'';
boot.initrd.kernelModules = [
"vfio_pci"
"vfio"
"vfio_iommu_type1"
];
}
Find the GPU’s PCI IDs:
lspci -nn | grep -i nvidia
# 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation ... [10de:2204]
# 01:00.1 Audio device [0403]: NVIDIA Corporation ... [10de:1aef]
Then hand both functions, video and audio, to the VM:
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</source>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
</source>
</hostdev>
The VM talks to the GPU directly. Install the vendor drivers in the guest and you have full hardware acceleration with no host in the path.
Network card passthrough and SR-IOV
Same principle for network cards. A 10GbE or 25GbE card can go straight to a VM:
# Find the NIC
lspci -nn | grep -i ethernet
# Pass it through like the GPU
SR-IOV (Single Root I/O Virtualization) is the more interesting option. It splits one physical NIC into several virtual functions, each of which you can hand to a different VM:
# Enable 4 virtual functions on the NIC
echo 4 > /sys/class/net/enp5s0f0/device/sriov_numvfs
# List them
lspci | grep "Virtual Function"
Each VF has its own PCI address. The VM gets hardware-speed networking without virtio in between.
CPU pinning
For latency-sensitive workloads you want the VM’s vCPUs nailed to specific host cores, so the scheduler stops moving them around:
<vcpu placement='static'>4</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='4'/>
<vcpupin vcpu='1' cpuset='5'/>
<vcpupin vcpu='2' cpuset='6'/>
<vcpupin vcpu='3' cpuset='7'/>
<emulatorpin cpuset='0-1'/>
</cputune>
That pins the VM’s 4 vCPUs to host cores 4-7 and keeps the emulator threads on cores 0-1. Add isolcpus to the kernel parameters and the latency gets close to bare metal.
VMs in the same config as the host
This is where NixOS pulls ahead of a stock Linux box with libvirt on it. The VMs themselves can live in the config:
{
virtualisation.libvirtd.enable = true;
# Define VMs as Nix expressions
systemd.services."libvirt-guest-ubuntu" = {
after = [ "libvirtd.service" ];
requires = [ "libvirtd.service" ];
wantedBy = [ "multi-user.target" ];
serviceConfig = {
Type = "oneshot";
RemainAfterExit = "yes";
};
script = ''
${pkgs.libvirt}/bin/virsh define /etc/libvirt/qemu/ubuntu-server.xml || true
${pkgs.libvirt}/bin/virsh start ubuntu-server || true
'';
preStop = ''
${pkgs.libvirt}/bin/virsh shutdown ubuntu-server || true
sleep 10
${pkgs.libvirt}/bin/virsh destroy ubuntu-server || true
'';
};
}
Or with NixVirt, which is cleaner:
{
virtualisation.libvirt.connections."qemu:///system" = {
domains = [{
definition = ./vms/ubuntu-server.xml;
active = true;
}];
};
}
The VMs are now in the same file as the host. Versioned, reproducible, and gone the moment you remove them from the file. There’s no stale definition somewhere that nobody remembers creating, because the file is the only place a definition can be.
Where the CLI hurts
So far this reads like the simpler stack wins. I’ve run it for years and I’m not going to pretend that’s the whole picture. Every feature is there. The day-to-day around the features is rougher than a good UI, and I’d rather say so than have you find out.
Four things get tedious. virt-install has a flag for everything, so I’m in the man page every time I make a VM. I keep wrapper scripts, they work, and they encode assumptions only I know about.
Monitoring means scripts that poll virsh for status and resource usage, or a Grafana dashboard with a libvirt exporter, which is a weekend of its own. Storage pools, volumes and snapshots are all doable from virsh, but the commands are long and I fat-finger them. And which VM sits on which bridge with which IP lives in my scripts or my head. There’s no screen for it.
Three things scripts do badly. A quick overview, the “how is everything doing” glance, is virsh list plus virsh domstats plus a parser, and that’s no match for a dashboard. Console access is fine locally with virt-viewer, and a web console is nicer when I’m remote and half awake. Delegation, letting someone manage two specific VMs without handing them the host, is awkward with raw libvirt.
Why Proxmox exists
Which brings me to Proxmox. The same KVM, QEMU and libvirt underneath, wrapped in a web UI that does everything in the previous section without a script.
Do I think you need it? For a homelab with one admin, no. NixOS plus virsh plus a few scripts has been fine for me for years. Where the UI earns its place is onboarding, where a new person reads the infrastructure off a screen instead of out of my scripts. Incidents, where clicking beats recalling virsh syntax under stress. Visibility, with every VM, its status and its network in one view.
Proxmox also ships clustering, Ceph integration and backup management, and rebuilding those from scripts is real work.
What Proxmox sells is the operations layer. The hypervisor underneath it is the one you already have.
Back to the shortlist
The shortlist was right about the dashboard and wrong about the hypervisor. VMware, Hyper-V and Proxmox run the same kernel module as my NixOS box, and NixOS adds declarative config and version control that none of them have on that axis. What those three sell is the dashboard, the delegation and the ergonomics during an incident.
Which one you want depends on which of those two you’d rather build yourself. If you want to understand virtualization down to the kernel module, start with NixOS, KVM and virsh. Nothing is hidden, and every “enterprise” feature is a command you typed. As I argued in Sovereign Infrastructure, a black box you can’t inspect is its own kind of risk, and there’s no black box in this stack.
If you run production with a team, the Proxmox UI is buying you an operations layer, and that’s a fair thing to buy. For my homelab, understanding the machine is half the reason it exists, so the config at the top of this post is my hypervisor. Declarative and versioned, with no vendor between me and the kernel. It’s still the whole install.