Advanced on Tom Meurs

Supply Chain Security in GitLab CI: SBOM, cosign, and the Policy That Blocks

Tue, 29 Sep 2026 09:00:00 +0200

TL;DR A pipeline that scans images and writes an SBOM, with nothing checking either at admission, had been green for a year and had stopped nothing. Four steps below: an SBOM Trivy can read back, a cosign signature on the digest with the key in Vault, the SBOM attached as an attestation, and a Kyverno policy that refuses the rest. Then a table of which SLSA Build L3 requirements a self-hosted GitLab runner meets, and the two it doesn’t.

NIS2 for Platform Teams: Which Control Satisfies Which Article

Sun, 20 Sep 2026 21:00:00 +0200

TL;DR The Cyberbeveiligingswet has applied since 15 August 2026. Below, the ten measures of article 21 mapped onto controls that actually run, a Kyverno policy that proves one of them, the reporting deadlines as three Prometheus alerting rules, and the four duties no YAML covers. The first run of that policy found seven pods in my own cluster I’d have sworn were mirrored.

Building an Internal Developer Platform: Where to Start

Wed, 06 May 2026 19:30:00 +0100

TL;DR An Internal Developer Platform is a thin layer between what a developer wants and what the cluster demands. Mine is a Helm base chart, ArgoCD and a handful of Kyverno policies, about 500 lines of my own code. Start with those three, and add observability and a portal only when the tickets tell you to.

Should you build an Internal Developer Platform? Usually, yes. It’s the how that wrecks teams.

Chaos Engineering: Breaking Your Cluster to Make It Stronger

Tue, 28 Apr 2026 19:45:00 +0100

TL;DR Chaos engineering is breaking your own cluster on purpose, with the hypothesis written down first and a rollback ready. I run it on the homelab with Litmus: a weekly pod-delete, a monthly node drain, a game day every quarter. So far it has found four things the dashboard would never have shown me.

My dashboard is a wall of green. Pods running, replicas matched, CPU comfortable, no alerts firing. I look at it and get that small hit of “everything’s fine.” And mostly it is. The cluster’s been up for weeks and nothing has fallen over.

Longhorn vs Rook-Ceph: Kubernetes Storage Compared

Mon, 20 Apr 2026 19:38:00 +0100

TL;DR Longhorn and Rook-Ceph both give you replicated block storage that survives a node dying. Longhorn is one Helm install and a UI that shows you what a volume is doing. Rook-Ceph is Ceph with a Kubernetes operator around it, so you get block, object and filesystem storage, and the operations that come with them. I run Longhorn on three nodes. Below is why, and when I’d switch.

Run your first stateful workload on a self-hosted cluster and you find out fast that nobody hands out PersistentVolumes for free. There’s no cloud provider storage class to lean on. You’ve got your nodes, their disks, and a Postgres pod that won’t schedule. An hour of reading later you’re left with two names that keep coming up. Longhorn and Rook-Ceph .

Quantum-safe GPG identity with multiple aliases

Sat, 18 Apr 2026 10:47:00 +0200

TL;DR One GPG masterkey that only certifies, kept offline. Three aliases bound to it, so anyone can check they’re the same person. Ed25519 subkeys for signing and SSH, an ML-KEM plus X448 subkey for encryption so recorded ciphertext stays closed after a quantum computer shows up, and three backups that each assume the other two are gone. Needs GnuPG 2.5.

A cryptographic signature still means exactly what it says. If the key is yours and the signature verifies, the content came from you. No vendor handed you that identity, no CA can pull it, no platform can suspend it. It exists because you generated the key, and it’s yours for as long as you hold the private half. Most of what gets called “online identity” is borrowed. A handle someone can ban, an email address the domain owner can take back the day they feel like it. A GPG signature sits outside all of that.

Alerting That Works: From Alert Fatigue to Actionable Notifications

Thu, 16 Apr 2026 20:15:00 +0100

TL;DR Most default alert rules page on a number with no user behind it, and after a week of that you stop reading pages. I alert on error rate, P99 latency, a disk that fills within 24 hours and a certificate under seven days, about ten rules in total, each with a runbook. Alertmanager sends critical to the pager, warning to Slack during office hours, info nowhere. Everything that doesn’t need a human gets deleted.

Cilium: eBPF Networking for Kubernetes

Wed, 08 Apr 2026 20:23:00 +0100

TL;DR Cilium is the CNI I run in every cluster. It replaces kube-proxy and its iptables chains with eBPF programs in the kernel, enforces network policies on identity and on HTTP paths, and Hubble shows me every flow and every drop. WireGuard between the pods, BGP to the switch, no sidecars. My production values are at the bottom.

The first time a service stopped resolving in one of my clusters, I spent an evening reading iptables chains. Hundreds of rules, generated by kube-proxy, evaluated top to bottom. I never found the problem. I restarted the node and it went away.

Thanos Remote Write: Push-Based Metrics for Edge and Multi-Cluster

Fri, 27 Mar 2026 20:38:00 +0100

TL;DR Thanos Receive turns the metrics flow around. Each edge Prometheus pushes samples over remote_write to one central endpoint, buffers in its WAL when the link drops, and never sees an object storage credential. Below is the Receive StatefulSet, the hashring, the edge config, two ways to lock the endpoint down, and the hybrid I run with sidecars on the core clusters.

In my previous post on Prometheus and Thanos, I set up the sidecar architecture. Thanos Sidecar runs next to Prometheus, uploads TSDB blocks to object storage, and exposes the data to the Querier over gRPC. For clusters in the same datacenter with a fat, stable link to the central infrastructure, I’ve got no complaints. Everything pulls, and everything can reach everything else.

Declarative Infrastructure as Compliance Documentation: Talos, NixOS, and Audit-Ready Systems

Mon, 23 Mar 2026 19:28:00 +0100

TL;DR Talos and NixOS boot from a file that says what the machine is. That file is what an ISO 27001, SOC 2 or NIS2 auditor asks for, so the evidence collection shrinks to a git log, a diff with no output and an inventory dump. Below is the mapping from config to control, the scripts, and the part that stays hard work.

An ISO 27001 audit starts weeks before the auditor shows up, with someone collecting screenshots. Configuration panels, firewall rules, a dashboard that shows which patches went out. Then the Word documents that describe what the systems are supposed to do. Then the change tickets, dug out of the ticketing system, each one pointing at a “server maintenance” nobody can reconstruct six months later. Everyone treats this as the price of being in business. I did too, for years.

NixOS vs Talos for Kubernetes Nodes: Two Flavors of Immutable Infrastructure

Sun, 15 Mar 2026 20:37:00 +0100

TL;DR Talos and NixOS are both declarative and both can be immutable, and that’s about where the resemblance stops. Talos runs Kubernetes and nothing else, with no shell to drift in. NixOS is a full Linux where the whole machine is one Nix config. I run Talos on my cluster nodes and NixOS on the machines around them, and this post is how I got to that split.

I’ve written about Talos Linux as the immutable Kubernetes OS, and I’ve compared Arch vs NixOS for workstations. After both, the same question kept arriving in my inbox. Why not NixOS on the Kubernetes nodes too?

Talos Linux: The Immutable Kubernetes OS That Changed How I Think About Nodes

Wed, 11 Mar 2026 21:15:00 +0100

TL;DR Talos Linux is a Linux that runs Kubernetes and nothing else. No SSH, no shell, no package manager. Every change goes through an API and a YAML file in Git. I’ve run my nodes on it for three years, and this post is how I stopped missing the shell.

The first time I tried to SSH into a Talos node, I got nothing. No shell, no connection, not even a login prompt. I sat there for a minute wondering how I was supposed to debug a machine I couldn’t get into.

Data-Driven Kubernetes Migration: Why You Need Metrics Before You Move

Sun, 08 Feb 2026 14:00:00 +0100

TL;DR An e-commerce client moved a PHP monolith to EKS in October, two weeks before Black Friday, with resource limits guessed from the old server specs. Mid-morning the pods hit their memory limit, died in a loop for four hours, and the estimate was €240,000 in lost revenue. The second half is how to get those numbers from load tests and VPA recommendations instead of guessing them.

“We want to migrate to Kubernetes by November.”

K8sGPT with a Local 70B Model on Apple Silicon

Thu, 05 Feb 2026 14:00:00 +0100

TL;DR I pointed K8sGPT at a Llama 3.3 70B model running on a Mac Studio in my own office, so no cluster metadata leaves the network, and fed it three broken workloads. It found all three and wrote fixes I’d accept from a colleague, except for the resource numbers, which it made up. Assisted diagnosis works, at 25 seconds a query. Autonomous cluster management is still marketing, and the security section is why I wouldn’t let a model change anything on its own.

Runtime Security with Falco: Detect Suspicious Behavior in Your Cluster

Thu, 07 Aug 2025 10:00:00 +0200

TL;DR Trivy and Kyverno check an image before it runs. Falco watches what the container does after that, syscall by syscall, and alerts when the behaviour doesn’t match a rule. This post goes from the one thing Falco does to the config I run, with the noise problem in between.

I scanned my images with Trivy. I enforced policies with Kyverno. My workloads got cryptographic identity through SPIFFE. Three layers of prevention, all green, and for a while that felt like enough.

SPIFFE and SPIRE: Zero Trust Service Identity

Sat, 26 Jul 2025 10:00:00 +0200

How does Service A know that Service B is actually Service B?

I keep coming back to that question because the usual answer is uncomfortable. For years we trusted network location. Traffic from the right IP was legitimate, end of story. Zero trust took that assumption out back and shot it. Now every service has to prove who it is, every single request, no matter where it sits on the network.

Kyverno Policies: Governance as Code for Kubernetes

Mon, 14 Jul 2025 10:00:00 +0200

I used to keep a wiki page titled “Cluster conventions”. Resource limits on everything. No :latest tags. No deploys in the default namespace. It was a good page. Nobody read it. Six months in, half the cluster broke those rules and I only found out when something fell over.

A rule that lives in a doc is a suggestion. A rule the API server refuses to accept is governance. That gap is the whole reason this post exists.

Progressive Delivery with Argo Rollouts: Canary and Blue-Green Deployments

Fri, 20 Jun 2025 10:00:00 +0200

A standard Kubernetes Deployment had served me well for a long time. Push a new image tag, watch the pods roll, done. It was simple, it was declarative, and most of the time nothing went wrong. The rolling update even gave me a warm feeling of safety: old pods only get torn down once new ones are ready.

That feeling is a lie. A rolling update protects you from pods that fail to start. It does nothing to protect you from pods that start perfectly and then serve broken responses. The container is healthy, the readiness probe is green, and your new code is quietly returning 500s to every single user. Within seconds, 100% of your traffic is hitting code that nobody validated under real load.

Drift Detection with ArgoCD: How to Know If Your Cluster Is Still in Sync

Sat, 03 May 2025 19:27:00 +0200

The whole pitch of GitOps is that Git is the source of truth. That promise holds right up until someone runs kubectl edit on a deployment at 2am to stop an incident, a mutating webhook quietly rewrites a resource, or a half-finished sync leaves your cluster somewhere between what Git wanted and what it got. Now Git says one thing and the cluster does another, and nobody told you.

That gap is configuration drift, and it is the part of GitOps people forget to defend. The good news: ArgoCD already watches for it. The catch is that the defaults don’t do what you probably assume, and a few of them will bite you. This post walks from the simplest possible drift check up to the setup I actually run, one layer at a time. Stop wherever you have enough.

App-of-Apps Pattern in ArgoCD: Scalable GitOps Architecture

Mon, 21 Apr 2025 20:18:00 +0200

My homelab started with one ArgoCD Application. Then a handful. The day I caught myself running kubectl apply -f for the fifteenth time to register yet another Application, I knew I’d built the exact thing I was trying to avoid: manual steps I had to remember, in an order I had to remember, with no record of what should exist.

The App-of-Apps pattern fixes that with one idea. You create a single root Application by hand, and it creates everything else. After a cluster wipe I can rebuild the whole thing with one kubectl apply. That property is the entire reason I run it, and it’s the same reason I self-host in the first place: I want the repository to be the truth, not my memory.

GitOps Disaster Recovery: Rebuilding Your Cluster from Git

Wed, 09 Apr 2025 19:52:00 +0200

For a couple of years my homelab cluster ran without much drama. Nodes came and went, workloads shifted around, and the worst I ever had to deal with was the occasional pod stuck in CrashLoopBackOff. The kind of stable where you stop thinking about what happens if it all goes away.

Then one evening I ran a terraform destroy against the wrong workspace. By the time I noticed, the control plane was gone. Not degraded. Gone.

Running Kubernetes Offline: Edge Computing Without Internet

Tue, 04 Mar 2025 20:47:00 +0100

What happens when your Kubernetes cluster can’t reach the internet? I don’t mean a slow connection. I mean no connection at all. Ships at sea. Remote mining sites. Factory floors with air-gapped networks. Military deployments.

For a lot of people that sounds exotic, like a problem someone else has. I treat it as a baseline design requirement, and I’ll explain why it makes my homelab better even though I almost never actually pull the cable.

Graceful Degradation in Kubernetes: What Happens When Components Fail

Thu, 20 Feb 2025 14:22:00 +0100

Everyone repeats the line that Kubernetes is self-healing. Pods die, they come back. Nodes drop, workloads reschedule. The system reconciles itself toward the state you declared, and most days you never have to think about it.

Then one day the thing doing the healing is the thing that broke. The API server is down. etcd won’t respond. The scheduler is wedged. Now what?

This is the question I actually care about, because “self-healing” is only useful if I understand its edges. I want to know what degrades gracefully and what takes the whole cluster with it. So I’ve run my clusters through a lot of failures on purpose: planned, unplanned, and a few “hold my beer” experiments on hardware I didn’t mind losing. Here is what actually happens when each piece breaks, and why most of it matters less than people fear.

How etcd Actually Works: The Heart of Your Kubernetes Cluster

Mon, 27 Jan 2025 21:47:00 +0100

When something goes wrong in Kubernetes, the trail usually leads back to etcd. API server timing out? Check etcd. Pods stuck in pending? Might be etcd. Cluster feels sluggish? Probably etcd.

For a long time I treated etcd the way most operators do: as a black box that hums along next to the control plane. “The database.” You back it up and otherwise leave it alone. But black boxes feel like splinters to me, and the first time an etcd cluster fell over at 2am I realised I had no idea what I was actually looking at. So I learned. And it turns out the whole thing is built on a handful of ideas that, once they click, make most etcd problems diagnosable instead of terrifying.

Kubernetes High Availability: Stacked vs External etcd Explained

Wed, 15 Jan 2025 20:23:00 +0100

The first “production” Kubernetes cluster I ran had a single control plane node. It hummed along happily for weeks, right up until a disk failed and took the whole thing with it. Every pod, every service, gone. That outage taught me what “single point of failure” actually feels like, and it pushed me toward a question that I keep seeing trip people up: when you build a cluster that survives node loss, do you run etcd on your control plane nodes, or on dedicated nodes of its own?