Monitoring on Tom Meurs

Grafana Dashboards That Actually Get Used

Sat, 02 May 2026 20:15:00 +0100

TL;DR Each dashboard answers one question, and within five seconds I know whether I need to care. Four kinds of dashboard cover everything, variables and links tie them together, and the JSON lives in Git. My homelab runs five, and anything that goes a month unopened gets deleted.

You have Grafana. You have Prometheus metrics. You have logs in Loki and traces in Tempo. The data’s all there.

You also have 47 dashboards nobody opens.

Alerting That Works: From Alert Fatigue to Actionable Notifications

Thu, 16 Apr 2026 20:15:00 +0100

TL;DR Most default alert rules page on a number with no user behind it, and after a week of that you stop reading pages. I alert on error rate, P99 latency, a disk that fills within 24 hours and a certificate under seven days, about ten rules in total, each with a runbook. Alertmanager sends critical to the pager, warning to Slack during office hours, info nowhere. Everything that doesn’t need a human gets deleted.

Distributed Tracing with Tempo and OpenTelemetry

Sat, 04 Apr 2026 19:47:00 +0100

TL;DR Tempo is where my traces go. The OpenTelemetry Collector receives spans from every service, tail sampling keeps every error and every slow request, and Tempo writes the rest to MinIO with three days of retention. From a span in Grafana I click through to the Loki lines with the same trace ID. My production values are at the bottom.

Your metrics say something is slow. Your logs say errors happened. Fine. Which request failed, where did the latency come from, and which service in the chain ate the timeout? Neither of them can tell you.

Loki for Kubernetes Logging: The Prometheus-Like Approach

Tue, 31 Mar 2026 21:15:00 +0100

TL;DR Loki indexes labels and stores the log text as compressed chunks, so a month of logs fits on commodity hardware. The trade is full-text search. Below is the single-binary install I’d hand anyone first, the same chart on MinIO with three replicas, and the knobs I didn’t touch until the volume was real.

You’ve got Prometheus for metrics, so you can already see what’s happening across your clusters. Metrics tell you request latency spiked at 14:32. They don’t tell you the payment service threw a null pointer because someone shipped a config change with a typo. For that you need the log line.

Data-Driven Kubernetes Migration: Why You Need Metrics Before You Move

Sun, 08 Feb 2026 14:00:00 +0100

TL;DR An e-commerce client moved a PHP monolith to EKS in October, two weeks before Black Friday, with resource limits guessed from the old server specs. Mid-morning the pods hit their memory limit, died in a loop for four hours, and the estimate was €240,000 in lost revenue. The second half is how to get those numbers from load tests and VPA recommendations instead of guessing them.

“We want to migrate to Kubernetes by November.”

Prometheus and Thanos: Metrics at Scale

Sun, 31 Aug 2025 10:00:00 +0200

TL;DR Prometheus on its own keeps a couple of weeks of metrics on local disk and can’t answer a question across clusters. Thanos puts a sidecar next to each Prometheus, ships the blocks to object storage, and puts a querier in front that reads all of it back through the same PromQL. Below is the install, the four components, the HA and multi-cluster wiring, and the numbers I run in production.

Runtime Security with Falco: Detect Suspicious Behavior in Your Cluster

Thu, 07 Aug 2025 10:00:00 +0200

TL;DR Trivy and Kyverno check an image before it runs. Falco watches what the container does after that, syscall by syscall, and alerts when the behaviour doesn’t match a rule. This post goes from the one thing Falco does to the config I run, with the noise problem in between.

I scanned my images with Trivy. I enforced policies with Kyverno. My workloads got cryptographic identity through SPIFFE. Three layers of prevention, all green, and for a while that felt like enough.

Drift Detection with ArgoCD: How to Know If Your Cluster Is Still in Sync

Sat, 03 May 2025 19:27:00 +0200

The whole pitch of GitOps is that Git is the source of truth. That promise holds right up until someone runs kubectl edit on a deployment at 2am to stop an incident, a mutating webhook quietly rewrites a resource, or a half-finished sync leaves your cluster somewhere between what Git wanted and what it got. Now Git says one thing and the cluster does another, and nobody told you.

That gap is configuration drift, and it is the part of GitOps people forget to defend. The good news: ArgoCD already watches for it. The catch is that the defaults don’t do what you probably assume, and a few of them will bite you. This post walks from the simplest possible drift check up to the setup I actually run, one layer at a time. Stop wherever you have enough.