Posts

A standard Kubernetes Deployment had served me well for a long time. Push a new image tag, watch the pods roll, done. It was simple, it was declarative, and most of the time nothing went wrong. The rolling update even gave me a warm feeling of safety: old pods only get torn down once new ones are ready. That feeling is a lie. A rolling update protects you from pods that fail to start. It does nothing to protect you from pods that start perfectly and then serve broken responses. The container is healthy, the readiness probe is green, and your new code is quietly returning 500s to every single user. Within seconds, 100% of your traffic is hitting code that nobody validated under real load. ...

Pull a base image, copy in your code, push it to production. That’s the loop most of us run on autopilot. The part nobody looks at is the base image itself, which quietly drags in a few hundred packages you never chose. Any one of them could carry a known CVE, and you’d have no idea. That bothers me. I don’t like running things I can’t inspect, and a container image you haven’t scanned is exactly that: a black box you’re trusting because looking inside felt like too much effort. ...

Picture the end state first. You merge a branch to main, walk away to get coffee, and by the time you’re back the project has a new version tag, a fresh changelog entry, and a GitLab release with notes written from your commits. Nobody decided “is this 1.4.0 or 1.3.5?” because nobody had to. The version fell out of the work itself. That’s where I want to get to in this post. I run a self-hosted GitLab and I do not want versioning to be a thing I think about. A version number that someone picks by hand is a tiny black box: it depends on whether that person remembered the rules, was paying attention, and agreed with everyone else about what counts as a breaking change. I would rather the number be a consequence of the commits, something I can inspect and reproduce. ...

For a long time my CI lived on someone else’s servers. Push code, wait for a hosted runner, watch the minutes tick down against a monthly quota, hope the provider doesn’t change the rules next quarter. It worked, right up until I started asking where my source code actually sat while it was being built, and who else could see it. So I moved GitLab in-house. I run it self-hosted now, on my own hardware, next to the clusters it deploys to. That’s sovereignty applied to CI/CD: my code, my builds, my artifacts, no vendor that can change pricing, deprecate a feature I depend on, or read my repositories without me knowing. ...

The whole pitch of GitOps is that Git is the source of truth. That promise holds right up until someone runs kubectl edit on a deployment at 2am to stop an incident, a mutating webhook quietly rewrites a resource, or a half-finished sync leaves your cluster somewhere between what Git wanted and what it got. Now Git says one thing and the cluster does another, and nobody told you. That gap is configuration drift, and it is the part of GitOps people forget to defend. The good news: ArgoCD already watches for it. The catch is that the defaults don’t do what you probably assume, and a few of them will bite you. This post walks from the simplest possible drift check up to the setup I actually run, one layer at a time. Stop wherever you have enough. ...