DevOps

The first cluster I ever ran in anger had exactly one permission model: everything was cluster-admin. My CI pipeline, my monitoring stack, the little webhook receiver I threw together one afternoon. All of it could read every secret, delete every deployment, and touch every namespace. It worked great right up until I started thinking about what happens when one of those pods gets popped. Kubernetes RBAC (Role-Based Access Control) answers a single question: who can do what to which resources? The default answer on most clusters is “everyone can do everything,” and that answer quietly becomes your biggest liability. ...

I used to keep a wiki page titled “Cluster conventions”. Resource limits on everything. No :latest tags. No deploys in the default namespace. It was a good page. Nobody read it. Six months in, half the cluster broke those rules and I only found out when something fell over. A rule that lives in a doc is a suggestion. A rule the API server refuses to accept is governance. That gap is the whole reason this post exists. ...

A standard Kubernetes Deployment had served me well for a long time. Push a new image tag, watch the pods roll, done. It was simple, it was declarative, and most of the time nothing went wrong. The rolling update even gave me a warm feeling of safety: old pods only get torn down once new ones are ready. That feeling is a lie. A rolling update protects you from pods that fail to start. It does nothing to protect you from pods that start perfectly and then serve broken responses. The container is healthy, the readiness probe is green, and your new code is quietly returning 500s to every single user. Within seconds, 100% of your traffic is hitting code that nobody validated under real load. ...

Picture the end state first. You merge a branch to main, walk away to get coffee, and by the time you’re back the project has a new version tag, a fresh changelog entry, and a GitLab release with notes written from your commits. Nobody decided “is this 1.4.0 or 1.3.5?” because nobody had to. The version fell out of the work itself. That’s where I want to get to in this post. I run a self-hosted GitLab and I do not want versioning to be a thing I think about. A version number that someone picks by hand is a tiny black box: it depends on whether that person remembered the rules, was paying attention, and agreed with everyone else about what counts as a breaking change. I would rather the number be a consequence of the commits, something I can inspect and reproduce. ...

For a long time my CI lived on someone else’s servers. Push code, wait for a hosted runner, watch the minutes tick down against a monthly quota, hope the provider doesn’t change the rules next quarter. It worked, right up until I started asking where my source code actually sat while it was being built, and who else could see it. So I moved GitLab in-house. I run it self-hosted now, on my own hardware, next to the clusters it deploys to. That’s sovereignty applied to CI/CD: my code, my builds, my artifacts, no vendor that can change pricing, deprecate a feature I depend on, or read my repositories without me knowing. ...