How does Service A know that Service B is actually Service B? I keep coming back to that question because the usual answer is uncomfortable. For years we trusted network location. Traffic from the right IP was legitimate, end of story. Zero trust took that assumption out back and shot it. Now every service has to prove who it is, every single request, no matter where it sits on the network. ...
I used to keep a wiki page titled “Cluster conventions”. Resource limits on everything. No :latest tags. No deploys in the default namespace. It was a good page. Nobody read it. Six months in, half the cluster broke those rules and I only found out when something fell over. A rule that lives in a doc is a suggestion. A rule the API server refuses to accept is governance. That gap is the whole reason this post exists. ...
The first time I ran kubectl get secret myapp -o yaml and base64-decoded the value, I felt my stomach drop. There was my database password, sitting in etcd, readable by anyone who could reach the API with the right RBAC. Kubernetes Secrets are not secrets. They’re base64-encoded plain text with a fancy name. That’s the default, and it’s the thing nobody warns you about on day one. Every cloud provider has a fix for sale. AWS has Secrets Manager, Google has Secret Manager, Azure has Key Vault. They work. The catch shows up later: the day you need to migrate, the day you want to know exactly what happens to a secret after you write it, the day you realise your most sensitive data lives in a system you can’t inspect. ...
A standard Kubernetes Deployment had served me well for a long time. Push a new image tag, watch the pods roll, done. It was simple, it was declarative, and most of the time nothing went wrong. The rolling update even gave me a warm feeling of safety: old pods only get torn down once new ones are ready. That feeling is a lie. A rolling update protects you from pods that fail to start. It does nothing to protect you from pods that start perfectly and then serve broken responses. The container is healthy, the readiness probe is green, and your new code is quietly returning 500s to every single user. Within seconds, 100% of your traffic is hitting code that nobody validated under real load. ...
For a long time my CI lived on someone else’s servers. Push code, wait for a hosted runner, watch the minutes tick down against a monthly quota, hope the provider doesn’t change the rules next quarter. It worked, right up until I started asking where my source code actually sat while it was being built, and who else could see it. So I moved GitLab in-house. I run it self-hosted now, on my own hardware, next to the clusters it deploys to. That’s sovereignty applied to CI/CD: my code, my builds, my artifacts, no vendor that can change pricing, deprecate a feature I depend on, or read my repositories without me knowing. ...