As a kid I had a word for things that fascinated me: unbreakable. Not “indestructible” — that implies something never breaks. Unbreakable is different. It means something even broken still works. I remember exactly when that fascination began. A photo of an A-10 Thunderbolt II, returned from a mission. Half the wing gone. Tail in tatters. Fuselage full of holes. And yet that thing had brought its pilot home. That’s not luck. That’s design. ...
Kubernetes is designed to be self-healing, but what does that actually mean? More importantly: what happens when the components doing the healing themselves fail? I’ve run Kubernetes clusters through all kinds of failures — planned, unplanned, and “hold my beer” experiments. Here’s what actually happens when things break. The Components That Can Fail Before diving into failure scenarios, let’s map out what we’re working with: Control Plane: kube-apiserver: The API that everything talks to etcd: The database storing all cluster state kube-scheduler: Decides where pods run kube-controller-manager: Runs controllers (ReplicaSet, Deployment, etc.) cloud-controller-manager: Cloud provider integrations (if applicable) Node Components: ...