I write a lot about Kubernetes, and I run it at home every day. I like it.
So it might surprise you that I spend a fair amount of my time talking people out of it.
The situation I keep walking into goes like this. A small team, one product, a roadmap full of features customers are waiting for, and someone has decided the first milestone is a Kubernetes cluster. Three nodes minimum, etcd, a CNI , an ingress controller, cert-manager, a monitoring stack. Weeks of work before a single customer sees anything. Everyone nods along, because that’s what a serious setup is supposed to look like.
There’s another version of that first milestone. You ship in an afternoon, you can name every moving part, and you still sleep fine.
This post is for whoever is standing at that decision. I’d like you to pick Kubernetes because you need it, or skip it because you don’t, and know which of the two you did.
The Kubernetes hype
Kubernetes won. It’s the default answer for container orchestration, every cloud sells a managed flavour, and every DevOps job ad lists it. Once something is the obvious choice, people stop asking whether it’s the right one. Nobody decides on Kubernetes anymore. They assume it.
I get the appeal. I picked it for my own homelab on purpose and I’ve written plenty about why. But “everyone runs it” is the weakest reason there is to run anything, and it’s a terrible reason to run infrastructure you’ll have to understand when it breaks.
Kubernetes earns its complexity by solving a specific set of problems:
- Orchestrating containers across multiple nodes
- Automatically scaling based on load
- Self-healing when containers crash
- Declarative infrastructure configuration
- Service discovery and load balancing
Read that list and be honest about which ones you have. All five? Then Kubernetes is a gift. None? Then it’s pure cost, and you’re paying in the one thing a small team has the least of, which is attention.
When Kubernetes is overkill
You have one application
One monolith. One database. Maybe a Redis cache. It runs fine on a single server today.
What is Kubernetes buying you here?
Your application → Kubernetes cluster → 3+ nodes → etcd →
control plane → ingress controller → cert-manager →
monitoring stack → ...
Versus:
Your application → Docker Compose → 1 server
Same functionality on both sides. The top one costs roughly ten times the time, money and headspace to get to. The bottom one you can hold in your head.
Your team is small
Kubernetes asks you to understand a lot before you can run it safely. The core, and then everything around it:
- Networking (CNI, service mesh, ingress)
- Storage (CSI , persistent volumes)
- Security (RBAC, network policies, pod security)
- Observability (metrics, logs, traces)
- GitOps (ArgoCD, Flux )
A team of two or three developers who also carry the pager doesn’t have the spare hours to learn and maintain all of that on top of building the product. Something gives. In my experience it’s the weekends first, and the uptime shortly after.
Your traffic is predictable
The headline feature is autoscaling. If your traffic sits at a steady 100 requests per second with no nasty spikes, autoscaling solves a problem you don’t have. Overprovision a little, put a reminder in for six months from now, move on.
Your budget is limited
Managed Kubernetes (EKS, GKE, AKS) isn’t free, and the line items add up fast:
- Control plane fees (~$70-150/month)
- Minimum 3 worker nodes for HA
- Load balancer costs
- Persistent volume costs
- Egress costs
For one simple app you’re quickly at $300-500 a month. That same app runs happily on a $20 VPS. You’re paying the difference for an abstraction you aren’t using.
You don’t have microservices
Kubernetes was built for microservices, many small independent services that scale and fail on their own schedules. A monolith doesn’t have those needs. Running one on Kubernetes is buying a Ferrari for the weekly grocery run. It works, but every feature you’re paying for sits idle in the driveway.
The hidden costs of Kubernetes
The sticker price is the easy part. The costs that hurt are the ones that never show up on an invoice.
Operational overhead
Even on managed Kubernetes, the cluster is a living thing you tend forever:
- Cluster upgrades (new version every 3-4 months)
- Node pool management
- Capacity planning
- Incident response when pods don’t start
- Debugging networking issues
That’s steady, recurring work, and someone on the team ends up owning it whether you planned for that or not.
Learning curve
Here’s the vocabulary you need before you can reason about a running cluster:
- Pods, Deployments, StatefulSets, DaemonSets
- Services, Ingress, NetworkPolicies
- ConfigMaps, Secrets
- PersistentVolumes, StorageClasses
- RBAC, ServiceAccounts
- Helm , Kustomize
- CRDs, Operators
And that’s before the ecosystem tools every cluster grows sooner or later. Prometheus, Grafana, ArgoCD, cert-manager, external-dns, and a dozen more. Each one is another thing to learn, patch, and understand when it misbehaves.
YAML hell
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
labels:
app: my-app
spec:
replicas: 3
selector:
matchLabels:
app: my-app
template:
metadata:
labels:
app: my-app
spec:
containers:
- name: my-app
image: my-app:v1.0.0
ports:
- containerPort: 8080
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 3
---
apiVersion: v1
kind: Service
metadata:
name: my-app
spec:
selector:
app: my-app
ports:
- port: 80
targetPort: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
# ... even more YAML
That’s the minimum to get one application running. Here’s the same intent in Docker Compose:
services:
my-app:
image: my-app:v1.0.0
ports:
- "80:8080"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
Debugging complexity
Container won’t start? On Docker the whole investigation is one line:
docker logs my-app
On Kubernetes the same question turns into a small expedition:
kubectl describe pod my-app-7d9f8b6c5d-x2k9p
kubectl logs my-app-7d9f8b6c5d-x2k9p
kubectl logs my-app-7d9f8b6c5d-x2k9p --previous
kubectl get events --field-selector involvedObject.name=my-app-7d9f8b6c5d-x2k9p
And the answer turns out to be an ImagePullBackOff from a typo in the image name. Or a resource limit. Or a full node. Every layer Kubernetes adds is a layer you walk through when you’re trying to work out why nothing happened. On a good day you don’t notice. On a bad day, four kubectl commands in, you do.
Alternatives to Kubernetes
So if not Kubernetes, then what? None of these are toys. Each one is a tool I’d reach for in the right situation, and several of them carry real production traffic for real companies right now.
Docker Compose
One server, multiple containers, a simple setup.
services:
app:
image: my-app:latest
ports:
- "80:8080"
depends_on:
- db
- redis
environment:
- DATABASE_URL=postgres://db:5432/app
restart: unless-stopped
db:
image: postgres:15
volumes:
- postgres_data:/var/lib/postgresql/data
restart: unless-stopped
redis:
image: redis:7
restart: unless-stopped
volumes:
postgres_data:
docker compose up -d and you’re running. Updates are docker compose pull && docker compose up -d. The whole mental model fits in one file you can read top to bottom.
What you get is something anyone on the team can understand, no cluster overhead, and a setup that works anywhere Docker runs. Developers love it. What you give up is automatic scaling, high availability, since it’s one node, and most of what the word orchestration means.
Docker Swarm
You need multiple nodes, and Kubernetes is too much.
Docker Swarm is the forgotten sibling. Simpler, fewer features, and often more than enough.
# Init swarm
docker swarm init
# Deploy stack
docker stack deploy -c docker-compose.yml myapp
# Scale
docker service scale myapp_web=5
It takes the same Compose files, it’s built into Docker, setup is easy and it spans nodes. The ecosystem is small, the community is smaller still since Docker put its weight behind Kubernetes, and the feature list is shorter.
HashiCorp Nomad
You want orchestration without the Kubernetes complexity, or you run more than just containers.
Nomad is a lightweight orchestrator for containers, VMs and standalone executables, from a single binary. If you run more than containers, that breadth is worth a serious look.
job "my-app" {
datacenters = ["dc1"]
group "web" {
count = 3
task "app" {
driver = "docker"
config {
image = "my-app:latest"
ports = ["http"]
}
resources {
cpu = 500
memory = 256
}
}
}
}
Simpler than Kubernetes, and it schedules more than containers (raw exec, Java, QEMU). It fits well with the rest of the HashiCorp tools, Consul and Vault, and a single binary is easy to deploy. The trade is a smaller ecosystem, fewer managed cloud options, and less of a standard, so fewer tutorials and fewer people who’ve solved your problem before.
Platform-as-a-Service
You want to deploy code, and you don’t want to run infrastructure.
Render does simple deploys with a managed PostgreSQL. Railway is developer-friendly with a good free tier. Fly.io deploys to the edge and performs well. Heroku is the original, and still solid.
# Fly.io example
fly launch
fly deploy
No YAML, no cluster, no ops. The trade is real, and I won’t pretend otherwise. You’re handing control to someone else’s platform, which cuts straight against the sovereignty I usually argue for. For a side project or an early product that needs to ship, that can be a fair deal. Read the pricing page and the terms before you build on it, because the platform can change both and you can’t.
What you get is zero infrastructure management, push code and it runs, and often a lower bill for a small project. What you give up is the exit, the control, and a predictable bill once you grow.
Plain VMs
You don’t need containers.
I mean it. Not everything has to be a container. A boring server with a service manager has kept things online for decades, and it’s about as inspectable as infrastructure gets.
A Python app with systemd:
[Unit]
Description=My App
After=network.target
[Service]
User=app
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/venv/bin/python app.py
Restart=always
[Install]
WantedBy=multi-user.target
Ansible for configuration management. No Docker, no Kubernetes.
Simple, no container overhead, familiar to most engineers, and easy to debug. What you lose is the isolation containers give you, dependency management gets harder, and the box is less reproducible.
Serverless / FaaS
Event-driven workloads, variable load, and nobody wants to manage servers.
AWS Lambda, Google Cloud Functions, Azure Functions, Cloudflare Workers.
// AWS Lambda
export const handler = async (event) => {
return {
statusCode: 200,
body: JSON.stringify({ message: "Hello!" })
};
};
You pay per execution, it scales to zero on its own, and there are no servers to manage. The other side is cold starts, vendor lock-in, a poor fit for long-running processes, and debugging that’s harder than it should be.
Decision framework
So how do you choose, instead of defaulting? Five questions get me most of the way there.
How many services
One to three, and Docker Compose or a PaaS will do. Four to ten, and Swarm, Nomad or Kubernetes all fit. Past ten, Kubernetes starts earning its place.
Whether the load moves
If it doesn’t, almost anything except Kubernetes. If it moves on a schedule, scheduled scaling covers it and you don’t need K8s for that. If it moves and you can’t predict it, Kubernetes or serverless.
What the team knows
No Kubernetes experience? Don’t start with Kubernetes. Some? Managed Kubernetes (EKS, GKE). Lots? Run whatever you want.
What the budget is
Under $100 a month, a VPS, a PaaS or serverless. Between $100 and $1000, it depends on the requirements. Over $1000, Kubernetes starts to make economic sense.
Whether it has to move between clouds
If not, use what the cloud gives you. If it does, Kubernetes is the best option for portability that I know of.
When Kubernetes is the right call
I’ve spent this whole post arguing for restraint, so let me be just as clear about the other side. When you have those problems, Kubernetes is the best tool I know, and I reach for it without hesitation.
Many microservices that scale independently. A team with Kubernetes expertise that can maintain it. Deployment requirements that go past a rolling restart, canary and blue-green. A multi-tenant platform for more than one team. Portability between clouds, which is the reason I run it in my own homelab. The same manifests move from a refurbished mini PC to a cloud and back, and that freedom is worth real complexity to me. And an ecosystem, Prometheus, ArgoCD and friends, that you’ll use rather than just install.
If a few of those describe you, Kubernetes is probably the right call, and everything before this section was about getting you here on purpose.
Why we pick it anyway
So why do teams keep reaching for Kubernetes when something smaller would do? Because Kubernetes is the safe answer in a meeting. Nobody gets second-guessed for choosing the industry standard. It looks good on the architecture diagram and good on a CV. Choosing Docker Compose for a production app feels like admitting you’re not serious, even when it’s the smarter engineering call.
I think that’s backwards. Picking the heaviest tool to look credible is the opposite of understanding your own system. What’s credible is matching the tool to the problem and being able to explain why.
My rule of thumb
Start simple. Add complexity the day you need it, and no sooner.
Docker Compose for development and small production workloads. Kubernetes when you have the problems Kubernetes solves, and not a sprint earlier.
A team that can say “we don’t need Kubernetes” out loud, and say why, understands its own system. That’s the team I’d want to be on.
Wrapping up
Before you adopt Kubernetes, sit with three questions. What problem am I solving? Is Kubernetes the simplest solution to it? Do I have the time, the money and the expertise to keep it healthy?
A “no” to any of those is a nudge to look at the alternatives. Docker Compose, Nomad, a PaaS and plain VMs all exist because each fits a shape of problem better than Kubernetes does.
You don’t have to decide for the whole company at once. Put the next internal tool on a Compose file and a VPS, and see how far it gets before anyone misses the cluster.
I run Kubernetes at home because I have the problems it solves, and because I like it. Most of the teams I talk out of it don’t have those problems yet. When they do, I’ll help them build the cluster.