cert-manager: Automatic TLS Certificates in Kubernetes

TL;DR cert-manager watches Certificate resources, fetches the certificate from Let’s Encrypt, your own CA or Vault, stores it in a Secret and renews it 30 days before it expires. I run one DNS-01 ClusterIssuer through Cloudflare and one wildcard certificate, copy that Secret into the namespaces that need it, and let two Prometheus alerts tell me when a renewal is stuck. Nothing left to remember.

For years my certificates renewed the way most people’s do. A calendar reminder, a manual certbot run, and the hope that I’d get to it before the thing expired. It worked, right up to the morning a service started throwing cert errors at me and I had no idea why. The renewal cron had been failing for weeks. Quietly.

That’s the part of manual TLS nobody warns you about. The failure doesn’t announce itself. The cert just expires, usually at a bad moment, and you find out because a browser is yelling at someone. Meanwhile the knowledge of how renewal works lives in one person’s head, and teams skip HTTPS on internal services because wiring it up by hand is annoying enough to put off.

I wanted the cluster to own its certificates so I wouldn’t have to. cert-manager does that. You declare which certificates you need, it handles issuance, renewal and the Kubernetes Secrets they live in, and you stop looking at expiry dates.

It’s one of the first things I install in a new cluster, right after the CNI .

How cert-manager works

The loop is simple once you see it:

1. You create a Certificate resource
2. cert-manager requests a certificate from the issuer (Let’s Encrypt, Vault, etc.)
3. cert-manager completes the challenge (HTTP-01 or DNS-01) to prove you control the domain
4. cert-manager stores the certificate in a Kubernetes Secret
5. Your Ingress or Gateway mounts that Secret for TLS

Renewal happens on its own, 30 days before expiry. No calendar.

Installation

helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set installCRDs=true

The --set installCRDs=true matters. Skip it and cert-manager’s custom resources never get registered, and the errors you get later don’t point at that. If you run everything through GitOps with ArgoCD like I do, the Application looks like this:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: cert-manager
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://charts.jetstack.io
    chart: cert-manager
    targetRevision: v1.14.0
    helm:
      values: |
        installCRDs: true
        prometheus:
          servicemonitor:
            enabled: true
  destination:
    server: https://kubernetes.default.svc
    namespace: cert-manager
  syncPolicy:
    syncOptions:
      - CreateNamespace=true

Issuers

An Issuer tells cert-manager where certificates come from. There are two scopes, and the difference trips people up:

• Issuer: works in one namespace
• ClusterIssuer: works across all namespaces

I default to ClusterIssuer unless I’ve got a real reason to keep a certificate authority inside one namespace. One issuer instead of a copy per namespace.

Let’s Encrypt (ACME)

The setup most people reach for first, free certificates from Let’s Encrypt:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
      - http01:
          ingress:
            class: nginx

Start with the staging issuer. Let’s Encrypt’s rate limits are strict, and if you’re iterating on config you’ll hit them. I learned that the slow way, locked out for a week after fat-fingering a solver config a few times too many. Staging hands you untrusted certs through the exact same flow, so you can break things there first:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-staging-account-key
    solvers:
      - http01:
          ingress:
            class: nginx

HTTP-01 vs DNS-01 challenges

The challenge is how Let’s Encrypt checks that you control the domain before it hands you a cert. There are two ways to prove it, and the one you pick shapes everything downstream.

HTTP-01 has Let’s Encrypt fetch a file from /.well-known/acme-challenge/ on your domain:

solvers:
  - http01:
      ingress:
        class: nginx

It works for anything reachable on port 80 from the public internet. Quick to set up, nothing extra to configure. That reachability is also the catch, because Let’s Encrypt has to be able to fetch the file, and an internal service can’t hand it one.

DNS-01 has Let’s Encrypt look for a DNS TXT record instead:

solvers:
  - dns01:
      cloudflare:
        email: admin@example.com
        apiTokenSecretRef:
          name: cloudflare-api-token
          key: api-token

DNS-01 is the one I end up using most, because it covers what HTTP-01 can’t:

• Wildcard certificates (*.example.com), which HTTP-01 flat out cannot issue
• Internal services that never touch the public internet
• Split-horizon DNS setups

Cloudflare DNS-01 setup

# API Token Secret
apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token
  namespace: cert-manager
type: Opaque
stringData:
  api-token: "your-cloudflare-api-token"
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-cloudflare
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-cloudflare-account
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token

The Cloudflare API token needs Zone:DNS:Edit, scoped to the zones you use. Don’t reach for the global API key here. A scoped token that leaks costs you the DNS of one zone. The global key costs you the account.

Private CA

Public certificates are fine until you have services that should never be reachable from the public internet at all. For those you run your own CA, and the whole trust chain stays inside infrastructure you control:

# Create CA key pair
apiVersion: v1
kind: Secret
metadata:
  name: ca-key-pair
  namespace: cert-manager
type: kubernetes.io/tls
data:
  tls.crt: <base64-encoded-ca-cert>
  tls.key: <base64-encoded-ca-key>
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: internal-ca
spec:
  ca:
    secretName: ca-key-pair

Or let cert-manager create a self-signed CA:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-ca
  namespace: cert-manager
spec:
  isCA: true
  commonName: my-ca
  secretName: my-ca-secret
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: selfsigned-issuer
    kind: ClusterIssuer
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: my-ca-issuer
spec:
  ca:
    secretName: my-ca-secret

HashiCorp Vault

If you already run Vault, its PKI engine makes a solid issuer. Short-lived certs, one audit log, and one place to revoke from:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: vault-issuer
spec:
  vault:
    server: https://vault.vault:8200
    path: pki/sign/my-role
    auth:
      kubernetes:
        role: cert-manager
        mountPath: /v1/auth/kubernetes
        secretRef:
          name: cert-manager-vault-token
          key: token

Requesting certificates

Once an issuer exists, there are two ways to get a certificate out of it. Ask for one explicitly, or let cert-manager infer it from an Ingress. I use both, depending on how much control I want over the details.

Certificate resource

The explicit route. You spell out what you want:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: api-example-com
  namespace: production
spec:
  secretName: api-example-com-tls
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
    - api.example.com
    - api-v2.example.com
  duration: 2160h  # 90 days
  renewBefore: 720h  # Renew 30 days before expiry

Ingress annotation (automatic)

The lazy route, in the good sense. One annotation on an Ingress and cert-manager does the rest:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
    - hosts:
        - api.example.com
      secretName: api-example-com-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 80

cert-manager spots the annotation, generates the Certificate resource for you and fills the Secret. For most app deployments that’s all you need, and the TLS config sits next to the routing config, which is where I’d go looking for it anyway.

Gateway API

Moving to the Gateway API instead of Ingress? Same idea, different resource:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: main
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  gatewayClassName: cilium
  listeners:
    - name: https
      port: 443
      protocol: HTTPS
      hostname: "*.example.com"
      tls:
        mode: Terminate
        certificateRefs:
          - name: wildcard-example-com-tls

Wildcard certificates

A wildcard cert covers every subdomain under one name, so you get one certificate instead of a fresh one per service. That alone keeps you well clear of Let’s Encrypt’s rate limits. The price is DNS-01, since HTTP-01 can’t validate a wildcard:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: wildcard-example-com
  namespace: production
spec:
  secretName: wildcard-example-com-tls
  issuerRef:
    name: letsencrypt-cloudflare
    kind: ClusterIssuer
  dnsNames:
    - "*.example.com"
    - "example.com"

List both *.example.com and example.com. The wildcard doesn’t cover the apex domain, and forgetting that line is how you end up serving a broken cert on your root domain.

Cross-namespace Secrets

Here’s a wrinkle that catches people. A certificate Secret lives in exactly one namespace, and you often need the same cert in several. A Secret in cert-manager can’t be mounted by a pod in production. So how do you share one cert across namespaces without minting a separate one each time? Two ways.

Reflector (external controller)

kubernetes-reflector copies a Secret into the namespaces you list:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: shared-cert
  namespace: cert-manager
  annotations:
    reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
    reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "production,staging"
spec:
  secretName: shared-cert-tls
  # ...

trust-manager

For CA bundles specifically, cert-manager ships its own answer. trust-manager pushes a trust bundle into ConfigMaps across namespaces, which is what you want when the thing you’re sharing is the CA cert rather than a leaf certificate:

helm install trust-manager jetstack/trust-manager \
  --namespace cert-manager

apiVersion: trust.cert-manager.io/v1alpha1
kind: Bundle
metadata:
  name: my-ca-bundle
spec:
  sources:
    - secret:
        name: my-ca-secret
        key: ca.crt
  target:
    configMap:
      key: ca-bundle.crt
    namespaceSelector:
      matchLabels:
        trust-bundle: enabled

Monitoring and troubleshooting

Automation is great until it fails without telling you, which is the exact trap I opened this post with. So cert-manager has to be observable, and a stuck renewal has to become an alert instead of an outage. Here’s how I poke at it when something looks off.

Check certificate status

kubectl get certificates -A
kubectl describe certificate api-example-com -n production

Check certificate requests

kubectl get certificaterequests -A
kubectl describe certificaterequest api-example-com-xyz -n production

Check orders and challenges

kubectl get orders -A
kubectl get challenges -A

# Debug a stuck challenge
kubectl describe challenge api-example-com-xyz-123 -n production

When a cert misbehaves, you walk down the chain. Certificate, then CertificateRequest, then Order, then Challenge. The error is almost always in one of those describe outputs, and it usually surfaces at the Challenge.

Prometheus metrics

Manual checks are for debugging. For the safety net you want metrics. cert-manager exports them, and these are the two alerts I rely on:

# Alert on expiring certificates
- alert: CertificateExpiringSoon
  expr: certmanager_certificate_expiration_timestamp_seconds - time() < 604800
  for: 1h
  labels:
    severity: warning
  annotations:
    summary: "Certificate {{ $labels.name }} expires in less than 7 days"

# Alert on failed renewals
- alert: CertificateRenewalFailed
  expr: certmanager_certificate_ready_status{condition="False"} == 1
  for: 1h
  labels:
    severity: critical
  annotations:
    summary: "Certificate {{ $labels.name }} is not ready"

That first alert is the one that would’ve saved me in my certbot-cron days. Seven days’ warning is plenty of room to fix the real problem before anything user-facing breaks.

Common issues

Challenge stuck pending:

• HTTP-01: Ingress not routing to cert-manager solver
• DNS-01: API credentials wrong or missing permissions

Rate limited:

• Too many certificates requested from Let’s Encrypt
• Use staging issuer for testing
• Consolidate into wildcard certificates

Secret not updating:

• Check Certificate resource status
• Check CertificateRequest and Order resources
• Look at cert-manager logs: kubectl logs -n cert-manager deploy/cert-manager

Best practices

A handful of habits that came out of running this for a while:

1. Use ClusterIssuers

Unless you need namespace isolation, ClusterIssuers cut down duplication:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
# Available to all namespaces

2. Separate staging and production

Always have both issuers:

# letsencrypt-staging for testing
# letsencrypt-prod for production

3. Monitor expiration

Even with automation, monitor as a safety net:

certmanager_certificate_expiration_timestamp_seconds - time() < 604800

4. Use DNS-01 for internal services

If your services aren’t publicly reachable, HTTP-01 won’t work.

5. Standardize Secret names

A convention like ${service}-tls makes it predictable:

secretName: api-example-com-tls

My production setup

Here’s what I run. One DNS-01 ClusterIssuer and one wildcard certificate, and almost everything flows from those two resources:

# ClusterIssuer for Let's Encrypt with Cloudflare DNS
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: certs@example.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token
        selector:
          dnsZones:
            - "example.com"

---
# Wildcard certificate for all services
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: wildcard-example-com
  namespace: cert-manager
spec:
  secretName: wildcard-example-com-tls
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
    - "*.example.com"
    - "example.com"

Why these four:

• DNS-01 via Cloudflare handles everything, wildcards included, so I never have to think about whether a service is publicly reachable
• A wildcard certificate means one cert for every subdomain and effectively zero rate limit worry
• A central namespace keeps the certificate in cert-manager and reflects it out to where it’s needed
• Prometheus alerts so I find out about a problem days before a user does

Why this matters

TLS stopped being optional a while ago. Browsers warn on plain HTTP, APIs reject it, and internal traffic needs encryption if you take zero trust at all seriously.

The deeper reason I care is friction. Anything that depends on me remembering to do a thing will fail at some point, because at some point I’ll forget. Manual certificate renewal is that kind of system with a calendar reminder taped to it. With cert-manager I define a certificate once and the cluster renews it, the same way for every service and every team.

The certbot cron is gone. The wildcard above renews itself 30 days before it expires, and if a renewal ever sticks, the alert reaches me seven days before a browser would.

• Automation
• Cryptography