How does Service A know that Service B is actually Service B?
I keep coming back to that question because the usual answer is uncomfortable. For years we trusted network location. Traffic from the right IP was legitimate, end of story. Zero trust took that assumption out back and shot it. Now every service has to prove who it is, every single request, no matter where it sits on the network.
That sounds great on a slide. The hard part is the proof. How does a pod actually demonstrate its identity to another pod, without me hand-distributing secrets to everything and then babysitting their rotation forever?
SPIFFE (Secure Production Identity Framework for Everyone) is a standard for service identity. SPIRE is its production-ready implementation. Together they hand every workload a cryptographic identity automatically, with no static secrets to leak. This post builds up from the smallest idea (a name) to the full thing I run in my own cluster.
The thing we keep getting wrong
Before SPIFFE, every approach I tried to service authentication had the same shape: bolt on a credential, then spend the rest of my life managing it. Walk through the options and the pattern shows up every time.
Shared secrets
# Every pod knows the same password
env:
- name: API_KEY
valueFrom:
secretKeyRef:
name: shared-secret
key: api-key
Problems:
- One compromised service exposes all services
- Rotation requires coordinated updates
- No identity distinction between services
Service accounts with tokens
# Kubernetes ServiceAccount tokens
automountServiceAccountToken: true
Better, but:
- Tokens are long-lived
- Work only within Kubernetes
- Limited attestation capabilities
mTLS with static certificates
# Manual certificate management
volumeMounts:
- name: certs
mountPath: /certs
Secure, but:
- Certificate management is operational overhead
- Rotation requires restarts
- Cross-cluster identity is complex
Every row in that list is me doing work a machine should do. That is the friction I want gone. So let me build the alternative up from nothing, one layer at a time.
SPIFFE: The Standard
The simplest version of identity is a name. SPIFFE starts there and adds exactly two more things on top. Three concepts total.
1. SPIFFE ID
A URI that uniquely identifies a workload:
spiffe://trust-domain/path/to/workload
Examples:
spiffe://example.com/ns/production/sa/api-server
spiffe://example.com/k8s/cluster-a/ns/default/pod/frontend-abc123
spiffe://example.com/aws/account-123/region/eu-west-1/instance/i-abc123
The format stays the same everywhere: Kubernetes, VMs, cloud instances, bare metal. One naming scheme for the whole estate, which already removes a category of “wait, how do we identify things over here” problems.
2. SVID (SPIFFE Verifiable Identity Document)
A name on its own proves nothing. Anyone can claim to be api-server. The SVID is the short-lived credential that backs the claim with crypto. It comes in two flavours:
X.509 SVID is a certificate with the SPIFFE ID in the SAN
Subject Alternative Name:
URI: spiffe://example.com/ns/production/sa/api-server
JWT SVID is a signed JWT token
{
"sub": "spiffe://example.com/ns/production/sa/api-server",
"aud": ["spiffe://example.com/ns/production/sa/database"],
"exp": 1690000000
}
3. Workload API
A local API (usually a Unix socket) where workloads fetch their SVIDs:
/run/spire/sockets/agent.sock
This is the part I like most. Workloads never touch private keys directly. They ask the Workload API for an SVID, and key generation plus rotation happens behind that socket. The application code stops caring about certificates entirely, which is one fewer thing for me to get wrong.
So: a name, a credential that proves the name, and a socket to fetch it from. That is the whole standard. Now for the machinery that makes it real.
SPIRE: The Implementation
SPIRE is the engine that hands out those SVIDs. Two components, and the split mirrors how trust actually flows.
SPIRE Server
- Central authority that issues SVIDs
- Stores registration entries (which workloads get which identities)
- Manages trust bundles
SPIRE Agent
- Runs on every node
- Attests workloads (proves they are who they claim)
- Exposes the Workload API
- Caches and rotates SVIDs
flowchart TD
subgraph server["SPIRE Server"]
REG["Registration Entries"]
TRUST["Trust Bundles"]
end
server --> A1["Agent<br/>(Node1)"]
server --> A2["Agent<br/>(Node2)"]
server --> A3["Agent<br/>(Node3)"]
A1 --> W1["Workload API"]
A2 --> W2["Workload API"]
A3 --> W3["Workload API"]
The server is the brain, the agents are the hands on each node. The agent is also the thing that physically vouches for a workload, which matters when we get to attestation later. For now, install it.
Installing SPIRE on Kubernetes
The minimum viable install is two Helm commands.
helm repo add spiffe https://spiffe.github.io/helm-charts-hardened
helm repo update
# Install SPIRE CRDs
helm install spire-crds spiffe/spire-crds \
--namespace spire-system \
--create-namespace
# Install SPIRE
helm install spire spiffe/spire \
--namespace spire-system \
--set global.spire.trustDomain=example.com
That gets you running, but I would never leave it as imperative Helm commands. Anything in my cluster lives in Git and gets reconciled, because I want the cluster to be a function of the repo and not of whatever I typed last Tuesday. Here it is as ArgoCD applications:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: spire-crds
namespace: argocd
spec:
project: default
source:
repoURL: https://spiffe.github.io/helm-charts-hardened
chart: spire-crds
targetRevision: 0.4.0
destination:
server: https://kubernetes.default.svc
namespace: spire-system
syncPolicy:
automated:
prune: true
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: spire
namespace: argocd
spec:
project: default
source:
repoURL: https://spiffe.github.io/helm-charts-hardened
chart: spire
targetRevision: 0.21.0
helm:
values: |
global:
spire:
trustDomain: example.com
clusterName: production
spire-server:
replicaCount: 3
dataStore:
sql:
databaseType: postgres
spire-agent:
socketPath: /run/spire/sockets/agent.sock
destination:
server: https://kubernetes.default.svc
namespace: spire-system
syncPolicy:
automated:
prune: true
Workload Registration
SPIRE will not hand an identity to a workload it has never heard of. Something has to say “a pod matching this shape should get this SPIFFE ID.” You can do that by hand, but the Kubernetes workload registrar turns it into a label-driven rule, which is the version I run:
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterSPIFFEID
metadata:
name: api-server
spec:
spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/sa/{{ .PodSpec.ServiceAccountName }}"
podSelector:
matchLabels:
app: api-server
namespaceSelector:
matchLabels:
spire-enabled: "true"
Any pod matching the selector in a labelled namespace gets the SPIFFE ID. New deployment, same labels, instant identity, no ticket to me. That is the friction reduction I was after at the start.
Using SVIDs in Applications
Now the practical question: your app has an identity available on a socket, so how does it actually use it? There are three ways in, sorted roughly by how much you want to touch your application code.
Option 1: SPIFFE Helper Sidecar
The least invasive option. A sidecar fetches SVIDs and writes them to disk as plain PEM files, and your app reads certificates the way it already does:
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
spec:
template:
spec:
containers:
- name: app
image: my-app:v1.0.0
volumeMounts:
- name: spiffe
mountPath: /spiffe
readOnly: true
env:
- name: TLS_CERT
value: /spiffe/svid.pem
- name: TLS_KEY
value: /spiffe/svid_key.pem
- name: CA_BUNDLE
value: /spiffe/bundle.pem
- name: spiffe-helper
image: ghcr.io/spiffe/spiffe-helper:latest
volumeMounts:
- name: spiffe
mountPath: /spiffe
- name: spire-agent-socket
mountPath: /run/spire/sockets
volumes:
- name: spiffe
emptyDir: {}
- name: spire-agent-socket
hostPath:
path: /run/spire/sockets
type: Directory
Option 2: Native SPIFFE Library
If you own the code, talking to the Workload API directly is cleaner. The library handles rotation in memory, so there is no file on disk to leak or to go stale:
import (
"github.com/spiffe/go-spiffe/v2/workloadapi"
)
func main() {
ctx := context.Background()
// Connect to Workload API
source, err := workloadapi.NewX509Source(ctx)
if err != nil {
log.Fatal(err)
}
defer source.Close()
// Get SVID
svid, err := source.GetX509SVID()
if err != nil {
log.Fatal(err)
}
fmt.Printf("SPIFFE ID: %s\n", svid.ID)
// Use for mTLS
tlsConfig := tlsconfig.MTLSClientConfig(source, source, tlsconfig.AuthorizeID(
spiffeid.RequireID(spiffeid.MustParseSpiffeID("spiffe://example.com/ns/production/sa/database")),
))
}
Option 3: Envoy Sidecar with SDS
When you cannot change the app and you want mTLS handled entirely outside it, put Envoy in front. It pulls SVIDs over the Secret Discovery Service and terminates TLS for you:
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: app
image: my-app:v1.0.0
ports:
- containerPort: 8080
- name: envoy
image: envoyproxy/envoy:v1.28-latest
volumeMounts:
- name: envoy-config
mountPath: /etc/envoy
- name: spire-agent-socket
mountPath: /run/spire/sockets
Envoy config for SDS:
static_resources:
clusters:
- name: spire_agent
connect_timeout: 1s
type: STATIC
http2_protocol_options: {}
load_assignment:
cluster_name: spire_agent
endpoints:
- lb_endpoints:
- endpoint:
address:
pipe:
path: /run/spire/sockets/agent.sock
- name: backend
connect_timeout: 1s
type: STRICT_DNS
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_certificate_sds_secret_configs:
- name: "spiffe://example.com/ns/default/sa/api-server"
sds_config:
api_config_source:
api_type: GRPC
grpc_services:
- envoy_grpc:
cluster_name: spire_agent
validation_context_sds_secret_config:
name: "spiffe://example.com"
sds_config:
api_config_source:
api_type: GRPC
grpc_services:
- envoy_grpc:
cluster_name: spire_agent
Service Mesh Integration
If you already run a service mesh, you do not have to bolt SPIRE on beside it. The mesh can use SPIRE as the thing that issues identities, so you get one source of truth instead of two overlapping certificate systems.
Istio
Istio can use SPIRE as its identity provider:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
meshConfig:
trustDomain: example.com
values:
global:
caAddress: spiffe-csi-driver:443
Linkerd
Linkerd’s identity system is SPIFFE-compatible. Integration is straightforward with the identity controller.
Cross-Cluster Federation
Here is where the single naming scheme pays off. Two clusters with separate trust domains can federate, exchange trust bundles, and then verify each other’s workloads. No shared CA, no copying certs around by hand:
Cluster A (production)
global:
spire:
trustDomain: production.example.com
federation:
enabled: true
bundleEndpoint:
address: spire-bundle.production.example.com
port: 8443
trustDomainBundles:
- endpointURL: https://spire-bundle.staging.example.com:8443
trustDomain: staging.example.com
Cluster B (staging)
global:
spire:
trustDomain: staging.example.com
federation:
enabled: true
bundleEndpoint:
address: spire-bundle.staging.example.com
port: 8443
trustDomainBundles:
- endpointURL: https://spire-bundle.production.example.com:8443
trustDomain: production.example.com
Now services in staging can verify identities from production and the other way round.
Attestation Methods
Time to answer the question I parked earlier: how does SPIRE know a pod claiming to be api-server actually is api-server? It cannot just take the pod’s word for it, or the whole thing is theatre. The answer is attestation, and it happens in two stages.
Node Attestation
# How nodes prove their identity to the server
nodeAttestor:
k8sPsat:
enabled: true
cluster: production
The node proves it’s a legitimate Kubernetes node using projected service account tokens.
Workload Attestation
# How workloads prove their identity to the agent
workloadAttestors:
k8s:
enabled: true
The agent verifies the workload’s container using the Kubernetes API. Two checks, chained: the node proves itself to the server, then the agent on that node proves the workload to itself. No step relies on a secret the workload could have copied from somewhere else.
Authorization with OPA
One honest caveat. SPIRE tells you who is calling. It does not tell you whether they are allowed to. Identity and authorization are different jobs, and SPIRE only does the first. For the second I reach for Kyverno or OPA, feeding the SPIFFE ID straight into the policy:
# OPA policy using SPIFFE IDs
package authz
default allow = false
allow {
# Allow api-server to access database
input.source == "spiffe://example.com/ns/production/sa/api-server"
input.destination == "spiffe://example.com/ns/production/sa/database"
}
allow {
# Allow frontend to access api-server
input.source == "spiffe://example.com/ns/production/sa/frontend"
input.destination == "spiffe://example.com/ns/production/sa/api-server"
}
My Production Setup
Pulling all the layers together, here is the actual config I run. Nothing exotic, just the defaults tightened to my taste:
global:
spire:
trustDomain: infrastructure.internal
clusterName: production
spire-server:
replicaCount: 3
dataStore:
sql:
databaseType: postgres
connectionString: "postgres://spire:password@postgres:5432/spire?sslmode=require"
nodeAttestor:
k8sPsat:
enabled: true
serviceAccountAllowList:
- spire-system:spire-agent
ca:
keyType: ec-p256
ttl: 24h
spire-agent:
socketPath: /run/spire/sockets/agent.sock
workloadAttestors:
k8s:
enabled: true
sds:
enabled: true
defaultBundleName: "null"
defaultAllBundlesName: ROOTCA
spiffe-csi-driver:
enabled: true
Key decisions, and why:
- PostgreSQL backend keeps server state persistent so I can run three replicas and survive a node dying
- EC-P256 keys sit at a sensible point between security and CPU cost
- 24h TTL means SVIDs rotate often, so a leaked credential is worthless by tomorrow
- CSI driver delivers the SVID as a volume, which is cleaner than poking at host paths
Troubleshooting
When something goes quiet, these are the three commands I run before anything else.
Check agent health
kubectl exec -n spire-system -it spire-agent-xxx -- \
/opt/spire/bin/spire-agent healthcheck
List registered workloads
kubectl exec -n spire-system -it spire-server-0 -- \
/opt/spire/bin/spire-server entry show
Verify workload identity
kubectl exec -it my-pod -- \
cat /run/spire/sockets/agent.sock # Check socket exists
kubectl exec -it my-pod -- \
openssl x509 -in /spiffe/svid.pem -text -noout | grep URI
Why This Matters
Go back to the start: a name, a credential, a socket. We built up from that to a system where every service in my cluster proves who it is with crypto, every request, and I never touch a certificate by hand.
Static secrets were always a liability. They leak, they are painful to rotate, and worst of all they never answer the only question that matters in zero trust: who is making this request? SPIFFE and SPIRE answer it with evidence instead of a password.
What I get out of it:
- Automatic identity, so workloads are named without me handing out secrets
- Short-lived credentials, so a stolen SVID expires before it is useful
- Cryptographic proof, so an identity cannot be forged or replayed
- One model everywhere, so a cluster, a VM, and a cloud instance all speak the same language
This matters to me for the same reason self-hosting does. I want to understand the mechanism I am trusting, all the way down, and I want it to keep working when one component falls over. Workload identity that I can reason about is the difference between a breach staying contained and a breach owning everything. That is worth the setup cost.
Identity should be automatic and cryptographically verifiable. SPIFFE and SPIRE give every workload an identity it never has to manage, and that an attacker cannot steal.
