Cilium Deep Dive: eBPF Networking for Kubernetes

Kubernetes networking is notoriously complex. CNI plugins, kube-proxy, iptables chains, service meshes — layers upon layers of abstraction that eventually break in ways nobody understands.

Cilium changes this. It uses eBPF to move networking logic into the Linux kernel, bypassing iptables entirely. The result: better performance, more visibility, and network policies that actually make sense.

This is what I run in my clusters. Let me show you why.

What is eBPF?

eBPF (extended Berkeley Packet Filter) lets you run sandboxed programs in the Linux kernel without changing kernel source code or loading kernel modules.

flowchart TD
    subgraph userspace["User Space"]
        APP["Application"]
        CILIUM["Cilium Agent"]
    end

    subgraph kernel["Kernel Space"]
        EBPF["eBPF Programs"]
        NET["Network Stack"]
        HOOKS["Kernel Hooks<br/>(XDP, TC, Socket)"]
    end

    APP --> NET
    CILIUM -->|"loads"| EBPF
    EBPF --> HOOKS
    HOOKS --> NET

Traditional networking uses iptables — a chain of rules evaluated sequentially. eBPF programs are compiled and executed directly in the kernel, with map-based lookups instead of linear rule scanning.

The difference at scale is dramatic. Iptables with 10,000 services? Performance nightmare. Cilium with 10,000 services? Constant-time lookups.

Installing Cilium

Replace your existing CNI with Cilium:

# Install Cilium CLI
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-amd64.tar.gz
sudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin
rm cilium-linux-amd64.tar.gz

# Install Cilium in cluster
cilium install --version 1.15.0

# Verify installation
cilium status --wait

For Helm-based GitOps deployment:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: cilium
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://helm.cilium.io/
    chart: cilium
    targetRevision: 1.15.0
    helm:
      values: |
        kubeProxyReplacement: true
        k8sServiceHost: "10.0.0.1"
        k8sServicePort: "6443"

        hubble:
          enabled: true
          relay:
            enabled: true
          ui:
            enabled: true

        operator:
          replicas: 2

        ipam:
          mode: kubernetes

  destination:
    server: https://kubernetes.default.svc
    namespace: kube-system

kube-proxy Replacement

Cilium can completely replace kube-proxy, implementing Kubernetes Services in eBPF:

# Cilium values
kubeProxyReplacement: true
k8sServiceHost: "10.0.0.1"  # API server IP
k8sServicePort: "6443"

Benefits:

No iptables rules for services
Direct server return for better performance
Socket-level load balancing — decisions made at connect(), not per-packet
Maglev hashing for consistent backend selection

Verify it’s working:

kubectl -n kube-system exec ds/cilium -- cilium status | grep KubeProxyReplacement
# KubeProxyReplacement: True [eth0 10.0.0.10 (Direct Routing)]

Network Policies

Cilium implements standard Kubernetes NetworkPolicy plus extended CiliumNetworkPolicy:

Kubernetes NetworkPolicy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-ingress
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

CiliumNetworkPolicy (Extended)

Cilium’s native policies offer more power:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: api-policy
  namespace: production
spec:
  endpointSelector:
    matchLabels:
      app: api
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: frontend
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              - method: "GET"
                path: "/api/v1/.*"
              - method: "POST"
                path: "/api/v1/orders"

Notice the rules.http — Cilium can enforce L7 policies, allowing specific HTTP methods and paths. No service mesh required.

DNS-Based Policies

Allow traffic to external services by DNS name:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-external-api
spec:
  endpointSelector:
    matchLabels:
      app: backend
  egress:
    - toFQDNs:
        - matchName: "api.stripe.com"
        - matchPattern: "*.amazonaws.com"
      toPorts:
        - ports:
            - port: "443"

Cilium intercepts DNS queries and dynamically updates policies when IPs change.

Cluster-Wide Policies

Apply policies across all namespaces:

apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: deny-external-by-default
spec:
  endpointSelector: {}
  egress:
    - toEntities:
        - cluster
        - host
    - toFQDNs:
        - matchPattern: "*.internal.company.com"
  egressDeny:
    - toEntities:
        - world

This denies internet access by default, allowing only internal traffic unless explicitly permitted.

Hubble: Network Observability

Hubble is Cilium’s observability layer. It shows you what’s actually happening on your network.

# Enable Hubble
hubble:
  enabled: true
  relay:
    enabled: true
  ui:
    enabled: true
  metrics:
    enabled:
      - dns
      - drop
      - tcp
      - flow
      - icmp
      - http

Hubble CLI

# Install Hubble CLI
export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
curl -L --fail --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-amd64.tar.gz
sudo tar xzvfC hubble-linux-amd64.tar.gz /usr/local/bin

# Port-forward to Hubble Relay
cilium hubble port-forward &

# Observe flows
hubble observe --namespace production

# Filter by pod
hubble observe --pod production/api-xyz123

# Filter by verdict
hubble observe --verdict DROPPED

# Filter by HTTP
hubble observe --protocol http --http-status 500

Hubble UI

Access the visual network flow map:

cilium hubble ui
# Opens browser to http://localhost:12000

The UI shows:

Service dependency graph
Real-time traffic flows
Policy verdicts (allowed/dropped)
HTTP request/response details

Hubble Metrics

Export to Prometheus:

hubble:
  metrics:
    serviceMonitor:
      enabled: true
    enabled:
      - dns:query;ignoreAAAA
      - drop
      - tcp
      - flow
      - icmp
      - http

Useful metrics:

hubble_flows_processed_total — Total observed flows
hubble_drop_total — Dropped packets by reason
hubble_http_requests_total — HTTP requests by method, status
hubble_dns_queries_total — DNS queries by type, response

Service Mesh (Without Sidecars)

Cilium can provide service mesh features without sidecar proxies:

# Enable L7 proxy
l7Proxy: true

# Enable mutual TLS
authentication:
  mutual:
    spiffe:
      enabled: true
      trustDomain: cluster.local

Features available:

mTLS between services (using SPIFFE identities)
L7 load balancing with retries
Traffic management (canary, header-based routing)
Observability (L7 metrics, distributed tracing)

The key difference: no sidecar containers. The eBPF dataplane handles everything, reducing resource overhead and latency.

apiVersion: cilium.io/v2
kind: CiliumEnvoyConfig
metadata:
  name: api-routing
spec:
  services:
    - name: api
      namespace: production
  backendServices:
    - name: api-v1
      namespace: production
    - name: api-v2
      namespace: production
  resources:
    - "@type": type.googleapis.com/envoy.config.listener.v3.Listener
      # ... Envoy configuration for traffic splitting

BGP for Bare Metal

Running on bare metal without a cloud load balancer? Cilium speaks BGP:

# Enable BGP
bgpControlPlane:
  enabled: true

apiVersion: cilium.io/v2alpha1
kind: CiliumBGPPeeringPolicy
metadata:
  name: rack-bgp
spec:
  nodeSelector:
    matchLabels:
      rack: rack-1
  virtualRouters:
    - localASN: 65001
      exportPodCIDR: true
      neighbors:
        - peerAddress: "10.0.0.1/32"
          peerASN: 65000

Your pods get routable IPs, advertised via BGP to your network infrastructure.

Bandwidth Management

Cilium can enforce bandwidth limits:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubernetes.io/ingress-bandwidth: "10M"
    kubernetes.io/egress-bandwidth: "10M"
spec:
  containers:
    - name: app
      image: my-app:latest

Implemented in eBPF, not tc rules. More efficient, more predictable.

Encryption

Encrypt all pod-to-pod traffic:

# WireGuard encryption (recommended)
encryption:
  enabled: true
  type: wireguard

# Or IPsec
encryption:
  enabled: true
  type: ipsec

WireGuard is faster and simpler. IPsec if you need compliance certifications that specifically require it.

Verify encryption:

kubectl -n kube-system exec ds/cilium -- cilium status | grep Encryption
# Encryption: Wireguard [NodeEncryption: Disabled, cilium_wg0 (Pubkey: xxx, Port: 51871, Peers: 2)]

Multi-Cluster with Cluster Mesh

Connect multiple Cilium clusters:

# Enable Cluster Mesh on each cluster
cilium clustermesh enable

# Connect clusters
cilium clustermesh connect --destination-context cluster2

Features:

Global services — Same service accessible from any cluster
Cross-cluster network policies — Allow traffic from cluster1 to cluster2 pods
Shared identities — Consistent security policies across clusters

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-from-cluster1
spec:
  endpointSelector:
    matchLabels:
      app: api
  ingress:
    - fromEndpoints:
        - matchLabels:
            io.cilium.k8s.policy.cluster: cluster1
            app: frontend

Troubleshooting

Check Endpoint Status

kubectl -n kube-system exec ds/cilium -- cilium endpoint list

Check Policy Verdicts

kubectl -n kube-system exec ds/cilium -- cilium policy get

Debug Dropped Traffic

hubble observe --verdict DROPPED

Check BPF Maps

kubectl -n kube-system exec ds/cilium -- cilium bpf lb list
kubectl -n kube-system exec ds/cilium -- cilium bpf endpoint list

My Production Configuration

cilium:
  kubeProxyReplacement: true
  k8sServiceHost: "10.0.0.1"
  k8sServicePort: "6443"

  ipam:
    mode: kubernetes

  # Enable Hubble observability
  hubble:
    enabled: true
    relay:
      enabled: true
    ui:
      enabled: true
    metrics:
      enabled:
        - dns:query
        - drop
        - tcp
        - flow
        - http

  # WireGuard encryption
  encryption:
    enabled: true
    type: wireguard

  # L7 policies
  l7Proxy: true

  # Operator HA
  operator:
    replicas: 2

  # Resource limits
  resources:
    limits:
      cpu: 1000m
      memory: 1Gi
    requests:
      cpu: 100m
      memory: 256Mi

Key decisions:

kube-proxy replacement — Simpler, faster
WireGuard encryption — Zero-config encryption
Hubble metrics — Export to Prometheus for alerting
L7 proxy — HTTP-aware policies without service mesh

Why Cilium?

Traditional CNI plugins work. Calico, Flannel, Weave — they all route packets. But they’re building on iptables, a technology from 1998.

Cilium represents the future:

eBPF — Programmable dataplane in the kernel
Identity-based — Policies based on what, not where
Observable — See actual traffic, not just rules
Extensible — Service mesh, BGP, encryption in one tool

For understanding your network, Cilium gives you visibility that iptables-based solutions can’t match. For security, L7-aware policies catch what L3/L4 can’t. For performance, eBPF beats linear rule evaluation.

This is what modern Kubernetes networking looks like.

iptables served us well for decades. But Kubernetes demands something more dynamic, more observable, more powerful. eBPF is that something, and Cilium is how you use it.

What is eBPF?#

Installing Cilium#

kube-proxy Replacement#

Network Policies#

Kubernetes NetworkPolicy#

CiliumNetworkPolicy (Extended)#

DNS-Based Policies#

Cluster-Wide Policies#

Hubble: Network Observability#

Hubble CLI#

Hubble UI#

Hubble Metrics#

Service Mesh (Without Sidecars)#

BGP for Bare Metal#

Bandwidth Management#

Encryption#

Multi-Cluster with Cluster Mesh#

Troubleshooting#

Check Endpoint Status#

Check Policy Verdicts#

Debug Dropped Traffic#

Check BPF Maps#

My Production Configuration#

Why Cilium?#