Services — Stable Networking for Dynamic Pods

In the previous chapter, you learned that every Pod gets a unique IP from the cluster CIDR. Here's the follow-up question: Pods are ephemeral — they die, get rescheduled, and receive entirely new IPs. How does any client reliably reach them?

Imagine mailing a letter to a friend who changes their street address every few hours. That is precisely the problem Services solve.

4.2.1 The Service Problem: Why Pod IPs Cannot Be Trusted

When a Deployment replaces a failing Pod, the new Pod gets a brand-new IP. When you scale from 3 to 5 replicas, two new IPs appear; scaling back down, two vanish. Any client hardcoding those Pod IPs breaks immediately.

Kubernetes needs a layer of indirection: a stable virtual IP backed by a dynamic, constantly-updated list of healthy Pod IPs. Enter the Service.

Analogy: PO Boxes and Mail Forwarding

Think of your cluster as an apartment complex where residents (Pods) move in and out constantly. A Service is the building's PO Box system — every PO Box has a stable number that never changes. Behind the scenes, the postal system maintains a forwarding list: "PO Box 304 → deliver to Apartment 2B." When the resident moves, the list updates to "PO Box 304 → now Apartment 5C." The sender never knows.

ClusterIP is an internal PO box — only accessible inside the building. NodePort is a lobby mailbox at each building entrance. LoadBalancer is a dedicated mail facility with its own public address. ExternalName is mail forwarding — the local PO box redirects to an address outside entirely.

The critical insight: the PO box number never changes, even though the apartments behind it do. That is stability amid change.

A Service has three essential ingredients: a stable virtual IP (the ClusterIP), a label selector defining which Pods are included, and an EndpointSlice that Kubernetes continuously updates with the current healthy Pod IPs.

Visual Description: Service Abstraction

The EndpointSlice controller watches Pods. When a matching Pod becomes Ready, its IP is added; when unready or deleted, removed. This reconciliation loop runs continuously.

⚠️ Common Misconception: A Service does not proxy traffic through a central process. kube-proxy programs distributed routing rules (iptables/IPVS) on every node so traffic to the ClusterIP is transparently redirected to a healthy backend Pod. The Service IP is virtual — no network interface actually holds it.

4.2.2 ClusterIP: The Internal Virtual IP

ClusterIP is the default Service type. Kubernetes assigns a virtual IP from the service CIDR (typically 10.96.0.0/12), reachable only within the cluster. kube-proxy — the network agent on every node — maintains the routing. In iptables mode, kube-proxy adds DNAT rules so packets destined for the ClusterIP rewrite to a backend Pod IP. In ipvs mode, it uses the Linux IP Virtual Server framework for higher scale.

apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  type: ClusterIP
  selector:
    app: web
    tier: frontend
  ports:
    - port: 80
      targetPort: 8080

The selector is the critical link — every Pod with matching labels becomes a backend. The port is what the Service exposes; targetPort is what the container listens on. Inspect live endpoints with:

kubectl get endpointslices -l kubernetes.io/service-name=web-service

Set sessionAffinity: ClientIP to pin traffic from the same client IP to the same backend Pod.

Visual Description: kube-proxy iptables Flow

🛑 PAUSE & RECALL — 2 minutes

1. Why is ClusterIP called a "virtual" IP? (Hint: is there a real network interface with that IP?)
2. What component maintains the iptables/IPVS rules for Services on each node?
3. What happens if you remove the selector from a Service spec?

4.2.3 NodePort: Exposing Services on Node IPs

NodePort extends ClusterIP by exposing the service on a static port (30000–32767) on every node's IP. If a node's IP is 10.0.0.5 and the NodePort is 30080, external clients can reach the service at http://10.0.0.5:30080.

apiVersion: v1
kind: Service
metadata:
  name: web-nodeport
spec:
  type: NodePort
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080
      nodePort: 30080

NodePort is useful for development and debugging. It is not recommended for production: node IPs change when nodes are replaced, the port is in a non-standard range, and traffic may traverse an extra network hop.

⚠️ Common Misconception: NodePort opens the port on every node, even those not running a backend Pod. kube-proxy handles routing traffic to a node that does have a healthy Pod.

4.2.4 LoadBalancer: Cloud-Native External Exposure

Setting type: LoadBalancer causes Kubernetes to ask the cloud provider to provision an external load balancer with a public IP. On GKE, this creates a Google Cloud Load Balancer automatically.

apiVersion: v1
kind: Service
metadata:
  name: web-lb
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

After creation, the Service shows an external IP:

kubectl get svc web-lb
# NAME     TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)
# web-lb   LoadBalancer   10.96.3.20     34.120.45.67    80:31234/TCP

The cloud load balancer performs health checks against backend Pods. On GKE, readiness probes determine whether the load balancer considers a backend healthy. If your readiness probe fails, the load balancer removes that Pod from rotation.

GKE Note: GKE LoadBalancer services support annotations for network tier (cloud.google.com/network-tier: Standard) and regional affinity. For container-native load balancing (traffic directly to Pods, not nodes), use the cloud.google.com/neg annotation with Ingress.

4.2.5 ExternalName: DNS CNAME for External Services

ExternalName is the simplest Service type. It creates a DNS CNAME record mapping a Kubernetes service name to an external domain — no ClusterIP, no NodePort, no load balancer.

apiVersion: v1
kind: Service
metadata:
  name: external-db
spec:
  type: ExternalName
  externalName: prod-db.mycompany.example.com

Any Pod in the namespace can connect to external-db.default.svc.cluster.local, and CoreDNS resolves it to prod-db.mycompany.example.com. Application code never hardcodes the real external address. This pattern is powerful for migrations — switch from an internal database to an external managed one by changing only the Service, zero code changes.

4.2.6 Headless Services: Direct Pod IP Resolution

Sometimes you need direct access to individual Pod IPs, not a single virtual IP. A headless Service sets clusterIP: None.

apiVersion: v1
kind: Service
metadata:
  name: db-headless
spec:
  clusterIP: None
  selector:
    app: database
  ports:
    - port: 5432

When you query DNS for db-headless, CoreDNS returns A records for all matching Pod IPs, not a single ClusterIP. For StatefulSets, this is essential. A StatefulSet named db with headless service db-headless gives each Pod a stable DNS name:

db-0.db-headless.default.svc.cluster.local
db-1.db-headless.default.svc.cluster.local
db-2.db-headless.default.svc.cluster.local

This pattern — <pod-name>.<service>.<namespace>.svc.cluster.local — provides stable network identity, making StatefulSets work for clustered databases (PostgreSQL, MongoDB, Cassandra, Kafka). These distributed systems must communicate directly with specific peers; a load-balancing virtual IP would defeat their internal topology awareness.

⚠️ Common Misconception: Headless Services don't exist for performance. They enable client-side service discovery where the client must know all individual backend addresses, not just one virtual IP.

4.2.7 Service Discovery: How Pods Find Services

Kubernetes provides two service discovery mechanisms. The older one injects environment variables at Pod startup:

WEB_SERVICE_SERVICE_HOST=10.96.1.10
WEB_SERVICE_SERVICE_PORT=80

These are set when the Pod starts. If the Service is created after the Pod, the variables won't exist — making this method brittle.

The reliable mechanism is DNS discovery. CoreDNS watches the API Server and creates DNS records for every Service:

<service>.<namespace>.svc.cluster.local

Pods in the same namespace use the short name (web-service). Cross-namespace requires <service>.<namespace>.

Readiness probes directly control endpoint inclusion. A Pod is added to the EndpointSlice only after passing its readiness probe. When a failing Pod receives SIGTERM during a rollout, the kubelet marks it unready, the EndpointSlice controller removes it, and no new traffic arrives — while existing connections drain during terminationGracePeriodSeconds. This is connection draining, essential for zero-downtime deployments.

🤔 TRY BEFORE YOU SEE

You have a StatefulSet cassandra and two Services: a headless Service cassandra (clusterIP: None) and a regular ClusterIP Service cassandra-lb, both pointing to the same Pods.

A client runs nslookup cassandra.default.svc.cluster.local and nslookup cassandra-lb.default.svc.cluster.local.

Predict the DNS output difference before reading the answer.

---

Reveal: The headless Service lookup returns multiple A records — the actual Pod IPs of cassandra-0, cassandra-1, cassandra-2. The regular Service returns one A record — the stable ClusterIP. Headless enables individual Pod discovery; regular Service abstracts them behind a single IP.

🛑 PAUSE & RECALL — 2 minutes

1. Why is DNS-based discovery more reliable than environment variables?
2. What determines whether a Pod appears in a Service's EndpointSlice?
3. When a Pod receives SIGTERM during a rollout, what two things prevent traffic loss?

4.2.8 Visual Description: All Four Service Types Compared

GKE in Practice: Cloud Load Balancer Integration

On GKE, LoadBalancer Services integrate deeply with Google Cloud networking:

1. External passthrough NLB — default for GKE LoadBalancer. Layer 4, no SSL termination.
2. External proxy-based LB — via GCE Ingress or Gateway. Layer 7, SSL, CDN, path routing.
3. Internal LB — add networking.gke.io/load-balancer-type: "Internal" for a private RFC 1918 IP.

Managed Certificate integration works at the Ingress/Gateway layer, not the Service layer. You create a ManagedCertificate, reference it from Ingress annotations, and Google provisions and renews SSL via Let's Encrypt. The Service provides the backend; the Ingress provides the HTTPS frontend.

GKE Note: For production, prefer Ingress or Gateway API over raw LoadBalancer Services. Ingress gives you a single external IP for multiple Services, SSL termination, CDN, and global load balancing — none of which a standalone LoadBalancer provides.

Readiness probes are required for GKE load balancer health checks. Without them, the load balancer may mark Pods healthy before they're actually ready, causing 502 errors.

Lab: LAB-4.2 — Services in Action (60 minutes)

Step 1: Create the Backend Deployment

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx:1.25
        ports:
        - containerPort: 80
        readinessProbe:
          httpGet:
            path: /
            port: 80
EOF

Verify: kubectl get pods -l app=web -o wide

Step 2: Create a ClusterIP Service and Test

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: web-clusterip
spec:
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80
EOF

kubectl run curl-test --rm -it --image=curlimages/curl --restart=Never -- http://web-clusterip

Step 3: Observe the EndpointSlice

kubectl get endpointslices -l kubernetes.io/service-name=web-clusterip

Note the three endpoints matching your Pod IPs.

Step 4: Watch EndpointSlice Update During Pod Churn

In a second terminal, run: kubectl get endpointslices -l kubernetes.io/service-name=web-clusterip -w

In the first terminal: kubectl delete pod -l app=web --all

Watch endpoints remove as Pods terminate, then new ones add as replacements start. This is kube-proxy's world updating in real time.

Step 5: Convert to NodePort

kubectl patch svc web-clusterip -p '{"spec":{"type":"NodePort"}}'
kubectl get svc web-clusterip

Access via: curl http://<NODE_IP>:<NODE_PORT>

Step 6: Create a LoadBalancer Service

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: web-lb
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80
EOF

kubectl get svc web-lb -w
# Wait for EXTERNAL-IP, then: curl http://<EXTERNAL_IP>

GKE Note: Check the GCP Console under Network Services > Load Balancing. The backend service should show your three Pods as healthy.

Step 7: Create an ExternalName Service

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: external-example
spec:
  type: ExternalName
  externalName: kubernetes.io
EOF

kubectl run nslookup-test --rm -it --image=busybox:1.36 --restart=Never -- nslookup external-example.default.svc.cluster.local

Step 8: Test Session Affinity

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: web-sticky
spec:
  sessionAffinity: ClientIP
  selector:
    app: web
  ports:
  - port: 80
EOF

Access repeatedly and observe the same Pod serving requests from the same client IP.

Step 9: Cleanup

kubectl delete deployment web-app
kubectl delete svc web-clusterip web-lb web-sticky external-example

Chapter Summary

Services solve pod ephemerality by providing stable virtual IPs backed by dynamically-updated EndpointSlices. ClusterIP is the default for internal communication. NodePort provides quick external access for development. LoadBalancer integrates with cloud providers for production-grade public IPs. ExternalName abstracts external dependencies behind Kubernetes DNS names. Headless Services (clusterIP: None) enable direct Pod IP discovery for StatefulSets and distributed databases.

The key insight is separation of concerns: the Service provides the stable contract, the EndpointSlice tracks the dynamic reality, and kube-proxy bridges the two. Readiness probes are the gatekeeper — they control which Pods receive traffic and feed directly into GKE load balancer health checks.

📇 KEY CONCEPT CARDS

1. Q: Why can't you use Pod IPs directly for service-to-service communication?
   A: Pod IPs are ephemeral — they change on restart, rescheduling, and scaling. Services provide a stable virtual IP that abstracts the dynamic set of backend Pods.

1. Q: What is the difference between a regular Service and a headless Service?
   A: A regular Service has a ClusterIP acting as a single virtual IP, load-balancing across Pods. A headless Service (clusterIP: None) returns actual Pod IPs via DNS, enabling direct Pod access required by StatefulSets.

1. Q: How does kube-proxy route traffic without a central proxy process?
   A: kube-proxy programs distributed iptables or IPVS rules on every node. The kernel's netfilter rewrites ClusterIP destinations to healthy backend Pod IPs — no user-space proxy involved.

1. Q: What determines whether a Pod appears in a Service's EndpointSlice?
   A: (1) Pod labels must match the Service's selector, and (2) the Pod must be Ready — all containers running AND readiness probe passing.

1. Q: When would you use ExternalName instead of a regular Service?
   A: ExternalName creates a DNS CNAME to an external domain, abstracting external dependencies behind a Kubernetes name. Ideal for third-party integrations and migration scenarios — switch services without code changes.