This guide is part of the Kubernetes Guide — a complete topic cluster covering Kubernetes concepts, operations, and production debugging.
Introduction
Pods are ephemeral. Every time a pod restarts, scales, or gets rescheduled to a new node, it gets a new IP address. Any application that connects directly to a pod IP will break the next time that pod is replaced — which in a production cluster happens constantly.
Kubernetes Services solve this with a single abstraction: a stable virtual IP address and a stable DNS name that always routes to healthy pod endpoints, regardless of how many pods are running, which nodes they are on, or what their current IPs are.
But Services are not just one thing. Kubernetes defines four Service types, each designed for a different connectivity scenario. Understanding when to use each one — and how they actually work under the hood — is essential for building reliable production systems.
This guide covers all four Service types, how kube-proxy implements routing, how Endpoints track live pod IPs, how DNS names are assigned, and the production patterns that make Services reliable under traffic.
1. Why Services Exist — The Pod IP Problem
Day 1:
api-pod-7d9f-xp2k1 10.244.0.4 Running
api-pod-7d9f-mn3q2 10.244.0.5 Running
Day 2 (after rolling update):
api-pod-8b3c-kx9p3 10.244.0.8 Running ← new pod, new IP
api-pod-8b3c-zl2m4 10.244.0.9 Running ← new pod, new IP
If your frontend is configured to call http://10.244.0.4:8080, it breaks the moment the deployment rolls out. Pod IPs are implementation details — they should never appear in application configuration.
DIAGRAM: The pod IP problem — show two states side by side. Left state (Day 1): Frontend pod pointing to api-pod-A (10.244.0.4) and api-pod-B (10.244.0.5) with green checkmarks. Right state (Day 2 after update): same frontend pod still pointing to old IPs 10.244.0.4 and 10.244.0.5 which now show as red X (pods gone). New pods api-pod-C (10.244.0.8) and api-pod-D (10.244.0.9) exist but frontend cannot reach them. Arrow between states labeled “Rolling Update”.
A Service solves this permanently:
Frontend Pod
│
│ http://api-service:8080 ← stable DNS name, never changes
▼
Service (ClusterIP: 10.96.14.200) ← stable virtual IP, never changes
│
├── api-pod-C (10.244.0.8) ← current healthy endpoints
└── api-pod-D (10.244.0.9) ← updated automatically by Endpoints controller
2. How Services Work Under the Hood
Understanding the internal mechanics prevents a lot of debugging confusion.
DIAGRAM: Service internal architecture — show three layers. Top layer: Service object with ClusterIP and selector. Middle layer: Endpoints object listing current pod IPs:ports. Bottom layer: kube-proxy on each node reading endpoints and programming iptables/IPVS rules in the kernel. Show the flow: pod starts → passes readiness probe → Endpoints controller adds IP → kube-proxy updates rules → traffic routes to pod.
The Endpoints Object
Every Service has a corresponding Endpoints object that lists the current healthy pod IPs and ports. The Endpoints controller watches pods and updates this list automatically:
- Pod starts and passes its readiness probe → added to Endpoints → receives traffic
- Pod fails readiness probe → removed from Endpoints → stops receiving traffic
- Pod is deleted → removed from Endpoints immediately
# See current endpoints for a service
kubectl get endpoints api-service -n production
# NAME ENDPOINTS AGE
# api-service 10.244.0.8:8080,10.244.0.9:8080 12d
# If ENDPOINTS shows <none>, your selector does not match any running pods
kube-proxy and iptables
kube-proxy runs on every node. It watches the Endpoints object and programs iptables rules in the Linux kernel so that traffic to the Service’s ClusterIP gets load-balanced across the pod endpoints.
# See the iptables rules kube-proxy creates
iptables -t nat -L KUBE-SERVICES -n | grep <cluster-ip>
iptables -t nat -L KUBE-SVC-<hash> -n # see the load-balancing rules
This is kernel-level routing — no userspace proxy, no application-level load balancer. The overhead is minimal and the performance scales to tens of thousands of Services.
3. Service Type 1 — ClusterIP (Internal)
ClusterIP is the default Service type. It creates a virtual IP that is reachable only from within the cluster. Use it for all internal service-to-service communication.
DIAGRAM: ClusterIP Service diagram — show cluster boundary as a rounded rectangle. Inside: Frontend Pod pointing to ‘api-service ClusterIP: 10.96.14.200’. Service routes to three pod replicas on port 8080. Outside the cluster boundary: ‘External Traffic’ with a red X blocked arrow trying to reach the ClusterIP. Label clearly: ‘Internal only — not reachable from outside cluster’.
DNS name assigned: api-service.production.svc.cluster.local
Short form within same namespace: api-service Cross-namespace form: api-service.production
4. Service Type 2 — NodePort (External via Node IP)
NodePort opens a static port (30000–32767) on every node’s IP address and forwards traffic on that port to the Service. Any external client that can reach a node IP can access the Service.
DIAGRAM: NodePort Service routing — show external client connecting to Node IP (10.240.0.4) on port 32080. Node shows iptables forwarding from NodePort to ClusterIP. ClusterIP routes to two pod replicas. Show all three nodes (Node 1, Node 2, Node 3) each with the same NodePort 32080 open, all forwarding to the same pod endpoints. Label the three routing hops clearly.
spec:
type: NodePort
selector:
app: api
ports:
- port: 8080 # ClusterIP port (internal)
targetPort: 8080 # pod port
nodePort: 32080 # static port on every node (30000–32767)
# omit nodePort to let Kubernetes assign one
When to use NodePort: Rarely in production cloud environments — LoadBalancer is preferred for external access. NodePort is useful for on-premises clusters without a cloud load balancer, or for development environments where you need external access without a full Ingress setup.
Limitations: You must know a node IP, which changes when nodes are replaced. No built-in health checking at the load balancer level. Port range is limited to 30000–32767.
5. Service Type 3 — LoadBalancer (Cloud External Access)
LoadBalancer is the standard way to expose a Service externally in cloud environments. When you create a Service with type: LoadBalancer, the cloud controller manager automatically provisions a cloud load balancer and assigns it a public IP.
DIAGRAM: LoadBalancer Service end-to-end flow — show Internet traffic entering Azure Load Balancer with a public IP. Load balancer health-checks nodes and forwards to NodePorts. NodePorts forward to ClusterIP. ClusterIP routes to pod replicas. Show the automatic provisioning arrow from ‘Cloud Controller Manager’ to the cloud load balancer with label ‘auto-provisioned on Service creation’.
apiVersion: v1
kind: Service
metadata:
name: api-external
namespace: production
annotations:
# Azure-specific: use internal load balancer (private IP, not public)
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
# AWS: use NLB instead of classic ELB
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
type: LoadBalancer
selector:
app: api
ports:
- port: 443
targetPort: 8080
protocol: TCP
When to use LoadBalancer: When you need a dedicated public (or private) IP for a specific Service — typically for non-HTTP protocols (databases, gRPC, MQTT) or when you have a single service that needs its own IP. For HTTP/HTTPS multi-service traffic, Ingress is almost always the better choice.
Cost consideration: Each LoadBalancer Service provisions a separate cloud load balancer, which has a per-hour cost in all cloud providers. In AKS, 10 LoadBalancer Services = 10 Azure Load Balancers. Use Ingress to consolidate HTTP traffic under a single load balancer.
6. Service Type 4 — ExternalName and Headless Services
ExternalName
Maps a Kubernetes Service name to an external DNS name. Useful for referencing external resources (databases, APIs, third-party services) using internal cluster DNS names without hardcoding external addresses in application config.
apiVersion: v1
kind: Service
metadata:
name: external-database
namespace: production
spec:
type: ExternalName
externalName: mydb.postgres.database.azure.com
Now pods can connect to external-database.production.svc.cluster.local and the query resolves to mydb.postgres.database.azure.com. If the external database hostname changes, you update one Service object — not every application that references it.
Headless Services
Setting clusterIP: None creates a headless Service — no virtual IP is assigned. Instead, DNS for the Service returns the individual pod IPs directly. Used by StatefulSets where clients need to address specific pods by identity.
spec:
clusterIP: None # headless — no ClusterIP assigned
selector:
app: kafka
# Regular Service DNS — returns ClusterIP
nslookup kafka-service
# Address: 10.96.44.100
# Headless Service DNS — returns individual pod IPs
nslookup kafka-headless
# Address: 10.244.0.12 (kafka-0)
# Address: 10.244.1.15 (kafka-1)
# Address: 10.244.2.8 (kafka-2)
# StatefulSet pod DNS via headless service
nslookup kafka-0.kafka-headless.messaging.svc.cluster.local
# Address: 10.244.0.12 (always routes to kafka-0 specifically)
DIAGRAM: Headless Service vs regular Service DNS comparison — two side-by-side DNS query flows. Left: regular Service query returns single ClusterIP (10.96.44.100). Right: headless Service query returns three pod IPs directly (10.244.0.12, 10.244.1.15, 10.244.2.8). Show StatefulSet pod stable DNS name format: kafka-0.kafka-headless.messaging.svc.cluster.local.
7. Service Discovery and DNS
Every Service in Kubernetes automatically gets a DNS name from CoreDNS in the format:
<service-name>.<namespace>.svc.cluster.local
Pods are pre-configured with search domains so short names work within the same namespace:
# All of these resolve to the same service from within the production namespace:
curl http://api-service # short name
curl http://api-service.production # namespace-qualified
curl http://api-service.production.svc.cluster.local # full FQDN
# Cross-namespace must use at minimum namespace-qualified form
curl http://api-service.production # from any other namespace
DIAGRAM: This diagram shows how a Kubernetes Service can be resolved using multiple DNS formats. A short name like api-service expands to api-service.production and finally to the full FQDN api-service.production.svc.cluster.local. The /etc/resolv.conf search domains enable this automatic expansion within the same namespace, while CoreDNS resolves all formats to the same ClusterIP.
8. Production Patterns and Considerations
Session Affinity
By default, Services load-balance each request independently. If your application requires a client to always hit the same pod (sticky sessions), use sessionAffinity: ClientIP:
spec:
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800 # 3 hours
This routes all requests from the same client IP to the same pod. Note: it reduces load distribution effectiveness and does not survive pod restarts.
Service for Multiple Ports
A single Service can expose multiple ports — useful for applications that serve HTTP on one port and gRPC or metrics on another:
spec:
ports:
- name: http
port: 8080
targetPort: 8080
- name: grpc
port: 9090
targetPort: 9090
- name: metrics
port: 9100
targetPort: 9100
Port names are required when you define multiple ports — Kubernetes uses them to match Ingress backend port references.
Topology-Aware Routing
In multi-zone clusters, by default a Service routes traffic to pods in any zone — even if a pod in the same zone is available. This adds cross-zone latency and egress cost. Topology-aware routing keeps traffic within the same zone when possible:
metadata:
annotations:
service.kubernetes.io/topology-mode: "Auto"
With this annotation, kube-proxy prefers endpoints in the same zone as the requesting pod. If the local zone has no healthy endpoints, it falls back to other zones.
9. Diagnosing Service Failures
DIAGRAM: Service debugging decision tree — A flowchart for troubleshooting unreachable Services. Start at “Service Not Reachable” and follow decision diamonds:
# Step 1: Check if service exists and has correct ClusterIP
kubectl get service <service-name> -n <namespace>
# Step 2: Check endpoints — are pods being selected?
kubectl get endpoints <service-name> -n <namespace>
# <none> means selector matches no running pods
# Step 3: Verify selector matches pod labels
kubectl get service <service-name> -o yaml | grep -A5 selector
kubectl get pods -n <namespace> --show-labels
# Step 4: Test by ClusterIP (bypasses DNS)
kubectl exec -it <any-pod> -- curl http://<cluster-ip>:<port>
# Step 5: Test by DNS name
kubectl exec -it <any-pod> -- curl http://<service-name>.<namespace>:<port>
# Step 6: Check kube-proxy is running on all nodes
kubectl get pods -n kube-system -l k8s-app=kube-proxy
10. When Things Go Wrong
Endpoints shows <none> — the Service selector does not match any running pods. Compare kubectl get service -o yaml | grep selector with kubectl get pods --show-labels. A single typo in a label key or value means no traffic is routed. See: Production Kubernetes Debugging Handbook
Service reachable by IP but not by name — CoreDNS is failing or misconfigured. Run nslookup kubernetes.default from inside a pod. See: Debugging Kubernetes DNS Issues
LoadBalancer Service stuck in Pending (no external IP) — cloud controller manager could not provision the load balancer. Check cloud provider quotas, subnet availability, and cloud controller manager logs. See: Production Kubernetes Debugging Handbook
Traffic routing to unhealthy pods — readiness probe is not configured or is checking the wrong endpoint. Pods that should be removed from rotation are still in the Endpoints list. See: How to Debug CrashLoopBackOff in Kubernetes
Intermittent failures during rolling updates — old pods removed from endpoints before new pods pass readiness. Add minReadySeconds to your Deployment and ensure readiness probe failureThreshold is tuned correctly.
Summary
Kubernetes Services solve the pod IP instability problem with a stable virtual IP and DNS name:
- ClusterIP — internal only, the default for service-to-service communication
- NodePort — external access via node IP, limited use in cloud production
- LoadBalancer — cloud-provisioned load balancer with public IP, one per Service
- ExternalName — maps cluster DNS name to external hostname
- Headless — returns pod IPs directly via DNS, used by StatefulSets
Under the hood: the Endpoints controller tracks healthy pod IPs, kube-proxy programs kernel routing rules, and CoreDNS handles name resolution. When a Service is not routing traffic, check Endpoints first — it tells you exactly which pods are in rotation.
Continue learning: