Full lesson
Explore the full explanation, examples, and visuals at your own pace.
Part 1
HPA adds Pods, and Cluster Autoscaler adds nodes. A traffic spike pushes CPU up, HPA asks for more replicas, and some of those Pods may sit Pending with no room. That's when Cluster Autoscaler adds a node. We'll follow one checkout Deployment through it.
Step 1: HPA raises replicas 3 to 6
Metrics Server reports average CPU at 120 percent of requests. HPA's target is 60 percent, so it computes ceiling of 3 times 120 over 60, which is 6. It updates the Deployment's replicas from 3 to 6, and the Deployment creates the new Pods.
Step 2: scheduler fits Pods by requests
The Deployment now wants six Pods, so three new ones appear, each requesting 500m CPU. The scheduler checks Node A and Node B. Each has 0.5 CPU unrequested, so each takes one Pod. The third has nowhere to go. It stays Pending, with an Insufficient cpu event.
Scheduling counts requested CPU, not live usage.
A node can look nearly idle and still reject a Pod. The scheduler counts what's requested, and those requests are already reserved.
A node has half a CPU unrequested but low live usage. A new Pod requests point seven CPU. What happens?
A node has half a CPU unrequested but low live usage. A new Pod requests point seven CPU. What happens?
- The Pod stays Pending on that node
- The Pod schedules because usage is low
- HPA deletes the Pod to save space
A node has half a CPU unrequested but low live usage. A new Pod requests point seven CPU. What happens?
The Pod stays Pending on that node. The scheduler compares the Pod's request with unrequested capacity, not with current usage. Point seven is more than the half CPU left, so the Pod cannot be placed there. That Pending state is exactly the signal Cluster Autoscaler watches for.
- The Pod stays Pending on that node
- The Pod schedules because usage is low
- HPA deletes the Pod to save space
Step 3: Cluster Autoscaler grows the node group
The Autoscaler sees the Pending Pod through the API Server. Before acting, it simulates whether a new node from the group would fit Pod 6. It does, so it asks the Cloud API to grow the group. Node C boots, registers Ready, and that boot usually takes minutes.
Step 4: Pod 6 runs on Node C
Once Node C is Ready, the scheduler retries the Pending Pod and binds Pod 6 there. The kubelet starts the container, and all six checkout Pods are running. Notice that Cluster Autoscaler never placed the Pod. It only added capacity. The scheduler did the placement.
What this means in practice
Requests drive both the HPA math and scheduling, so set them realistically. Caps like maxReplicas or node group max can leave Pods Pending. And node boot lag is real, so keep some headroom or scale earlier.
- Set realistic CPU and memory requests
- Cap with maxReplicas and node group max
- Expect node boot lag during spikes
Pods first, nodes only when Pods cannot fit
Pods first, nodes only when Pods cannot fit. HPA orders more Pods, and Cluster Autoscaler adds nodes when they can't fit. So when you see a Pod stuck in Pending with Insufficient cpu, that's the handoff signal between them.