Full lesson
Explore the full explanation, examples, and visuals at your own pace.
kubectl get pods, one month later
Your Job finished last week. It's still there. Look at the status column: Completed, on every row. And the ages show 31 days, 30 days, with hundreds more hidden in between. Nothing is wrong with these Pods. They're just finished, and Kubernetes hasn't removed them.
$ kubectl get pods
NAME READY STATUS AGE
test-run-1042-x7k2p 0/1 Completed 31d
test-run-1043-m9q4z 0/1 Completed 31d
test-run-1044-c3v8r 0/1 Completed 30d
... (hundreds more)
test-run-1461-b2c8n 0/1 Completed 2hOne Job per test run, never removed
Our CI pipeline creates a new Job for every test run. At an illustrative fifteen runs a day, that's about 105 Completed Pods after week one, and 420 by week four. The line climbs steadily, because nothing is removing them.
Chart values
| Time | Completed Pods |
|---|---|
| Week 1 | 105 |
| Week 2 | 210 |
| Week 3 | 315 |
| Week 4 | 420 |
Kept on purpose: so you can read logs
A Job is a task that runs to completion, and a Pod is the container group that does the work. When the tests exit, the Pod becomes Completed, but Kubernetes keeps it on purpose, so kubectl logs still works. It deletes nothing unless you tell it to.
Same pipeline, with and without a cleanup timer
Here's the same CI pipeline twice. On the left, with no TTL, every finished run stays, so the namespace fills with Completed Pods week after week. On the right, a cleanup timer removes each finished Job after a set time, so only recent runs remain.
test-run-job.yaml
Here's the fix, and it's one line. Under spec, we add ttlSecondsAfterFinished set to 3600. That's seconds, so about one hour after the Job succeeds or fails, Kubernetes deletes it. The container, the command, and restartPolicy all stay exactly as they were.
apiVersion: batch/v1
kind: Job
metadata:
name: test-run-1461
spec:
ttlSecondsAfterFinished: 3600
template:
spec:
restartPolicy: Never
containers:
- name: tests
image: myapp-tests:latest
command: ["npm", "test"]What the TTL timer does
The timer starts only when the Job finishes, whether it succeeds or fails. After 3600 seconds, the Job is deleted, and its Pod goes with it. Pick a window long enough to read logs. CronJobs differ, since by default they already keep only a few recent runs.
Kubernetes keeps finished Jobs until you tell it when to let go.
Kubernetes keeps finished Jobs until you tell it when to let go. So add ttlSecondsAfterFinished to your pipeline's Job templates.
