What happens when CoreDNS is overloaded or down in Kubernetes?

Both Pods say Running, yet the app can't reach its database. Look at the error: a lookup timeout against 10.96.0.10, port 53. The connection never even reached Postgres. The failure happens earlier, while the app is trying to turn a name into an address.

Here's the healthy path. In namespace shop, orders-api wants Postgres. The Service holds a stable ClusterIP, 10.96.45.12 on port 5432, and kube-proxy routes it to postgres-0. But the app only knows postgres.shop.svc.cluster.local. Something has to turn that name into the IP first.

Watch the order. orders-api reads resolv.conf, which points at the kube-dns Service, 10.96.0.10. That Service routes the query to a CoreDNS Pod. CoreDNS answers from Service records it watches through the API, returning 10.96.45.12. Only then does TCP on port 5432 begin. The database connection never starts without that first answer.

With ndots:5, a name with fewer than five dots isn't treated as absolute. So api.stripe.com tries each search suffix first, and each one returns NXDOMAIN. Often that happens for both A and AAAA records. One connection becomes several queries, and that load lands on CoreDNS.

The postgres Service is healthy, and postgres-0 is ready to answer. But orders-api never gets there. Its query goes to kube-dns, lands on a throttled CoreDNS, and times out.

Every connection by Service name first asks CoreDNS for directions. If that step stalls, nothing else runs.

Overloaded CoreDNS answers some queries slowly and drops others, so you see latency spikes and intermittent timeouts across unrelated apps. Unavailable is harsher: no ready endpoints, so every new lookup fails. Either way, it looks like a flaky database or downstream service, even though the real fault is DNS.

An outage can hide at first. Pooled connections to postgres are already open, so they need no lookup. Failures show up on reconnects, rollouts, or new Pods. And resolver caching varies by runtime, so each app behaves differently.

CoreDNS goes down for two minutes. Which orders-api requests are most likely to fail first?

Requests needing a new database connection. An open connection already knows the ClusterIP, so it needs no lookup. Failures appear when the app reconnects, scales, or restarts and must resolve the name again. That delay is why DNS outages often look intermittent rather than total.

Start with the name. Can orders-api's hostname resolve? If not, you've found a DNS problem. If it does, bypass DNS and hit the ClusterIP directly. A failure there points to the network. If that works and the app still fails, look at the database. The first no localizes the fault.

We start a throwaway netshoot Pod in the shop namespace, then run the tree in order. dig on the name times out. nc straight to the ClusterIP on 5432 succeeds, and pg_isready accepts connections. Outputs are illustrative, but this pattern pins the fault on DNS.

Once DNS is the suspect, look at CoreDNS itself. Check replica count, node spread, and restarts, including OOMKilled. Confirm the kube-dns Service has ready endpoints. Then read recent logs for errors, and compare kubectl top against the configured CPU and memory limits.

A failing ready check on 8181 means CoreDNS isn't serving. Upstream i/o timeouts or SERVFAIL point to the forward step. CPU at its limit, throttling, or OOMKills point to capacity. Metrics on 9153 can show latency.

To reduce the blast radius, put a NodeLocal cache next to orders-api so repeat lookups never leave the node. Run several CoreDNS replicas, with enough CPU to avoid throttling. And ask less: a fully qualified name with a trailing dot skips the search-path queries that ndots:5 triggers.

Every connection by Service name first asks CoreDNS for directions. So when orders-api can't reach postgres, don't guess. Test the name, then the ClusterIP, then the database protocol, separately. The first layer that fails tells you where to look.