Health probes: liveness, readiness and startup
Kubernetes can see if your process is running. It cannot see if your app is actually working. A process can be alive but frozen, or still starting up, or unable to reach its database. Probes are small checks Kubernetes runs again and again to ask your app "are you OK?". The answer decides whether the container is restarted, and whether it gets traffic.
Running is not the same as working
Think of a shop assistant. "Is the assistant in the building?" is not the same question as "Can the assistant serve customers right now?". They might be asleep at the counter, or still unpacking boxes in the morning.
What Kubernetes sees for free
The main process exists. If it crashes, the container is restarted (chapter 7). That is all.
What it cannot see without help
- A frozen app: stuck in an endless loop or deadlock, process still alive.
- An app still loading data, not ready for users yet.
- An app that lost its database connection.
Three probes, three questions
| Probe | Question | If it fails | Runs |
|---|---|---|---|
| Liveness | Are you stuck beyond repair? | The container is killed and restarted | All the time |
| Readiness | Can you serve traffic right now? | The Pod is removed from Service endpoints (chapter 11). Not restarted. Added back once it passes. | All the time |
| Startup | Have you finished starting? | Keeps checking; after too many failures the container is restarted. Liveness and readiness wait until it succeeds. | Only at the start, until it passes once |
How a probe checks
livenessProbe:
httpGet:
path: /healthz
port: 4000Kubernetes calls the URL. Any status from 200 to 399 counts as healthy; anything else, or no answer in time, is a failure. The best choice for web apps like posts.
readinessProbe:
tcpSocket:
port: 5432Only checks that something accepts a connection on the port. Good for databases or anything that does not speak HTTP. It cannot tell "listening but broken" from "fine".
livenessProbe:
exec:
command: ["sh", "-c", "test -f /tmp/healthy"]Runs a command inside the container. Exit code 0 means healthy. Flexible, but each check starts a process, so keep it cheap.
livenessProbe:
grpc:
port: 9000For gRPC servers that implement the standard gRPC health-checking protocol. Built into Kubernetes, no extra tool needed.
The timing knobs
Every probe has the same settings. Click each one:
livenessProbe:
httpGet:
path: /healthz
port: 4000
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3
successThreshold: 1initialDelaySeconds
Wait this long after the container starts before the first check. For slow starters, prefer a startup probe instead of guessing a delay.
periodSeconds
How often to check. Default 10 seconds.
timeoutSeconds
How long to wait for an answer. Default 1 second, which is easy to miss when the app is busy. 2–5 seconds is safer.
failureThreshold
How many failures in a row before acting. Default 3. So with the defaults, a stuck app is restarted after about 30 seconds.
successThreshold
How many passes in a row to count as healthy again. Only matters for readiness; for liveness and startup it must be 1.
Simple maths: time until action ≈ periodSeconds × failureThreshold. A startup probe with periodSeconds: 5 and failureThreshold: 30 allows up to 150 seconds to start.
Probes for the posts service
Give the app two small endpoints. /healthz answers "is this process OK?" and checks nothing outside it. /ready answers "can I serve requests?" and may check what the app truly needs.
// posts/index.js
let ready = false;
let shuttingDown = false; // set to true in your SIGTERM handler
app.get('/healthz', (req, res) => res.sendStatus(200)); // alive = event loop answers
app.get('/ready', (req, res) => {
if (ready && !shuttingDown) return res.sendStatus(200);
res.sendStatus(503); // "not now, send traffic elsewhere"
});
await loadCache(); // slow startup work
ready = true;containers:
- name: posts
image: micro-posts:1.0
ports:
- containerPort: 4000
startupProbe:
httpGet: { path: /healthz, port: 4000 }
periodSeconds: 5
failureThreshold: 30 # up to 150s to start
livenessProbe:
httpGet: { path: /healthz, port: 4000 }
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3
readinessProbe:
httpGet: { path: /ready, port: 4000 }
periodSeconds: 5
failureThreshold: 2Readiness also makes deploys safe: during a rolling update, an old Pod is removed only after a new one is ready (chapter 10). Without a readiness probe, "ready" just means "the process started", even if the app cannot answer yet. And on shutdown, flipping /ready to 503 helps traffic drain away (chapter 14).
Try it: which probes save you?
Pick a problem and a probe setup. Read what Kubernetes does. Find the setup that is green in every case:
Common mistakes
- Liveness probe checks the database connection
A database outage then restarts every Pod over and over. Restarting your app cannot fix the database. Keep liveness about the process itself.
- Liveness with
timeoutSeconds: 1,failureThreshold: 1One slow answer during a traffic spike kills the container, which makes the spike worse. Be patient: a few failures in a row, a few seconds each.
- Readiness checks that the cache has finished loading
Exactly what readiness is for: "do not send me users until I can answer them".
- A big
initialDelaySeconds: 120on liveness for a slow starterIt works, but a frozen app now goes unnoticed for 2 minutes after every start, and if startup ever takes 121s you get a restart loop. Use a startup probe instead.
- Readiness probe, but no liveness probe
A reasonable start. Readiness is the most useful probe. Add liveness only when your app can really get stuck while its process keeps running.
Two extras
Seeing why it failed
$ kubectl describe pod posts-depl-4kq2p
Events:
Warning Unhealthy Liveness probe failed:
HTTP probe failed with statuscode: 500
Normal Killing Container posts failed liveness
probe, will be restartedWhen a container dies, it can also write a short reason to /dev/termination-log. Kubernetes shows that text as the "Message" of the last state in kubectl describe.
Readiness gates
Sometimes something outside the Pod must agree it is ready, for example a cloud load balancer that has finished registering it. A readinessGates entry adds such an extra condition, set by another controller. You will mostly meet this in cloud load balancer setups, not write it yourself.
Recap
- Liveness fails → container restarted. Use it only for "stuck beyond repair".
- Readiness fails → no traffic, no restart. Use it for "not able to serve right now".
- Startup protects slow starters; the other probes wait for it.
- Checks:
httpGet(200–399 is OK),tcpSocket,exec,grpc. - Time to action ≈ period × failureThreshold. Do not make it hair-trigger.
- Never let liveness depend on other services.
Quick check
Show answer
The liveness probe checks the database. Move that check to the readiness probe (or drop it), and keep liveness to "does my process answer".
Show answer
Liveness kills it at about 30 seconds, before it finishes starting. Add a startup probe, for example periodSeconds: 5 and failureThreshold: 24 (up to 120s).
Health probes: common questions
What is the difference between liveness and readiness probes?
A failing liveness probe restarts the container. A failing readiness probe only removes the Pod from Service traffic until it passes again.
Should a liveness probe check the database?
No. A database outage would then restart every Pod again and again. Keep liveness about the process itself and put dependency checks in readiness, if anywhere.