Chapter 36 of 47 · More workload types

Jobs and CronJobs: work that finishes

A Deployment keeps an app running forever. But some work has an end: send this week's digest email, delete old rejected comments, resize uploaded images. A Job runs a task until it has succeeded, then stops. A CronJob creates such a Job on a schedule, like an alarm clock.

Why not a Deployment?

Deployment

"Keep 3 copies running, always." If the process exits, even successfully, Kubernetes starts it again. Perfect for posts and comments, useless for a cleanup script: it would run in a loop forever.

Job

"Run this until it succeeds once." Exit code 0 means done: the Pod stays Completed and nothing restarts. A non-zero exit means "try again", up to a limit.

Same chain as Deployment → ReplicaSet → Pod (chapter 12), but for work that ends.

A Job, line by line

The blog app collects rejected comments. This Job deletes the ones older than 30 days. Click the parts:

apiVersion: batch/v1
kind: Job
metadata:
  name: cleanup-comments
spec:
  backoffLimit: 3
  activeDeadlineSeconds: 600
  ttlSecondsAfterFinished: 86400
  template:
    spec:
      containers:
        - name: cleanup
          image: micro-comments:1.0
          command: ["node", "scripts/delete-old-rejected.js"]
      restartPolicy: Never

`template`

A normal Pod template, like in a Deployment. Here: reuse the comments image and run a script instead of the server. No replicas, no Service: nobody calls this Pod.

When the task fails

$ kubectl apply -f cleanup-job.yaml
$ kubectl get pods
NAME                     READY   STATUS    RESTARTS
cleanup-comments-7xk2p   1/1     Running   0

The Job creates a Pod, which runs the script.

NAME                     READY   STATUS      RESTARTS
cleanup-comments-7xk2p   0/1     Completed   0

$ kubectl get job
NAME               STATUS     COMPLETIONS   DURATION
cleanup-comments   Complete   1/1           14s

Exit code 0. The Pod stays Completed, so you can still read its logs. Nothing restarts.

```
cleanup-comments-7xk2p   0/1   Error   0
```
The script cannot reach the database and exits with 1.

cleanup-comments-7xk2p   0/1   Error   0
cleanup-comments-q9z3b   0/1   Error   0
cleanup-comments-h2v8n   0/1   Error   0

With restartPolicy: Never, each retry is a new Pod, after a growing pause.

$ kubectl get job
NAME               STATUS   COMPLETIONS
cleanup-comments   Failed   0/1
$ kubectl describe job cleanup-comments
  Warning  BackoffLimitExceeded  Job has reached the specified backoff limit

After backoffLimit failures the Job gives up. Read kubectl logs cleanup-comments-7xk2p to see why.

Many Pods at once

Two numbers control how much work a Job does and how fast:

Field	Means	Default
`completions`	How many Pods must succeed before the Job is done	1
`parallelism`	How many Pods may run at the same time	1

```
running:  [pod 1] [pod 2]
done:     0 of 6
```
Two workers start, never more than parallelism.

running:  [pod 3] [pod 2]
done:     1 of 6   ✓ pod 1

As soon as one succeeds, the next one starts.

running:  [pod 5] [pod 6]
done:     4 of 6   ✓ 1 ✓ 2 ✓ 3 ✓ 4

Like a checkout with two tills open: the queue moves two at a time.

```
running:  —
done:     6 of 6   → Job Complete
```
Six successes reached. About three times faster than one worker.

Three ways to split work

spec:
  # completions and parallelism both default to 1
  template: ...

One Pod, one task: a nightly cleanup, a database migration, sending the weekly digest. The most common Job by far.

spec:
  completions: 4
  parallelism: 4
  completionMode: Indexed
  template:
    spec:
      containers:
        - name: resize
          image: image-resizer:1.0
          # each Pod gets JOB_COMPLETION_INDEX = 0, 1, 2 or 3

// pick my quarter of the work
const part = Number(process.env.JOB_COMPLETION_INDEX);   // 0..3
const files = allFiles.filter((_, i) => i % 4 === part);

The work is split into a known number of pieces. With Indexed, every Pod gets its own number, so each knows which piece is its own.

spec:
  parallelism: 5
  # no completions: done when the workers say the queue is empty
  template:
    spec:
      containers:
        - name: worker
          image: email-worker:1.0   # pulls tasks from Redis/RabbitMQ until none are left

Tasks wait in a queue (Redis, RabbitMQ). Five workers take tasks until the queue is empty, then exit 0. You do not need to know the number of tasks in advance.

CronJob: a Job on a schedule

A CronJob is a Job template plus an alarm clock. At each scheduled time it creates a fresh Job:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: weekly-digest
spec:
  schedule: "0 9 * * 1"
  timeZone: "Asia/Kolkata"
  concurrencyPolicy: Forbid
  startingDeadlineSeconds: 300
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  suspend: false
  jobTemplate:
    spec:
      backoffLimit: 2
      template:
        spec:
          containers:
            - name: digest
              image: micro-query:1.0
              command: ["node", "scripts/send-weekly-digest.js"]
          restartPolicy: OnFailure

`schedule`

When to run, in cron format: five fields. 0 9 * * 1 = minute 0, hour 9, any day, any month, Monday. Try it in the explainer below.

Reading a schedule

Five fields: minute hour day-of-month month day-of-week. A * means "every". Type a schedule or use the buttons:

Shortcuts such as @daily, @hourly and @weekly also work in a CronJob.

When runs overlap

The digest usually takes 2 minutes. One Monday the mail server is slow and it takes 2 hours, while a CronJob runs every hour. What should happen at the next tick?

09:00  run A ███████████████████████
10:00        run B ███████████████   ← both run at the same time

The default. Fine when runs are independent; dangerous when two copies would send the same emails twice.

09:00  run A ███████████████████████
10:00        (skipped, A still running)

Never two at once. The next run is skipped. Usually the safest choice.

09:00  run A ████████ ✗ stopped
10:00           run B ███████████

Stop the old run and start the new one. Good when only the latest data matters, such as refreshing a cache.

Safe to run twice

Kubernetes promises that a task runs at least once, not exactly once. A Pod can be killed halfway and retried, and in rare cases a scheduled run can start twice. So write tasks that do no harm if repeated (idempotent):

✗ Risky

// sends again on every retry
for (const user of users) await sendDigest(user);

✓ Safe

for (const user of users) {
  if (await alreadySent(user, thisWeek)) continue;
  await sendDigest(user);
  await markSent(user, thisWeek);
}

Weekly digest email: two runs must never overlap
Refresh a "trending posts" cache every 5 minutes
Ping an external health page every minute; each run is independent

With kubectl

# a one-off Job without YAML
$ kubectl create job hello --image=busybox:1.36 -- echo "hi"

# test a CronJob right now instead of waiting until Monday
$ kubectl create job digest-test --from=cronjob/weekly-digest

$ kubectl get cronjobs
NAME            SCHEDULE    TIMEZONE       SUSPEND   ACTIVE   LAST SCHEDULE
weekly-digest   0 9 * * 1   Asia/Kolkata   False     0        5d

$ kubectl get jobs --watch
$ kubectl logs job/digest-test

# pause and resume the schedule
$ kubectl patch cronjob weekly-digest -p '{"spec":{"suspend":true}}'

Recap

A Job runs Pods until a task succeeds; a CronJob creates Jobs on a schedule.
restartPolicy must be Never or OnFailure; backoffLimit caps retries.
activeDeadlineSeconds limits run time; ttlSecondsAfterFinished cleans up.
completions + parallelism run many Pods; Indexed gives each its piece.
Set timeZone, pick a concurrencyPolicy, and make tasks safe to run twice.

Quick check

You put your cleanup script in a Deployment. What happens?

Show answer

It runs, exits successfully, and the Deployment immediately starts it again, forever, because a Deployment's job is to keep containers running. Use a Job or CronJob.

A CronJob with schedule: "0 9 * * *" sends emails at 14:30 in India. Why?

Show answer

No timeZone is set, so 09:00 means UTC, which is 14:30 in India. Add timeZone: "Asia/Kolkata".

Jobs and CronJobs: common questions

What is the difference between a Job and a CronJob?

A Job runs Pods until a task succeeds. A CronJob creates a new Job on a schedule, such as every night at 3.

Why does my CronJob run at the wrong hour?

Without timeZone the schedule uses the cluster time zone, usually UTC. Set timeZone, for example Asia/Kolkata.

Jobs and CronJobs: work that finishes

Why not a Deployment?

Deployment

Job

A Job, line by line

`template`

`restartPolicy`

`backoffLimit`

`activeDeadlineSeconds`

`ttlSecondsAfterFinished`

When the task fails

Many Pods at once

Three ways to split work

CronJob: a Job on a schedule

`schedule`

`timeZone`

`concurrencyPolicy`

`startingDeadlineSeconds`

History limits

`suspend`

`jobTemplate`

Reading a schedule

When runs overlap

Safe to run twice

✗ Risky

✓ Safe

With kubectl

Recap

Quick check

Jobs and CronJobs: common questions