Chapter 47 of 47 · Under the hood

Building images inside the cluster

So far you built images on your laptop with docker build. In a team, images are usually built by a CI system, and that CI system often runs on Kubernetes. But a Pod has no Docker on it. This chapter shows the tools that build images inside a cluster, safely and without a Docker daemon.

Why build in the cluster?

Build flow inside the clusterSource code in Git is fetched by a build Pod, which builds an image without Docker and pushes it to a registry. A Deployment then pulls the new image. Git repoposts/ + Dockerfile Build Podno Docker daemon,not privileged push Registrymicro-posts:1.5 pull Deploymentposts

Why not just run Docker in a Pod?

You could start the Docker daemon inside a container ("Docker-in-Docker"), or mount the node's container socket into the Pod. Both work, and both are bad ideas on a shared cluster:

Docker-in-Docker

The container must run privileged: it can do almost anything on the node. A malicious Dockerfile could break out and take over the machine.

Mounting the node's socket

Gives the build full control over every container on that node, including other teams' apps. Modern nodes often do not even run Docker; they use containerd (chapter 2).

A clean cluster with Pod Security rules (chapter 42) usually forbids privileged Pods anyway. So we need builders that do not need a daemon or special rights.

Daemonless builders

An image is just a stack of files (layers) plus a small JSON description. You do not need a daemon to produce that; a normal program can do it and upload the result. Pick a tool:

The engine behind modern docker build, which can also run on its own in rootless mode in a Pod. Fast, smart caching, understands every Dockerfile feature.

Example: a build Job

Building is work that finishes, so it fits a Job (chapter 36). Here a Job builds the posts service from Git and pushes it. The registry password comes from a docker-registry Secret (chapter 19). Click the parts:

apiVersion: batch/v1
kind: Job
metadata:
  name: build-posts-1-5
spec:
  backoffLimit: 1
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: build
          image: gcr.io/kaniko-project/executor:latest   # or a maintained fork
          args:
            - --context=git://github.com/acme/blog.git#refs/heads/main
            - --context-sub-path=posts
            - --dockerfile=Dockerfile
            - --destination=ghcr.io/acme/micro-posts:1.5
          volumeMounts:
            - name: registry-login
              mountPath: /kaniko/.docker
      volumes:
        - name: registry-login
          secret:
            secretName: regcred
            items:
              - key: .dockerconfigjson
                path: config.json

What to build, and where to push

Fetch the repo, use the posts folder and its Dockerfile, and push the result as micro-posts:1.5. Exactly what you did with docker build and docker push, in one container.

$ kubectl apply -f build-posts.yaml
$ kubectl logs -f job/build-posts-1-5
INFO Retrieving image manifest node:22-alpine
INFO RUN npm ci --omit=dev
INFO Pushing image to ghcr.io/acme/micro-posts:1.5
$ kubectl set image deployment/posts-depl posts=ghcr.io/acme/micro-posts:1.5

Buildpacks: no Dockerfile at all

Cloud Native Buildpacks look at your source code, notice "this is a Node app with a package.json", and build a good image on their own: right base image, dependencies installed, start command set. No Dockerfile to write or keep up to date.

On your laptop

$ pack build ghcr.io/acme/micro-posts:1.5 \
    --path ./posts \
    --builder paketobuildpacks/builder-jammy-base

Why teams like it

  • Every service gets the same, well-made image layout.
  • Security fix in the base image? Rebuild all apps with one command, often without re-running the full build ("rebase").
  • Less Docker knowledge needed per developer.

The trade-off: less control than a hand-written Dockerfile for unusual needs.

Shipwright: builds as Kubernetes objects

Writing a Job per build gets repetitive. Shipwright adds build objects to Kubernetes (with an operator behind them, chapter 46): you describe what to build once, and start a run whenever you want. The how is a pluggable strategy: BuildKit, Buildah, Buildpacks and others.

  1. apiVersion: shipwright.io/v1beta1
    kind: Build
    metadata:
      name: posts
    spec:
      source:
        type: Git
        git:
          url: https://github.com/acme/blog
        contextDir: posts
      strategy:
        kind: ClusterBuildStrategy
        name: buildkit
      output:
        image: ghcr.io/acme/micro-posts
        pushSecret: regcred

    The Build: "the posts service, from this repo, with BuildKit, pushed here". Written once.

  2. apiVersion: shipwright.io/v1beta1
    kind: BuildRun
    metadata:
      generateName: posts-run-
    spec:
      build:
        name: posts

    A BuildRun is one execution, like pressing "build now". Create one per commit, from CI or by hand.

  3. $ kubectl get buildruns
    NAME            SUCCEEDED   REASON      AGE
    posts-run-x7k2  True        Succeeded   2m

    The operator creates the Pods, streams the logs and reports success, like any other Kubernetes object.

OpenShift has had build objects for a long time (BuildConfig, including "source-to-image"); newer OpenShift versions offer Shipwright-based builds too. CI tools such as Tekton and Argo Workflows run these builders as steps.

Which one?

Recap

Quick check

Your build Pod fails with "unauthorized: authentication required" when pushing. What do you check first?
Show answer

The registry credentials: that the docker-registry Secret exists in the same namespace, is mounted where the builder expects it, and has push rights for that image name.

Building images in Kubernetes: common questions

Why not run Docker-in-Docker in a Kubernetes Pod?

It needs a privileged container, which can take over the node. Daemonless builders such as BuildKit rootless or Buildah build and push images from a normal Pod.