Dockerfile Best Practices: Build Efficient, Lightweight, Secure Images

Overview

Dockerfile quality directly affects image size, build time, and runtime security. An unoptimized Dockerfile that uses ubuntu:latest and installs many packages can produce a 1.2 GB image with a 15‑minute build, while an optimized Dockerfile can shrink the image to ~80 MB, reduce build time to 2 minutes (or 30 seconds with cache), and cut high‑severity vulnerabilities by roughly 90%.

Key Dockerfile Features

Layer caching : each instruction creates a layer that can be reused if unchanged, turning minute‑scale builds into second‑scale builds.

Multi‑stage builds : separate build and runtime stages so that only the artifacts needed at runtime are copied, reducing image size by 70‑90%.

BuildKit engine (Docker 18.09+) : parallel builds, cache mounts, secret mounts, and 2‑3× speed improvements.

Repeatable builds : identical Dockerfile yields identical image on any host.

Security scanning integration : tools such as Trivy can be run in CI to block images with HIGH or CRITICAL findings.

Environment Requirements

Docker Engine 23.0+ (24.0+ recommended) with BuildKit enabled.

Build host OS: Linux, macOS, or Windows (runtime image must be Linux).

Disk space: ≥20 GB (for cache and intermediate layers).

Memory: ≥4 GB (8 GB+ recommended for Go/Java compilation).

Step‑by‑step Guide

1. Enable BuildKit

# Check Docker version
docker version

# Verify BuildKit (Docker 23.0+ enables it automatically)
docker buildx version

# Manually enable for older versions
export DOCKER_BUILDKIT=1
# Or set in daemon.json
# "features": { "buildkit": true }

# Test BuildKit
docker build --progress=plain -t test-buildkit -f - . <<'EOF'
FROM alpine:3.19
RUN echo "BuildKit is working"
EOF

2. Create .dockerignore

.git
.gitignore
.dockerignore
Dockerfile
docker-compose*.yml
README.md
LICENSE
docs/
tests/
*.md
*.log
*.tmp
*.swp
# Node.js
node_modules/
npm-debug.log
# Java
target/
*.jar
*.class
.gradle/
build/
# Python
__pycache__/
*.pyc
.venv/
venv/
*.egg-info/
# IDE files
.idea/
.vscode/
*.iml
# OS files
.DS_Store
Thumbs.db

Excluding unnecessary files reduces the build context size and speeds up transfers.

3. Base Image Selection

# ❌ Bad examples
FROM ubuntu:22.04          # 77 MB, many unnecessary packages
FROM node:latest           # mutable tag, unpredictable layers

# ✅ Good examples
FROM node:20.11-alpine3.19   # ~5 MB Alpine variant
FROM gcr.io/distroless/java17-debian12  # runtime‑only, no shell
FROM python:3.12-slim-bookworm          # slim Debian, good compatibility

Size comparison (approximate): ubuntu:22.04 ≈ 77 MB, debian:bookworm‑slim ≈ 74 MB, alpine:3.19 ≈ 7 MB, distroless 2‑20 MB, scratch 0 MB.

4. Instruction Order for Cache Utilization

# ❌ Bad: copy source before installing dependencies – any source change forces reinstall
FROM node:20.11-alpine3.19
WORKDIR /app
COPY . .
RUN npm ci --production

# ✅ Good: copy only lock files first, install, then copy source
FROM node:20.11-alpine3.19
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --production
COPY . .

Recommended order: FROM → system dependencies → copy dependency descriptors (package.json, pom.xml, go.mod) → install application dependencies → copy source code → build application → configure runtime parameters (CMD/ENTRYPOINT).

5. RUN Optimization

# Combined apt install with cleanup (Debian/Ubuntu)
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        curl wget ca-certificates && \
    rm -rf /var/lib/apt/lists/*

# Alpine example (tzdata handling)
RUN apk add --no-cache tzdata && \
    cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
    echo "Asia/Shanghai" > /etc/timezone && \
    apk del tzdata

Key flags: --no-install-recommends reduces installed size by 30‑50%; cleaning the package manager cache ( rm -rf /var/lib/apt/lists/* or --no-cache for apk) prevents leftover files from persisting in the image.

6. COPY vs ADD

# Preferred: simple file copy
COPY app.jar /app/
COPY --chown=app:app config/ /app/config/

# When archive extraction is required, use curl+tar instead of ADD
RUN curl -fsSL https://example.com/file.tar.gz -o /tmp/file.tar.gz && \
    tar xzf /tmp/file.tar.gz -C /app/ && \
    rm /tmp/file.tar.gz

ADD automatically extracts archives and can download from URLs, but its implicit behavior can lead to unexpected layers. Use COPY for deterministic builds.

7. Multi‑stage Builds

Go example

# ---------- Builder stage ----------
FROM golang:1.22-alpine AS builder
WORKDIR /build
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
    go build -ldflags="-s -w" -o /app/server ./cmd/server
# Optional UPX compression (≈60% size reduction)
RUN apk add --no-cache upx && upx --best /app/server

# ---------- Runtime stage ----------
FROM scratch
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=builder /app/server /server
EXPOSE 8080
ENTRYPOINT ["/server"]

Final image size ≈ 45 MB (vs 1.2 GB original).

Java example

# ---------- Builder stage ----------
FROM maven:3.9-eclipse-temurin-17 AS builder
WORKDIR /build
COPY pom.xml .
RUN mvn dependency:go-offline -B
COPY src ./src
RUN mvn package -DskipTests -B

# ---------- Runtime stage ----------
FROM eclipse-temurin:17-jre-alpine
RUN addgroup -g 1000 app && adduser -u 1000 -G app -s /bin/sh -D app
WORKDIR /app
COPY --from=builder --chown=app:app /build/target/*.jar app.jar
USER app
EXPOSE 8080
ENTRYPOINT ["java","-jar","app.jar"]

Builder image ≈ 800 MB; final runtime image ≈ 180 MB.

8. CI/CD Integration

Build script (build.sh)

#!/bin/bash
set -euo pipefail
APP_NAME="myapp"
REGISTRY="registry.example.com"
GIT_COMMIT=$(git rev-parse --short HEAD)
GIT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
BUILD_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
VERSION=${CI_COMMIT_TAG:-${GIT_BRANCH}-${GIT_COMMIT}}
IMAGE_TAG="${REGISTRY}/${APP_NAME}:${VERSION}"
IMAGE_LATEST="${REGISTRY}/${APP_NAME}:latest"

echo "Building ${IMAGE_TAG}"

docker build \
    --build-arg BUILD_TIME="${BUILD_TIME}" \
    --build-arg GIT_COMMIT="${GIT_COMMIT}" \
    --build-arg VERSION="${VERSION}" \
    -t "${IMAGE_TAG}" -t "${IMAGE_LATEST}" .

# Security scan – fail on HIGH/CRITICAL
trivy image --exit-code 1 --severity HIGH,CRITICAL "${IMAGE_TAG}"

docker push "${IMAGE_TAG}"
docker push "${IMAGE_LATEST}"

echo "Successfully built and pushed ${IMAGE_TAG}"

BuildKit cache mounts (Maven)

# syntax=docker/dockerfile:1
FROM maven:3.9-eclipse-temurin-17 AS builder
WORKDIR /build
COPY pom.xml .
RUN --mount=type=cache,target=/root/.m2/repository \
    mvn dependency:go-offline -B
COPY src ./src
RUN --mount=type=cache,target=/root/.m2/repository \
    mvn package -DskipTests -B

First build downloads all dependencies (≈ 8 min). Subsequent builds reuse the cache and finish in ~40 seconds.

9. Best Practices & Pitfalls

Cache utilization : place low‑frequency layers (base image, system packages) first. In a typical Node.js project, this can reduce build time from 3 minutes to 15 seconds when only source files change.

Parallel BuildKit stages : independent stages run concurrently, further cutting overall build time.

Layer count : keep < 20 layers; each extra layer adds metadata overhead and can slow pulls.

Security :

Never store secrets in the image; use --mount=type=secret instead of ARG/ENV for credentials.

Pin base‑image versions (e.g., node:20.11.1-alpine3.19) to avoid unexpected updates.

Run containers as a non‑root user (create user with adduser and set USER).

Integrate Trivy or Docker Scout scans in CI and block pushes on HIGH/CRITICAL findings.

High‑availability & multi‑arch :

Use a private Harbor registry with master‑slave replication for fast local pulls.

Persist BuildKit cache in a remote registry via --cache-from and --cache-to so that all CI runners share the same cache.

Build multi‑architecture images with

docker buildx build --platform linux/amd64,linux/arm64 -t myapp:1.0.0 --push .

.

10. Troubleshooting

Image size unexpectedly large : ensure package manager caches are cleaned in the same RUN, and verify that .dockerignore excludes large directories (e.g., node_modules, .git).

Cache always invalidated : avoid COPY . . before dependency installation; copy only lock files first, then source.

Container exits immediately : use exec‑form ENTRYPOINT (JSON array) so the main process runs as PID 1 and receives signals correctly.

Network timeouts during build : build with --network=host or set HTTP_PROXY/HTTPS_PROXY build arguments.

11. Monitoring Metrics

Build duration: normal < 5 min, alert > 10 min.

Final image size: normal < 200 MB, alert > 500 MB.

Cache usage: normal < 20 GB, alert > 50 GB.

Layer count: normal < 20, alert > 40.

High‑severity vulnerabilities: must be zero.

# Example Prometheus alert for slow builds
alert: DockerBuildSlow
  expr: docker_build_duration_seconds > 600
  for: 0m
  labels:
    severity: warning
  annotations:
    summary: "Build time too long"
    description: "Build took {{ $value }} seconds, exceeding 10‑minute threshold."

Summary

Choosing minimal base images, applying multi‑stage builds, ordering instructions for cache efficiency, consolidating RUN commands, and leveraging BuildKit features (cache and secret mounts) together enable dramatically smaller, faster, and more secure container images. Coupled with CI/CD automation, security scanning, and systematic monitoring, these practices form a robust, production‑ready Docker workflow.