Optimizing Docker Build Time for a BI Project Using Jenkins and Cache Strategies

Background

The recently taken over BI project builds on a Jenkins build agent take more than 20 minutes even for a single line change, causing significant delays in testing and release cycles.

The long build time impacts daily testing and production releases, prompting a need for optimization.

Optimization Analysis

Understanding the build flow is the first step:

Developers push code to Gitlab , triggering Gitlab Push Events .

Push Events invoke configured Jenkins webhooks.

Jenkins webhooks start the project’s build job.

The build pulls source code, then runs docker build to create an image.

docker builds the image in two stages: an npm scripts step builds the frontend, then the artifacts are copied into an nginx base image.

Only the docker image build step offers meaningful optimization potential; the rest is negligible. Two cache types can be leveraged:

docker layer cache

application layer cache

Docker layer cache reuses image layers when the Dockerfile commands and source files remain unchanged, potentially reducing build time to under 10 seconds for unchanged code. For frequently changing code, application layer cache is needed.

Application layer cache refers to intermediate artifacts such as the node_modules directory, the npm install dependencies, and the .cache directory generated by npm run build . Because each docker build starts with a fresh environment, these directories are recreated every time unless persisted.

Thus the optimization focuses on two actions: enabling application‑level build cache (e.g., webpack cache ) and persisting the node_modules directory so that npm install and npm run build can reuse cached files.

Enabling Application‑Level Build Cache

The project uses React with react‑[email protected] , which internally calls [email protected] . Enabling cache in webpack.config.js is as simple as setting cache: true :

module.exports = {
  // ...
  cache: true
};

Initial local builds without cache take about 13 minutes. After enabling cache, subsequent builds (even with source changes) take roughly 4 minutes, as shown in the screenshots below.

In practice, webpack@4 cache only works in watch and development modes, so the observed speedup actually comes from other caches enabled by react‑scripts , such as babel-loader , eslint-webpack-plugin , and terser-webpack-plugin , which store files in node_modules/.cache .

Further speed gains could be achieved by upgrading to webpack@5 .

Persisting node_modules Directory

To persist node_modules inside the Docker build, BuildKit’s --mount=type=cache feature is required. Preconditions:

Docker version > 18.09.

BuildKit enabled via DOCKER_BUILDKIT=1 .

If unavailable, Jenkins must have read/write access to adjust the build environment.

Modify the Dockerfile to use cache mounts for npm install and npm run build :

FROM node:alpine as builder

WORKDIR /app

COPY package.json /app/

RUN --mount=type=cache,target=/app/node_modules,id=my_app_npm_module,sharing=locked \
    --mount=type=cache,target=/root/.npm,id=npm_cache \
    npm i --registry=https://registry.npm.taobao.org

COPY src /app/src

RUN --mount=type=cache,target=/app/node_modules,id=my_app_npm_module,sharing=locked \
    npm run build

Enable BuildKit in Jenkins by prefixing the build command:

DOCKER_BUILDKIT=1 docker build .

Note: Adding the experimental syntax line # syntax = docker/dockerfile:experimental may cause errors on Docker 20.10 due to external resource loading failures.

Enabling BuildKit also improves log output, making each step’s duration visible.

Optimization Results

After applying the cache strategy, a simulated daily code change triggered the CI pipeline, reducing build time from over 20 minutes to about 4 minutes – an 80% reduction. The changes involved a single line in Jenkins configuration and three lines added to the Dockerfile.

References

[1] Docker documentation: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache

[2] Chinese documentation: https://vuepress.mirror.docker-practice.com/appendix/best_practices/#%E6%9E%84%E5%BB%BA%E7%BC%93%E5%AD%98

[3] Webpack cache options: https://v4.webpack.js.org/configuration/other-options/#cache

[4] BuildKit cache mount: https://vuepress.mirror.docker-practice.com/buildx/buildkit/#run-mount-type-cache

[5] Docker build enhancements: https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-command-line-build-output