How We Supercharged Jenkins Performance with Docker Swarm and OpenJ9

Background

Continuous delivery extends classic agile methods, aiming to release reliable software in shorter cycles without affecting users, thereby increasing release efficiency and frequency. The DevOps philosophy emphasizes high‑efficiency continuous delivery to improve product development effectiveness and ensure reliable releases at any time.

The ICBC Software Development Center began a DevOps transformation and built a dedicated continuous delivery system. However, with daily build tasks reaching tens of thousands, the early system could not handle the load, leading to queueing, long wait times, and task failures due to insufficient resources or server crashes.

Analysis revealed severe performance and stability issues on the existing Jenkins server, making optimization urgent. This article describes the practical experience of optimizing the Jenkins server and its environment.

Practice Experience

1. Infrastructure upgrade and JVM memory tuning

To address Jenkins performance deficits, the team upgraded the existing Jenkins master to a high‑I/O, high‑CPU virtual machine, raising the underlying server performance. They also migrated traditional agents to containerized agents, alleviating resource contention and task kills. Because the Docker host had 128 GB memory, unrestricted JVM Xmx of 32 GB caused OOM‑killer incidents; the team limited container memory to around 10 GB.

The master’s JVM was switched to OpenJ9, which consumes less memory and starts faster than HotSpot. Reducing the Xmx parameter also shortened full GC times, decreasing task blockage and improving server responsiveness.

Figure 1: Read/write performance comparison before and after the upgrade (MiB/s)

2. Load balancing with Docker‑Swarm

Beyond hardware upgrades, the center adopted Docker‑Swarm to balance Jenkins agents across multiple nodes, ensuring rational resource distribution. The previous Docker plugin allocated tasks to the first server until saturated before moving to the next, causing “one‑full, many‑idle” situations.

Compared with Kubernetes, Docker‑Swarm suits simple Jenkins agents, offering lightweight clustering, easy scaling, and automatic task isolation with container destruction after completion. This maximizes resource utilization and enhances Jenkins stability.

Figure 2: Docker‑Swarm node structure diagram

Through these measures, the center resolved the throughput limitation of its continuous delivery pipeline, supporting over 36,000 daily builds across product lines—a tenfold increase from the previous 3,000 builds, with markedly improved pipeline efficiency.

Figure 3: Organizational relationship diagram of the optimized continuous delivery system and Jenkins servers

Outlook

As digital transformation deepens, the center will continue to face new challenges in continuous delivery. By integrating industry practices with business needs—focusing on agile collaboration, standardized capability building, and enterprise‑level DevOps toolchains—it aims to overcome technical hurdles and deliver higher‑quality services.