Alibaba Cloud Server R&D Papers Accepted at DesignCon 2024 and ECTC 2024: Immersion Cooling Impact on PCIe 5.0/6.0 and Long‑Term Reliability of Crystal Oscillators

Recently, the top conferences DesignCon (high‑speed circuit and chip design) and ECTC (electronic components and technology) announced their 2024 paper selections, and Alibaba Cloud’s server R&D team had one paper accepted at each conference. The papers focus on high‑speed interconnect performance of PCIe 5.0/6.0 in air‑cooled and liquid‑immersed environments, and on the long‑term reliability of server electronic components under immersion cooling.

Conference background: DesignCon is the premier global forum for high‑speed signal processing, board‑level, and system design, gathering semiconductor giants, academia, and emerging technology startups. ECTC, co‑organized by IEEE, targets electronic packaging, components, and microsystem technologies, attracting leading industry and research institutions worldwide.

Alibaba Cloud, which operates the world’s largest immersion‑cooled data centers, has now achieved four consecutive years of DesignCon paper selections, demonstrating its strong technical capability in immersion cooling research and its commitment to advancing the technology.

Paper 1 – Immersion‑Cooling Impact on PCIe 5.0 (NRZ) and PCIe 6.0 (PAM4) Link Performance from Measurements

Research background: Next‑generation server platforms are moving from PCIe 5.0 (32 Gbps, NRZ) to PCIe 6.0 (64 Gbps, PAM4). PAM4 signaling is three times more sensitive to noise, and current PCIe 6.0 receivers cannot fully mitigate reflected noise, especially when the surrounding dielectric changes from air to high‑permittivity liquid (Er). This raises significant challenges for high‑speed channel design in immersion‑cooled servers, requiring real‑world measurements rather than simulations.

Research results: Tests were performed on both air‑cooled and immersion‑cooled setups for PCIe 5.0 and PCIe 6.0 using three types of PCIe 6.0 cable‑connector assemblies. In liquid with Er = 2.1, PCIe 5.0 error‑rate showed no noticeable difference between air and liquid, while PCIe 6.0 error‑rate increased by two orders of magnitude (≈100×) compared to air. The findings confirm that PAM4 signaling and more complex topologies suffer greater performance loss, especially under immersion cooling. Recommendations include:

Reduce the number of end‑to‑end high‑speed connectors to enable a unified air‑/liquid‑cooled channel design.

Develop differentiated connectors for multi‑connector topologies, optimizing footprints for both cooling methods.

Adopt separate air‑ and liquid‑cooled architectural solutions where integration is not feasible.

Collaborate with chip vendors to create stronger receivers supporting floating DFE/RXFFE for merged cooling designs.

Paper 2 – Long‑Term Reliability Analysis of Crystal Oscillators under Immersion Cooling with Various Coolants

Research background: The surge in digital and AI workloads drives massive compute demand, raising concerns about energy consumption. Data centers must adopt advanced cooling to meet carbon‑neutral goals. Immersion cooling can achieve PUE as low as 1.09 and power densities up to 100 kW per rack, but the long‑term effects of cooling liquids on passive (e.g., PCBs, coatings) and active (e.g., chips, semiconductors) components need thorough evaluation.

Research results: Crystal oscillators and key server parts were immersed in silicone oil, synthetic oil, and fluorinated fluid at elevated temperatures for extended periods. Accelerated aging, degradation, and failure analyses demonstrated that prolonged exposure can cause physical and chemical changes affecting electrical performance. A complete material‑compatibility test methodology was devised, showing that with proper selection of coolant and design, oscillators can meet a three‑year operational lifetime under immersion cooling, supporting reliable large‑scale deployment.

Since 2015, Alibaba Cloud has continuously advanced immersion‑cooled data center technologies, achieving online‑maintainable high‑reliability architectures, reducing power consumption by over 10 % compared with air cooling, and improving reliability by 50 %. Notable milestones include the 2018 Zhangbei immersion‑cooled cluster, the 2020 Zhejiang‑Hangzhou largest liquid‑cooled data center, and the establishment of an open immersion‑cooling ecosystem through forums and white‑papers.

Aligning with carbon‑neutral strategies, Alibaba Cloud has earned multiple industry awards and will keep collaborating with partners to promote green, high‑performance computing.

/ END /