Why AI Server Market Is Shifting: Key Trends and Winners in 2024

AI Server Market Growth in China

According to IDC data, the total domestic server market will maintain roughly 10% annual growth over the next four years, but the AI server segment is expanding much faster, reaching $5 billion in the first half of 2024 with a 63% year‑over‑year increase.

GPU servers continue to dominate the AI segment, accounting for $4.3 billion (86% of the market), while non‑GPU accelerators such as NPU, ASIC, and FPGA grew 182% year‑over‑year to approach $0.7 billion.

IDC projects that by 2028 the Chinese accelerated server market will reach $25.3 billion, with a compound annual growth rate above 20%; the share of non‑GPU servers, represented mainly by ASIC, is expected to rise to nearly 50%.

Global Perspective

Globally, AI server demand is rising as large CSPs and enterprises increase AI compute spending. The Next Platform cites IDC data forecasting the worldwide server market to hit $189.1 billion by 2027 (CAGR 10.2% for 2023‑2027), while Statista predicts the global AI server market will grow ~30% annually, reaching $88 billion by 2027 and approaching parity with general‑purpose servers.

Competitive Landscape in China

In the domestic AI server market, leaders such as Inspur and H3C retain the top positions, with emerging Ascend‑based vendors like Super Fusion and Huakun Zhenyu gaining market share.

IDC statistics for 2023 show that in the x86 server market, Inspur, H3C, Super Fusion, and Ningchang hold 28%, 16%, 11%, and 10% respectively; in the AI server market, their shares are 36%, 19%, 7%, and 14%.

Inspur leads both general‑purpose and AI servers by leveraging a Joint Design Manufacture (JDM) model, securing the majority of cloud and AI server contracts from internet service providers.

H3C, backed by Tsinghua Unigroup, maintains strong positions in government, telecom, and state‑owned enterprises. Super Fusion ranks third in x86 servers and fourth in AI servers.

The Ascend 910B chip has become the primary domestic alternative to Nvidia A100 for AI training, driving rapid growth for Ascend‑based server manufacturers.

Operator Procurement and Ecosystem

Since 2023, major telecom operators have placed large AI server procurements, with China Mobile’s 2024‑2025 tender covering 7,994 units, predominantly awarded to Ascend partners; China Unicom ordered 2,503 AI servers, three of four winners being Ascend partners; China Telecom purchased 4,175 AI servers in October 2023, with Ascend‑based vendors capturing 67% of the market.

The Ascend ecosystem now includes four categories of partners: component partners (e.g., Huakun Zhenyu, Kunlun Technology), application software partners (e.g., Zhipu, Jingying Shuzhi), foundational software partners (e.g., Silicon Flow, Mianbi Intelligent), and ecosystem operators managing AI compute centers.