Trainium3 UltraServers Deliver 4.4× Faster AI Training and 40% Better Energy Efficiency

At re:Invent 2025 Amazon Web Services announced the Amazon EC2 Trainium3 UltraServers (Trn3 UltraServers), a vertically integrated system that combines a new 3 nm Trainium3 chip with a purpose‑built software stack. Each UltraServer can house up to 144 Trainium3 chips , delivering up to 4.4× the compute performance of the previous Trainium2 UltraServers.

Benchmarking with OpenAI’s open‑source GPT‑OSS model shows a 3× increase in single‑chip throughput and a 4× faster response time compared with Trainium2, allowing AI applications to scale on a smaller hardware footprint while reducing inference cost per request.

The performance boost is attributed to three architectural advances: (1) a redesigned Trainium3 core that removes bottlenecks for large‑scale models, (2) an optimized inter‑chip interconnect called NeuronSwitch‑v1 that doubles per‑server bandwidth, and (3) an enhanced Neuron Fabric network that cuts chip‑to‑chip latency to under 10 µs. These improvements enable near‑real‑time AI services such as instant decision‑making systems and fluid conversational agents.

Energy efficiency also improves dramatically, with Trainium3 achieving a 40% reduction in power consumption versus earlier generations, a critical factor for large‑scale deployments and data‑center sustainability.

Customers are already seeing tangible benefits. Amazon Bedrock runs production workloads on Trainium3, proving enterprise‑grade readiness. The AI startup Decart reports a 4× increase in frame‑generation speed for real‑time video generation while cutting costs to roughly half of comparable GPU solutions, making high‑throughput generative workloads economically viable.

For massive scaling, the new Amazon EC2 UltraClusters 3.0 can connect thousands of UltraServers, supporting up to one million Trainium chips—ten times the capacity of the prior generation. This scale opens possibilities such as training multimodal models on trillion‑token datasets and serving millions of concurrent inference requests.

Looking ahead, AWS is developing Trainium4, targeting at least 6× FP4 performance , 3× FP8 performance , and 4× memory bandwidth . Trainium4 will incorporate NVIDIA NVLink Fusion and seamless integration with Graviton CPUs and Elastic Fabric Adapter (EFA), creating a flexible, high‑performance rack‑level AI infrastructure that can interoperate with GPU‑based systems.

For further technical details, readers can consult the Trainium documentation and the Trainium Getting‑Started guide linked at the end of the article.