Machine Heart
May 30, 2026 · Artificial Intelligence
Solving AdamW & Muon Instability: Pion Optimizer Updates Large Models on an Iso‑Spectral Manifold
The Pion optimizer leverages iso‑spectral manifold updates to preserve the spectral norm of weight matrices, eliminating additive‑update instability and enabling stable, efficient training of billion‑parameter LLMs across pre‑training, fine‑tuning, and reinforcement‑learning stages, outperforming AdamW and Muon.
AdamWLarge Language ModelsMuon
0 likes · 14 min read