DeepSeek‑V4 Deep Dive: Engineering Million‑Token Context Efficiency

The article provides a thorough technical analysis of DeepSeek‑V4, detailing how mixed sparse attention (CSA + HCA), manifold‑constrained hyper‑connections, the Muon optimizer, FP4 quantization, and a suite of infrastructure tricks enable stable training and inference with up to one‑million token contexts while achieving state‑of‑the‑art benchmark results.

CSADeepSeek V4FP4 Quantization

0 likes · 22 min read

DeepSeek‑V4 Deep Dive: Engineering Million‑Token Context Efficiency

Machine Learning Algorithms & Natural Language Processing

Mar 3, 2026 · Artificial Intelligence

Identity Constraint Beats DeepSeek mHC After 150B Tokens: A Surprising Reversal

Extensive experiments on DeepSeek's 1.7B and 8B models reveal that replacing the manifold hyper‑connection (mHC) constraint with a simple identity matrix consistently outperforms the original mHC, improves signal flow stability, and avoids the collapse caused by repeated Sinkhorn‑Knopp projections.

DeepSeekHyper-ConnectionSinkhorn

0 likes · 12 min read

Identity Constraint Beats DeepSeek mHC After 150B Tokens: A Surprising Reversal

Design Hub

Jan 2, 2026 · Artificial Intelligence

DeepSeek’s “Mathematical Tight‑Fit” Tames AI: Constraints Drive Performance Gains

DeepSeek’s new mHC architecture replaces unconstrained hyper‑connections with manifold‑constrained doubly‑stochastic matrices, stabilizing large‑scale training, reducing signal explosion from 3000× to 1.6×, and delivering consistent accuracy improvements across BBH, DROP, GSM8K, and MMLU benchmarks while adding only 6.7% training overhead.

AI training stabilityDeepSeekhyper-connections

0 likes · 10 min read

DeepSeek’s “Mathematical Tight‑Fit” Tames AI: Constraints Drive Performance Gains

AI Insight Log

Jan 1, 2026 · Artificial Intelligence

Can DeepSeek’s mHC Architecture Break ResNet’s Decade-Long Dominance?

DeepSeek’s new paper “mHC: Manifold‑Constrained Hyper‑Connections” proposes a novel architecture that replaces traditional residual connections with mathematically constrained hyper‑connections, showing on a 27B model a modest 6.7 % training‑time increase but significant stability gains and superior performance on BBH, DROP and GSM8K benchmarks.

DeepSeekLLM trainingResNet

0 likes · 8 min read

Can DeepSeek’s mHC Architecture Break ResNet’s Decade-Long Dominance?