1 views collected around this technical thread.
Prof. Yao Quanming presents a comprehensive overview of his research on simplifying deep learning, discussing scaling laws, data, compute and trust bottlenecks, and proposing minimalist approaches in model design, training, and interpretability, with a focus on drug interaction prediction using graph neural networks.
Mathematical modeling thrives on useful approximations rather than flawless precision, as illustrated by Keynes’s economic insights, Box’s famous quote, and real‑world examples like weather forecasting and epidemic models, showing that simplified, “good enough” models often provide more actionable guidance amid complexity and uncertainty.
A recent ETH Zurich paper shows that standard Transformer blocks can be drastically simplified by removing residual connections, LayerNorm, projection and value parameters, and even MLP sub‑block components, achieving up to 16% fewer parameters and comparable training speed and downstream performance on both GPT‑style decoders and BERT models.