Anthropic's two May papers reveal that simple SFT/RLHF is insufficient for safe LLMs; inserting a model‑spec mid‑training stage and synthetic‑document fine‑tuning dramatically reduces agentic misalignment, improves data efficiency, and enables models to reason about values before acting.
Anthropic's Claude Mythos Preview outperforms GPT‑5.4, Gemini 3.1 Pro and Opus 4.6 across dozens of AI benchmarks, autonomously discovers thousands of software vulnerabilities, exploits them without human guidance, and raises serious alignment and security concerns for the industry.
The article analyzes the poor generalization of supervised fine‑tuning (SFT) for large language models, reveals its gradient as a high‑variance inverse‑probability policy gradient, proposes a one‑line Dynamic Fine‑Tuning correction, and shows substantial gains on challenging math and offline RL benchmarks.
ChatTS is a time‑series‑native multimodal large language model trained on purely synthetic data, offering superior understanding and reasoning over both real and synthetic time‑series datasets, and outperforming existing LLM baselines across alignment and inference tasks.
This article breaks down the mathematical derivation of Direct Preference Optimization (DPO), showing how it replaces the traditional RLHF‑PPO pipeline by directly training an alignment model from human preference data, eliminating the need for a separate reward model and simplifying the overall training process.
PAI-ChatLearn is an open‑source framework that abstracts and decouples alignment training for large language models, offering flexible resource scheduling, multi‑backend support, and significant speedups—up to 208% for 70B models—while supporting RLHF, DPO, and custom training flows.
This article reviews recent step‑level DPO research, compares it with instance‑level DPO, explains the underlying Monte Carlo Tree Search formulation, and presents the author’s own replication experiments that demonstrate consistent performance gains across multiple LLM sizes on GSM8K and MATH benchmarks.
