Tagged articles

9 articles

Page 1 of 1

Machine Learning Algorithms & Natural Language Processing

May 25, 2026 · Artificial Intelligence

Why Large Models Get More Stable with More Edits: Unveiling Lifelong Normalization

The paper analyzes lifelong model editing, showing that Lifelong Normalization (LN) is essential for preventing catastrophic forgetting and model collapse, explains the positive cumulative effect of early edits, and introduces StableEdit with warm‑up and full whitening to achieve robust, million‑scale editing.

Catastrophic ForgettingLifelong Model EditingLifelong Normalization

0 likes · 17 min read

Why Large Models Get More Stable with More Edits: Unveiling Lifelong Normalization

Machine Learning Algorithms & Natural Language Processing

Apr 10, 2026 · Artificial Intelligence

Agent-Dice: Geometric Consensus Filtering Beats Catastrophic Forgetting in LLM Agents

Agent-Dice introduces a geometric consensus filtering and curvature‑based importance weighting framework that disentangles knowledge updates, preventing catastrophic forgetting in large‑language‑model agents while enhancing plasticity, and demonstrates superior stability‑plasticity trade‑offs on GUI and tool‑use benchmarks across multiple base models.

AgentCatastrophic ForgettingGUI

0 likes · 8 min read

Agent-Dice: Geometric Consensus Filtering Beats Catastrophic Forgetting in LLM Agents

DeepHub IMBA

Mar 23, 2026 · Artificial Intelligence

How KgCoOp Uses Knowledge‑Guided Context Optimization to Prevent Prompt Tuning Forgetting

The article analyzes why standard prompt tuning (CoOp) causes catastrophic forgetting in visual‑language models, introduces the KgCoOp framework that adds a knowledge‑guided loss to regularize prompts, and shows through extensive experiments on 11 benchmarks that KgCoOp improves unseen‑class accuracy, harmonic mean, and efficiency while discussing trade‑offs and limitations.

Catastrophic ForgettingKnowledge-guided OptimizationPrompt Tuning

0 likes · 11 min read

How KgCoOp Uses Knowledge‑Guided Context Optimization to Prevent Prompt Tuning Forgetting

Machine Learning Algorithms & Natural Language Processing

Feb 22, 2026 · Artificial Intelligence

What Is On-Policy Distillation? A Deep Dive into On-Policy and Self-Distillation

The article explains On-Policy Distillation, derives its forward and reverse KL gradients, introduces Self‑Distillation where the policy serves as its own teacher, discusses practical implementation tricks such as extra‑knowledge injection, EMA or trust‑region teacher stabilization, and highlights benefits like reduced catastrophic forgetting, fewer Aha moments, and a narrower train‑test gap, especially for larger models.

Catastrophic ForgettingEMAKL divergence

0 likes · 6 min read

What Is On-Policy Distillation? A Deep Dive into On-Policy and Self-Distillation

AI Frontier Lectures

Jan 12, 2026 · Artificial Intelligence

How GraphKeeper Tackles Catastrophic Forgetting in Domain‑Incremental Graph Learning

This article analyzes the GraphKeeper framework, which combines multi‑domain graph decoupling, unbiased ridge‑regression knowledge preservation, and a domain‑aware distribution discriminator to overcome catastrophic forgetting in domain‑incremental graph neural network training, and validates its superiority through extensive experiments and ablations.

Catastrophic ForgettingDomain Incremental LearningGraphKeeper

0 likes · 15 min read

How GraphKeeper Tackles Catastrophic Forgetting in Domain‑Incremental Graph Learning

Baobao Algorithm Notes

Dec 7, 2025 · Artificial Intelligence

Can RL Really Boost LLM Reasoning? A Critical Review of Recent Findings

This article critically examines recent RL‑for‑LLM studies, revealing that reinforcement learning improves search efficiency but does not extend the intrinsic reasoning capabilities of base models, and explores the underlying model‑conditioned optimization bias, comparisons with SFT distillation, and the trade‑off with catastrophic forgetting.

Catastrophic ForgettingLLMSFT

0 likes · 11 min read

Can RL Really Boost LLM Reasoning? A Critical Review of Recent Findings

Baobao Algorithm Notes

Nov 20, 2025 · Artificial Intelligence

Why Reinforcement Learning Preserves LLM Generality Better Than Supervised Fine‑Tuning

The article analyzes why reinforcement learning (RL) fine‑tuning retains a large language model's general abilities better than supervised fine‑tuning (SFT), explaining the off‑policy distribution shift of SFT and the on‑policy data consistency, KL penalty, and trust‑region mechanisms that give RL its anti‑forgetting properties.

Catastrophic ForgettingLLMOn-Policy Data

0 likes · 8 min read

Why Reinforcement Learning Preserves LLM Generality Better Than Supervised Fine‑Tuning

AsiaInfo Technology: New Tech Exploration

Feb 24, 2025 · Artificial Intelligence

Can Multi‑Teacher Distillation Overcome Catastrophic Forgetting in Continual Learning?

This paper proposes a multi‑teacher distillation framework for continual learning that combines active data rehearsal with feature‑decoupled distillation, demonstrating superior performance on PASCAL VOC and COCO benchmarks while mitigating catastrophic forgetting and balancing stability‑plasticity trade‑offs.

AICatastrophic ForgettingKnowledge Distillation

0 likes · 12 min read

Can Multi‑Teacher Distillation Overcome Catastrophic Forgetting in Continual Learning?

Baobao Algorithm Notes

Oct 25, 2024 · Artificial Intelligence

How to Use Importance Sampling for Effective Continue Pretraining of LLMs

Continuing pretraining (CP) bridges pretraining and SFT to inject domain knowledge, but faces catastrophic forgetting; this article explores leveraging importance sampling to balance common and domain data, discusses data selection, annealing strategies, and practical tips for mitigating forgetting while enhancing specialized capabilities.

Catastrophic ForgettingContinue PretrainingImportance Sampling

0 likes · 8 min read

How to Use Importance Sampling for Effective Continue Pretraining of LLMs