DataFunTalk
Feb 2, 2024 · Artificial Intelligence
Utilizing Negative Samples for Knowledge Distillation of Large Language Models
This paper presents a novel framework that leverages negative samples during large language model distillation through three stages—Negative Assistive Training, Negative Calibration Enhancement, and Adaptive Self‑Consistency—demonstrating significant accuracy gains on challenging mathematical reasoning benchmarks and improved generalization to out‑of‑distribution tasks.
Chain-of-ThoughtLLM distillationMachine Learning
0 likes · 13 min read