Instruction Fine-Tuning

DataFunSummit

Sep 1, 2024 · Artificial Intelligence

Data Management in Large Language Model Training: Overview, Pre‑training, SFT, and Future Challenges

This article surveys data management for large language model training, covering an overview, pre‑training data composition, scaling‑law‑driven quantity control, quality filtering, deduplication, harmful‑content removal, instruction fine‑tuning strategies, dynamic data selection, and emerging research challenges such as bias mitigation, multimodal data handling, and synthetic‑data filtering.

Artificial IntelligenceData qualityInstruction Fine-Tuning

0 likes · 18 min read

Data Management in Large Language Model Training: Overview, Pre‑training, SFT, and Future Challenges

DataFunSummit

Mar 3, 2024 · Artificial Intelligence

Instruction Fine-Tuning Practices for Huawei's Pangu Large Language Model

This presentation details the concepts, methodologies, and experimental results of instruction fine‑tuning for Huawei's Pangu large language model, covering model scale, architecture, training strategies, data quality, parallelism techniques, and case studies on Chinese‑English translation and Thai language adaptation.

AI infrastructureEfficient Fine-TuningInstruction Fine-Tuning

0 likes · 19 min read

Instruction Fine-Tuning Practices for Huawei's Pangu Large Language Model

DataFunSummit

Feb 10, 2023 · Artificial Intelligence

Why ChatGPT Shows Strong General Intelligence: Insights from Andrew Ng’s DeepLearning.AI Article

The article explains how techniques such as Reinforcement Learning from Human Feedback, Instruction Fine‑Tuning, Supervised Fine‑tuning and Chain‑of‑Thought contribute to ChatGPT’s impressive general‑intelligence performance, as analyzed by DeepLearning.AI founder Andrew Ng.

Artificial IntelligenceChain-of-ThoughtChatGPT

0 likes · 2 min read

Why ChatGPT Shows Strong General Intelligence: Insights from Andrew Ng’s DeepLearning.AI Article