Tag

zero

1 views collected around this technical thread.

Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
May 10, 2024 · Artificial Intelligence

GPU Memory Analysis and Distributed Training Strategies

This article explains how GPU memory is allocated during model fine‑tuning, describes collective communication primitives, and compares data parallel, model parallel, ZeRO, pipeline parallel, mixed‑precision, and checkpointing techniques for reducing memory consumption in large‑scale AI training.

GPU memoryMixed PrecisionPipeline Parallel
0 likes · 9 min read
GPU Memory Analysis and Distributed Training Strategies
Model Perspective
Model Perspective
Nov 28, 2023 · Fundamentals

The 5 Greatest Mathematical Symbols and Why They Changed the World

This article explores five of the most iconic mathematical symbols—e, π, i, 0, and =—detailing their definitions, historical origins, and profound impact across calculus, physics, engineering, computer science, and beyond, illustrating how each symbol bridges abstract theory and real‑world applications.

Equalitye constantimaginary unit
0 likes · 7 min read
The 5 Greatest Mathematical Symbols and Why They Changed the World
DataFunSummit
DataFunSummit
Apr 2, 2023 · Artificial Intelligence

Efficient Training of Large Models with the Open‑Source Distributed Framework Easy Parallel Library (EPL)

This article introduces the challenges of scaling deep‑learning model training, explains the design and components of the open‑source Easy Parallel Library (EPL) that unifies data, pipeline, and operator‑split parallelism, and demonstrates its best‑practice results on large‑scale classification, BERT‑large, and massive multimodal models.

EPLLarge-Scale TrainingParallelism
0 likes · 15 min read
Efficient Training of Large Models with the Open‑Source Distributed Framework Easy Parallel Library (EPL)