training tricks

DataFunSummit

Feb 14, 2023 · Artificial Intelligence

Deep Learning Hyperparameter Tuning and Training Tips: Insights from Zhihu Experts

This article compiles practical deep learning training and hyperparameter tuning advice from Zhihu contributors, covering model debugging, learning‑rate strategies, optimizer choices, data preprocessing, regularization techniques, initialization methods, common pitfalls, recommended research papers, and ensemble approaches.

Deep Learninggradient clippinghyperparameter tuning

0 likes · 13 min read

Deep Learning Hyperparameter Tuning and Training Tips: Insights from Zhihu Experts

DataFunTalk

Dec 4, 2021 · Artificial Intelligence

Practical Deep Learning Training Tricks: Cyclic LR, Flooding, Warmup, RAdam, Adversarial Training, Focal Loss, Dropout, Normalization and More

This article compiles essential deep learning training techniques—including cyclic learning rates, flooding, warmup, RAdam optimizer, adversarial training, focal loss, dropout, batch/group/weight normalization, label smoothing, Wasserstein GAN, skip connections, and weight initialization—providing concise explanations and code snippets for each method.

Deep LearningOptimizationneural networks

0 likes · 11 min read

Practical Deep Learning Training Tricks: Cyclic LR, Flooding, Warmup, RAdam, Adversarial Training, Focal Loss, Dropout, Normalization and More

DataFunTalk

Aug 10, 2021 · Artificial Intelligence

Practical Deep Learning Tricks: Cyclic LR, Flooding, Warmup, RAdam, Adversarial Training, Focal Loss, Dropout, Normalization, ReLU, Group Normalization, Label Smoothing, Wasserstein GAN, Skip Connections, Weight Initialization

This article presents a concise collection of practical deep‑learning techniques—including cyclic learning‑rate, flooding, warmup, RAdam, adversarial training, focal loss, dropout, various normalization methods, ReLU, group normalization, label smoothing, Wasserstein GAN, skip connections, and weight initialization—along with code snippets and references for implementation.

Deep LearningGANadversarial training

0 likes · 8 min read

Practical Deep Learning Tricks: Cyclic LR, Flooding, Warmup, RAdam, Adversarial Training, Focal Loss, Dropout, Normalization, ReLU, Group Normalization, Label Smoothing, Wasserstein GAN, Skip Connections, Weight Initialization