BestHub
Sign in
Tagged articles
11 articles
Page 1 of 1
Machine Heart
Machine Heart
May 29, 2026 · Artificial Intelligence

DiffusionOPD: A New Online Policy Distillation Paradigm for Multi‑Task Diffusion Models

DiffusionOPD introduces a unified on‑policy distillation framework for diffusion models that decouples single‑task online policy exploration from multi‑task capability integration, training expert teachers per task and distilling their skills into a single student model, achieving faster convergence and higher performance across composition, OCR, and aesthetic tasks.

KL divergencePPOdiffusion models
0 likes · 8 min read
DiffusionOPD: A New Online Policy Distillation Paradigm for Multi‑Task Diffusion Models
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Apr 12, 2026 · Artificial Intelligence

Deep Dive into Forward vs Reverse KL Divergence: When to Use Which?

The article explains the definitions, properties, and asymmetric nature of KL divergence, compares Forward KL (mean‑seeking) and Reverse KL (mode‑seeking) through bimodal examples, and provides practical guidelines for choosing between them based on sampling and probability‑evaluation capabilities in machine‑learning tasks.

Forward KLKL divergenceMachine Learning
0 likes · 10 min read
Deep Dive into Forward vs Reverse KL Divergence: When to Use Which?
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Feb 22, 2026 · Artificial Intelligence

What Is On-Policy Distillation? A Deep Dive into On-Policy and Self-Distillation

The article explains On-Policy Distillation, derives its forward and reverse KL gradients, introduces Self‑Distillation where the policy serves as its own teacher, discusses practical implementation tricks such as extra‑knowledge injection, EMA or trust‑region teacher stabilization, and highlights benefits like reduced catastrophic forgetting, fewer Aha moments, and a narrower train‑test gap, especially for larger models.

Catastrophic ForgettingEMAKL divergence
0 likes · 6 min read
What Is On-Policy Distillation? A Deep Dive into On-Policy and Self-Distillation
AI Algorithm Path
AI Algorithm Path
May 10, 2025 · Artificial Intelligence

Master KL Divergence: Definitions, Properties, and Real‑World Applications

This article explains the Kullback‑Leibler (KL) divergence for discrete and continuous distributions, outlines its non‑negativity and asymmetry, walks through a uniform‑distribution example, provides a simple Python demonstration, and discusses key applications in variational autoencoders, reinforcement‑learning policy optimization, and other machine‑learning contexts.

KL divergenceMachine Learninginformation theory
0 likes · 7 min read
Master KL Divergence: Definitions, Properties, and Real‑World Applications
Code DAO
Code DAO
May 6, 2022 · Fundamentals

Information Theory Foundations for Machine Learning and Deep Learning

The article explains Shannon information content, entropy, cross‑entropy, KL‑divergence, conditional entropy and mutual information, illustrating each concept with coin‑flip and dice examples, visual formulas, and discusses their roles as loss functions and evaluation metrics in machine‑learning models.

KL divergenceMachine Learningcross entropy
0 likes · 8 min read
Information Theory Foundations for Machine Learning and Deep Learning
Code DAO
Code DAO
Dec 20, 2021 · Artificial Intelligence

Exploring Latent Space with a Variational Autoencoder in TensorFlow

This article explains the theory behind variational autoencoders, details their KL‑divergence loss, provides a complete TensorFlow implementation, and demonstrates reconstruction, latent‑space visualization, and novel image generation through sampling and interpolation.

KL divergenceLatent SpacePython
0 likes · 13 min read
Exploring Latent Space with a Variational Autoencoder in TensorFlow
Code DAO
Code DAO
Dec 10, 2021 · Artificial Intelligence

Understanding Variational Autoencoders: From Dimensionality Reduction to Generative Modeling

This article explains the principles of variational autoencoders, starting with dimensionality reduction techniques such as PCA and standard autoencoders, highlighting their limitations for data generation, and then detailing VAE's regularized latent space, variational inference, re‑parameterization, and loss formulation.

Generative ModelsKL divergenceVAE
0 likes · 18 min read
Understanding Variational Autoencoders: From Dimensionality Reduction to Generative Modeling
21CTO
21CTO
Feb 7, 2018 · Artificial Intelligence

Demystifying Entropy: From Basic Concepts to Cross‑Entropy and KL Divergence

This article explains entropy, joint entropy, conditional entropy, and related measures such as KL divergence and cross‑entropy, using intuitive coin‑flip examples and mathematical formulas to show how they quantify uncertainty and information in probability distributions.

KL divergenceMachine Learningcross entropy
0 likes · 14 min read
Demystifying Entropy: From Basic Concepts to Cross‑Entropy and KL Divergence
Architecture Digest
Architecture Digest
Feb 3, 2018 · Artificial Intelligence

Understanding Entropy, Joint Entropy, Conditional Entropy, Relative Entropy, and Cross Entropy

This article explains the concepts of entropy, joint entropy, conditional entropy, relative entropy (KL divergence) and cross‑entropy, illustrating their definitions, mathematical formulas, intuitive interpretations, and relationships through simple probability examples and visual diagrams.

KL divergenceMachine Learningcross entropy
0 likes · 14 min read
Understanding Entropy, Joint Entropy, Conditional Entropy, Relative Entropy, and Cross Entropy
Qunar Tech Salon
Qunar Tech Salon
Mar 14, 2015 · Artificial Intelligence

Common Distance and Similarity Measures in Machine Learning and Data Mining

This article reviews the most frequently used distance and similarity formulas in machine learning and data mining, explaining their definitions, mathematical properties, practical examples, and when each metric is appropriate for measuring differences between data points or probability distributions.

Cosine SimilarityKL divergenceMachine Learning
0 likes · 13 min read
Common Distance and Similarity Measures in Machine Learning and Data Mining