activation functions

IT Services Circle

May 2, 2025 · Artificial Intelligence

Understanding Gradient Vanishing in Deep Neural Networks and How to Mitigate It

The article explains why deep networks suffer from gradient vanishing—especially when using sigmoid or tanh activations—covers the underlying mathematics, compares activation functions, and presents practical techniques such as proper weight initialization, batch normalization, residual connections, and code examples to visualize the phenomenon.

Deep LearningNeural NetworksResNet

0 likes · 7 min read

Understanding Gradient Vanishing in Deep Neural Networks and How to Mitigate It

Cognitive Technology Team

Apr 8, 2025 · Artificial Intelligence

Understanding Neural Networks: Structure, Layers, and Activation

This article explains how a simple neural network can recognize handwritten digits by preprocessing images, organizing neurons into input, hidden, and output layers, using weighted sums, biases, sigmoid compression, and matrix multiplication to illustrate the fundamentals of deep learning.

Deep LearningNeural Networksactivation functions

0 likes · 16 min read

Understanding Neural Networks: Structure, Layers, and Activation

Model Perspective

Dec 5, 2024 · Artificial Intelligence

Choosing the Right Activation Function: Pros, Cons, and Best Practices

Activation functions are crucial for neural networks, providing non‑linearity, normalization, and gradient flow; this article reviews common functions such as Sigmoid, Tanh, ReLU, Leaky ReLU, ELU, Noisy ReLU, Softmax, and Swish, comparing their characteristics, advantages, drawbacks, and guidance for selecting the appropriate one.

Deep LearningNeural Networksactivation functions

0 likes · 10 min read

Choosing the Right Activation Function: Pros, Cons, and Best Practices

DaTaobao Tech

Apr 22, 2024 · Artificial Intelligence

Neural Networks and Deep Learning: Principles and MNIST Example

The article reviews recent generative‑AI breakthroughs such as GPT‑5 and AI software engineers, explains that AI systems are deterministic rather than black boxes, and then teaches neural‑network fundamentals—including activation functions, back‑propagation, and a hands‑on MNIST digit‑recognition example with discussion of overfitting and regularization.

Deep LearningMNISTNeural Networks

0 likes · 17 min read

Neural Networks and Deep Learning: Principles and MNIST Example

Python Programming Learning Circle

Dec 9, 2020 · Artificial Intelligence

Introduction to Artificial Neural Networks and BP Neural Network Implementation with Keras and Scikit-learn

This article introduces artificial neural networks, explains various activation functions, describes common ANN models such as BP, RBF, FNN and LM, and provides step‑by‑step implementation of BP neural networks for classification and regression using Keras Sequential and scikit‑learn’s MLPClassifier/MLPRegressor.

BP Neural NetworkKerasactivation functions

0 likes · 6 min read

Introduction to Artificial Neural Networks and BP Neural Network Implementation with Keras and Scikit-learn

Tencent Cloud Developer

Oct 11, 2018 · Artificial Intelligence

Demystifying Neural Networks: A Mathematical Approach (Part 1)

The article mathematically demystifies neural networks by first illustrating a linear predictor for kilometre‑to‑mile conversion and a basic bug classifier, then exposing the limits of single linear boundaries (e.g., XOR), before introducing artificial neurons, activation functions, and multi‑layer weight‑adjustment training.

Artificial NeuronNeural Networksactivation functions

0 likes · 15 min read

Demystifying Neural Networks: A Mathematical Approach (Part 1)