How Kernel Functions Enable SVMs to Classify Non‑Linear Data

Linear Non‑Separable Support Vector Machine

When the two classes in a training set overlap significantly, the linear support vector classifier used for linearly separable problems becomes ineffective. By introducing a transformation from the input space to a higher‑dimensional Hilbert (feature) space, the original training set is mapped into a new set that is linearly separable in that space.

The separating hyperplane is then found in the Hilbert space, turning the original problem into a quadratic programming problem. Using kernel functions avoids explicit computation in the high‑dimensional feature space; different kernels lead to different algorithms.

The commonly used kernel functions include:

Linear kernel

Polynomial kernel

Radial basis function (RBF) kernel

Sigmoid kernel

Fourier kernel

When the kernel is positive definite, the dual problem becomes a convex quadratic programming problem that is guaranteed to have a solution. Solving this optimization yields the optimal coefficients, from which the classification function is constructed to classify unknown samples.

Reference

Si Shoukuai, Sun Xijing. Python Mathematics Experiments and Modeling