Kernel Methods: Support Vector Machines

Support Vector Machines (SVMs) and Kernel methods have found a natural and effective coexistence since their introduction in the early 90s. In this article, we will describe the main concepts that motivate the importance of this relationship. In fact SVMs use kernels for learning linear predictors in high dimensional feature spaces. First, we will describe intuitively how this mechanism is realized, introducing the main concepts and definitions, i.e., maximum margin hyperplane, kernels and non-linearly separable problems. Then the main mathematical issues for linear and nonlinear SVM-based classification will be detailed. We will also introduce some important extensions of the SVMs ideas, by considering the Soft Margin Classification, SVM multi-class classification, SVM clustering, and SVM regression.