Support Vector Machines

Module by: Siddharth Gupta, Veena Padmanabhan, Heather Johnston, Grant Lee

Summary: Brief explanation of Support Vector Machines (SVM) and the related optimization problem

A Support Vector Machine (SVM) is a decision-based prediction algorithm which can classify data into several groups. It is based on the concept of decision planes where the training data is mapped to a higher dimensional space and separated by a plane defining the two or more classes of data [1].

A simple example is seen in Figure 1. Squares are data of class one while circles are data of class two. The SVM sets up the decision plane (in this case a simple line) and separates the two classes.

Figure 1: A simple 2-D example for the decision algorithm in an SVM

2-D Example

However, often the data is not distinguishable in two dimensions in which case it is mapped to higher dimensions and the same process is done. An example is shown in Figure 2.

Figure 2: When there is no solution in a lower dimension it can be mapped to a higher dimension and a decision plane is easier to construct

Mapping In Higher Dimensional Space

Support Vector Machine models can be classified into four major groups.

The first two are classification algorithms which minimize different error functions while the second two perform similar algorithms by regression [1].

[1] Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

Comments, questions, feedback, criticisms?

Send feedback

• E-mail the authors of the module, Support Vector Machines

Footer

More about this content: Metadata | Version History

• How to reuse and attribute this content
• How to cite and attribute this content

This work is licensed by Siddharth Gupta, Veena Padmanabhan, Heather Johnston, and Grant Lee under a Creative Commons Attribution License (CC-BY 2.0).

Last edited by Heather Johnston on Dec 23, 2005 1:17 pm US/Central.