1 / 32

An Introduction to Support Vector Machines

An Introduction to Support Vector Machines. Presenter: Celina Xia University of Nottingham. Outline. Maximizing the Margin Linear SVM and Linear Separable Case Primal Optimization Problem Dual Optimization Problem Non-Separable Case Non-Linear Case Kernel Functions Applications.

ida
Download Presentation

An Introduction to Support Vector Machines

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. An Introduction to Support Vector Machines Presenter: Celina Xia University of Nottingham

  2. Outline • Maximizing the Margin • Linear SVM and Linear Separable Case • Primal Optimization Problem • Dual Optimization Problem • Non-Separable Case • Non-Linear Case • Kernel Functions • Applications

  3. Margin Any of these separating lines would be fine.. ..but which is best?

  4. Margin Margin: the width that the boundary could be increased by before hitting a datapoint. margin margin Wide margin Narrow margin Decisionboundary

  5. SVMs reckon… Decision boundary The decision boundary with maximal margin deliver the best generalization ability. margin w Orientation of the decision boundary

  6. 2 maximize w w 2 minimize 2 SVM—Linear Separable • Objective: • maximize the margin wTx+b=1 wTx+b=0 wTx+b=-1

  7. 2 maximize w w 2 minimize 2 SVM—Linear Separable • Objective: • maximize the margin wTx+b=1 wTx+b=0 wTx+b=-1 Support Vectors

  8. SVM—Linear Separable

  9. The Lagrangian trick Moving the constraint to objective function Lagrangian:

  10. The Lagrangian trick Optimality conditons:

  11. The Lagrangian trick Replace with Solving:

  12. SVM—Linear Separable

  13. SVM—Linear Separable Lagrangian: Optimality conditons:

  14. Dual Optimization Problem

  15. Linearly Non-separable Case(Soft Margin Optimal Hyperplane)

  16. Linearly Non-separable Case(Soft Margin Optimal Hyperplane)

  17. Lagrangian

  18. Lagrangian

  19. Dual Optimization Problem

  20. Problems with linear SVM What if the decison function is not a linear?

  21. Problems with linear SVM

  22. Dual Optimization Problem

  23. Dual Optimization Problem

  24. Kernel Functions • A kernel function K enables the explicit mapping of input data without exact knowledge of • Gaussian radial basis function (RBF) is one of widely-used kernel functions

  25. Dual Optimization Problem replace the dot product of the inputs with the kernel function

  26. Dual Optimization Problem

  27. Some kernel functions • Polynomial type: • Polynomial type: • Gaussian radial basis function (RBF) • Multi-Layer Perceptron:

  28. Two-Spiral Pattern Given 194 training data points on X-Y plane: 97 of class “ red circle’’ and another 97 of class “blue cross ’’.Question: how to distinguish between these two spirals ?

  29. What’s the challenge? A proper learning of these 194 training data points A piece of cake for a variety of methods. After all, it’s just a limited number of 194 points Correct assignment of an arbitrary data point on XY plane to the right “spiral stripe” Very challenging since there are an infinite number of points on XY-plane, making it the touchstone of the power of a classification algorithm

  30. This is exactly what we want!

  31. AN INTRODUCTION TO SUPPORT VECTOR MACHINES(and other kernel-based learning methods)N. Cristianini and J. Shawe-TaylorCambridge University Press2000 ISBN: 0 521 78019 5 References http://www.kernel-machines.org/ http://www.support-vector.net/ Papers by Vapnik C.J.C. Burges: A tutorial on Support Vector Machines. Data Mining and Knowledge Discovery 2:121-167, 1998.

More Related