1 / 22

COMP5331

This presentation by Raymond Wong provides an in-depth look at Support Vector Machines (SVM), a powerful classification model used in machine learning. It explores linear and non-linear SVMs, emphasizing the importance of maximizing the margin between data points for accurate predictions. Key concepts include decision trees, Bayesian classifiers, and neural networks, showcasing the advantages of SVMs, such as their visualizability and accuracy when data is well-partitioned. The presentation covers the mathematical foundation and applications of SVMs in various domains.

germaine-vega
Download Presentation

COMP5331

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. COMP5331 Other Classification Models: Support Vector Machine (SVM) Prepared by Raymond Wong Presented by Raymond Wong raywong@cse

  2. What we learnt for Classification • Decision Tree • Bayesian Classifier • Nearest Neighbor Classifier

  3. Other Classification Models • Support Vector Machine (SVM) • Neural Network

  4. Support Vector Machine • Support Vector Machine (SVM) • Linear Support Vector Machine • Non-linear Support Vector Machine

  5. Support Vector Machine • Advantages: • Can be visualized • Accurate when the data is well partitioned

  6. Linear Support Vector Machine x2 w1x1 + w2x2 + b > 0 x1 w1x1 + w2x2 + b = 0 w1x1 + w2x2 + b < 0

  7. Linear Support Vector Machine

  8. Linear Support Vector Machine

  9. Linear Support Vector Machine

  10. Linear Support Vector Machine x2 Support Vector x1 Margin We want to maximize the margin Why?

  11. Linear Support Vector Machine x2 w1x1 + w2x2 + b - D = 0 x1 w1x1 + w2x2 + b = 0 w1x1 + w2x2 + b + D = 0

  12. Linear Support Vector Machine Let y be the label of a point x2 +1 +1 w1x1 + w2x2 + b - 1  0 +1 +1 -1 w1x1 + w2x2 + b - 1 = 0 -1 -1 -1 x1 w1x1 + w2x2 + b + 1  0 w1x1 + w2x2 + b = 0 w1x1 + w2x2 + b + 1 = 0

  13. Linear Support Vector Machine Let y be the label of a point y(w1x1 + w2x2 + b)  1 x2 +1 +1 w1x1 + w2x2 + b - 1  0 +1 +1 -1 w1x1 + w2x2 + b - 1 = 0 -1 -1 y(w1x1 + w2x2 + b)  1 -1 x1 w1x1 + w2x2 + b + 1  0 w1x1 + w2x2 + b = 0 w1x1 + w2x2 + b + 1 = 0

  14. Margin |(b+1) – (b-1)| = 2 = Linear Support Vector Machine Let y be the label of a point y(w1x1 + w2x2 + b)  1 x2 +1 +1 +1 +1 -1 w1x1 + w2x2 + b - 1 = 0 -1 -1 y(w1x1 + w2x2 + b)  1 -1 x1 Margin w1x1 + w2x2 + b + 1 = 0 We want to maximize the margin

  15. 2 = Linear Support Vector Machine • Maximize • Subject to • for each data point (x1, x2, y)where y is the label of the point (+1/-1) Margin y(w1x1 + w2x2 + b)  1

  16. Linear Support Vector Machine • Minimize • Subject to • for each data point (x1, x2, y)where y is the label of the point (+1/-1) 2 y(w1x1 + w2x2 + b)  1

  17. Linear Support Vector Machine • Minimize • Subject to • for each data point (x1, x2, y)where y is the label of the point (+1/-1) Quadratic objective Linear constraints y(w1x1 + w2x2 + b)  1 Quadratic programming

  18. Linear Support Vector Machine • We have just described 2-dimensional space • We can divide the space into two parts by a line • For n-dimensional space where n >=2, • We use a hyperplane to divide the space into two parts

  19. Support Vector Machine • Support Vector Machine (SVM) • Linear Support Vector Machine • Non-linear Support Vector Machine

  20. Non-linear Support Vector Machine x2 x1

  21. Non-linear Support Vector Machine • Two Steps • Step 1: Transform the data into a higher dimensional space using a “nonlinear” mapping • Step 2: Use the Linear Support Vector Machine in this high-dimensional space

  22. Non-linear Support Vector Machine x2 x1

More Related