Introduction to Linear Classifiers for Binary Classification

1. Linear Classifiers Based on slides by William Cohen, Andrej Karpathy, Piyush Rai

8. Linear Classifiers • Let’s simplify life by assuming: • Every instance is a vector of real numbers, x=(x1,…,xn). (Notation: boldface x is a vector.) • First we consider only two classes, y=(+1) and y=(-1) • A linear classifier is vector w of the same dimension as x that is used to make this prediction:

9. x2 Visually, x · w is the distance you get if you “project x onto w” w X1 . w X1 In 3d: lineplane In 4d: planehyperplane … X2 . w The line perpendicular to w divides the vectors classified as positive from the vectors classified as negative. -W

10. w -W Wolfram MathWorld Mediaboost.com

11. w -W Notice that the separating hyperplane goes through the origin…if we don’t want this we can preprocess our examples: or where b=w0 is called bias

13. Question

14. Back to Image Classification

15. 3072 numbers in total reshaped into a column vector x

19. Interactive Web Demo: http://vision.stanford.edu/teaching/cs231n/linear-classify-demo/

21. ^ ^ Compute: yi = sign(wk . xi ) If mistake: wk+1 = wk + yixi yi yi Perceptron learning instancexi B A • 1957: The perceptron algorithm by Frank Rosenblatt • 1960: Perceptron Mark 1 Computer – hardware implementation • 1969: Minksky & Papert book shows perceptrons limited to linearly separable data • 1970’s: learning methods for two-layer neural networks