Machine Learning

1 / 14

# Machine Learning - PowerPoint PPT Presentation

Machine Learning. Lecture 6 K-Nearest Neighbor Classifier. x 1. 2. 3. 1. 4. 5. 6. 7. 10. 9. 11. 12. 13. 14. 8. x 2. 15. 16. Elliptical blobs (objects). Objects, Feature Vectors, Points. X(15). X(1). X(7). X(16). X(3). X(8). X(25). X(12). X(13). X(6). X(9). X(10).

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Machine Learning' - aren

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Machine Learning

Lecture 6

K-Nearest Neighbor Classifier

G53MLE Machine Learning

Dr Guoping Qiu

x1

2

3

1

4

5

6

7

10

9

11

12

13

14

8

x2

15

16

Elliptical blobs (objects)

Objects, Feature Vectors, Points

X(15)

X(1)

X(7)

X(16)

X(3)

X(8)

X(25)

X(12)

X(13)

X(6)

X(9)

X(10)

X(4)

X(11)

X(14)

x1

x2

Nearest Neighbours

X(j)=(x1(j), x2(j), …,xn(j))

X(i)=(x1(i), x2(i), …,xn(i))

G53MLE Machine Learning Dr Guoping Qiu

Nearest Neighbour Algorithm
• Given training data (X(1),D(1)), (X(2),D(2)), …, (X(N),D(N))
• Define a distance metric between points in inputs space. Common measures are:

Euclidean Distance

G53MLE Machine Learning Dr Guoping Qiu

K-Nearest Neighbour Model

Given test point X

• Find the K nearest training inputs to X
• Denote these points as

(X(1),D(1)), (X(2), D(2)), …, (X(k), D(k))

x

G53MLE Machine Learning Dr Guoping Qiu

K-Nearest Neighbour Model

Classification

• The class identification of X

Y = most common class in set {D(1), D(2), …, D(k)}

x

x

G53MLE Machine Learning Dr Guoping Qiu

K-Nearest Neighbour Model
• Example : Classify whether a customer will respond to a survey question using a 3-Nearest Neighbor classifier

G53MLE Machine Learning Dr Guoping Qiu

K-Nearest Neighbour Model
• Example : 3-Nearest Neighbors

15.16

15

152.23

122

15.74

G53MLE Machine Learning Dr Guoping Qiu

K-Nearest Neighbour Model
• Example : 3-Nearest Neighbors

15.16

15

152.23

122

15.74

Three nearest ones to David are: No, Yes, Yes

G53MLE Machine Learning Dr Guoping Qiu

K-Nearest Neighbour Model
• Example : 3-Nearest Neighbors

15.16

15

152.23

122

15.74

Yes

Three nearest ones to David are: No, Yes, Yes

G53MLE Machine Learning Dr Guoping Qiu

K-Nearest Neighbour Model
• Picking K
• Use N fold cross validation – Pick K to minimize the cross validation error
• For each of N training example
• Find its K nearest neighbours
• Make a classification based on these K neighbours
• Calculate classification error
• Output average error over all examples
• Use the K that gives lowest average error over the N training examples

G53MLE Machine Learning Dr Guoping Qiu

K-Nearest Neighbour Model
• Example: For the example we saw earlier, pick the best K from the set {1, 2, 3} to build a K-NN classifier

G53MLE Machine Learning Dr Guoping Qiu