Tamara Berg Machine Learning - PowerPoint PPT Presentation

tamara berg machine learning n.
Skip this Video
Loading SlideShow in 5 Seconds..
Tamara Berg Machine Learning PowerPoint Presentation
Download Presentation
Tamara Berg Machine Learning

play fullscreen
1 / 110
Download Presentation
Tamara Berg Machine Learning
Download Presentation

Tamara Berg Machine Learning

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. 790-133 Recognizing People, Objects, & Actions Tamara Berg Machine Learning

  2. Announcements • Topic presentation groups posted. Anyone not have a group yet? • Last day of background material • For Monday- Object recognition papers will be posted online. Please read!

  3. What is machine learning? • Computer programs that can learn from data • Two key components • Representation: how should we represent the data? • Generalization: the system should generalize from its past experience (observed data items) to perform well on unseen data items.

  4. Types of ML algorithms • Unsupervised • Algorithms operate on unlabeled examples • Supervised • Algorithms operate on labeled examples • Semi/Partially-supervised • Algorithms combine both labeled and unlabeled examples

  5. Unsupervised Learning

  6. K-means clustering • Want to minimize sum of squared Euclidean distances between points xi and their nearest cluster centers mk • Algorithm: • Randomly initialize K cluster centers • Iterate until convergence: • Assign each data point to the nearest center • Recompute each cluster center as the mean of all points assigned to it source: Svetlana Lazebnik

  7. Different clustering strategies • Agglomerative clustering • Start with each point in a separate cluster • At each iteration, merge two of the “closest” clusters • Divisive clustering • Start with all points grouped into a single cluster • At each iteration, split the “largest” cluster • K-means clustering • Iterate: assign points to clusters, compute means • K-medoids • Same as k-means, only cluster center cannot be computed by averaging • The “medoid” of each cluster is the most centrally located point in that cluster (i.e., point with lowest average distance to the other points) source: Svetlana Lazebnik

  8. Supervised Learning

  9. Slide from Dan Klein

  10. Slide from Dan Klein

  11. Slide from Dan Klein

  12. Slide from Dan Klein

  13. Example: Image classification input desired output apple pear tomato cow dog horse Slide credit: Svetlana Lazebnik

  14. http://yann.lecun.com/exdb/mnist/index.html Slide from Dan Klein

  15. Example: Seismic data Earthquakes Surface wave magnitude Nuclear explosions Body wave magnitude Slide credit: Svetlana Lazebnik

  16. Slide from Dan Klein

  17. The basic classification framework y = f(x) • Learning: given a training set of labeled examples{(x1,y1), …, (xN,yN)}, estimate the parameters of the prediction function f • Inference: apply f to a never before seen test examplex and output the predicted value y = f(x) output classification function input Slide credit: Svetlana Lazebnik

  18. Some ML classification methods Neural networks Nearest neighbor 106 examples LeCun, Bottou, Bengio, Haffner 1998 Rowley, Baluja, Kanade 1998 … Shakhnarovich, Viola, Darrell 2003 Berg, Berg, Malik 2005 … Conditional Random Fields Support Vector Machines and Kernels Guyon, Vapnik Heisele, Serre, Poggio, 2001 … McCallum, Freitag, Pereira 2000 Kumar, Hebert 2003 … Slide credit: Antonio Torralba

  19. Example: Training and testing • Key challenge: generalization to unseen examples Training set (labels known) Test set (labels unknown) Slide credit: Svetlana Lazebnik

  20. Slide credit: Dan Klein

  21. Classification by Nearest Neighbor Word vector document classification – here the vector space is illustrated as having 2 dimensions. How many dimensions would the data actually live in? Slide from Min-Yen Kan

  22. Classification by Nearest Neighbor Slide from Min-Yen Kan

  23. Classification by Nearest Neighbor Classify the test document as the class of the document “nearest” to the query document (use vector similarity to find most similar doc) Slide from Min-Yen Kan

  24. Classification by kNN Classify the test document as the majority class of the k documents “nearest” to the query document. Slide from Min-Yen Kan

  25. Classification by kNN What are the features? What’s the training data? Testing data? Parameters? Slide from Min-Yen Kan

  26. Slide from Min-Yen Kan

  27. Slide from Min-Yen Kan

  28. Slide from Min-Yen Kan

  29. Slide from Min-Yen Kan

  30. Slide from Min-Yen Kan

  31. Classification by kNN What are the features? What’s the training data? Testing data? Parameters? Slide from Min-Yen Kan

  32. NN for vision Fast Pose Estimation with Parameter Sensitive Hashing Shakhnarovich, Viola, Darrell

  33. NN for vision J. Hays and A. Efros, Scene Completion using Millions of Photographs, SIGGRAPH 2007

  34. NN for vision J. Hays and A. Efros, IM2GPS: estimating geographic information from a single image, CVPR 2008

  35. Decision tree classifier Example problem: decide whether to wait for a table at a restaurant, based on the following attributes: • Alternate: is there an alternative restaurant nearby? • Bar: is there a comfortable bar area to wait in? • Fri/Sat:is today Friday or Saturday? • Hungry: are we hungry? • Patrons: number of people in the restaurant (None, Some, Full) • Price: price range ($, $$, $$$) • Raining: is it raining outside? • Reservation: have we made a reservation? • Type: kind of restaurant (French, Italian, Thai, Burger) • WaitEstimate: estimated waiting time (0-10, 10-30, 30-60, >60) Slide credit: Svetlana Lazebnik

  36. Decision tree classifier Slide credit: Svetlana Lazebnik

  37. Decision tree classifier Slide credit: Svetlana Lazebnik

  38. Linear classifier • Find a linear function to separate the classes f(x) = sgn(w1x1 + w2x2 + … + wDxD) = sgn(w  x) Slide credit: Svetlana Lazebnik