1 / 70

PATTERN RECOGNITION Fatoş Tunay Yarman Vural

PATTERN RECOGNITION Fatoş Tunay Yarman Vural. Textbook : Pattern Recognition and Machine Learning , C. Bishop Reccomended Book : Pattern Theory , U. Granander , M. Miller. Course Requirement. Final:50% Project: 50% Literature Survey report : 1 April Algorithm Development :1 May

ardith
Download Presentation

PATTERN RECOGNITION Fatoş Tunay Yarman Vural

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PATTERN RECOGNITIONFatoş Tunay Yarman Vural Textbook: PatternRecognitionandMachineLearning, C. Bishop ReccomendedBook: PatternTheory, U. Granander, M. Miller

  2. CourseRequirement Final:50% Project: 50% LiteratureSurveyreport: 1 April AlgorithmDevelopment:1 May FullPaperwithimplementation 1 June

  3. Content 1.What is Pattern 2. ProbabilityTheory 3. BayesianParadigm 4. InformationTheory 5. LinearMethods 6.KernelMethods 7. GraphMethods

  4. WHAT İS PATTERN Structuresregulatedbyrules Goal:Representempiricalknowledge in mathematicalforms the Mathematics of Perception Need: Algebra, probabilitytheory, graphtheory

  5. ACTUAL SOUND The ?eel is on the shoe The ?eel is on the car The ?eel is on the table The ?eel is on the orange PERCEIVED WORDS The heel is on the shoe The wheel is on the car The meal is on the table The peel is on the orange What you perceive is not what you hear: (Warren & Warren, 1970) Statistical inference is being used!

  6. Allflows! Heraclitos It is onlytheinvariance, thepermanentfacts, thatenable us tofindthemeaning in a world of flux. We can onlyperceivevariances Ouraim is tofindtheinvariantlaws of ourvaryingobserbvations PatternRecognition

  7. SOURCE: Hypothesis Classes Obljects CHANNEL: Noisy OBSERVATION: Multiple sensor Variations assumption

  8. Example Handwritten Digit Recognition

  9. Polynomial Curve Fitting

  10. Sum-of-Squares Error Function

  11. 0th Order Polynomial

  12. 1st Order Polynomial

  13. 3rd Order Polynomial

  14. 9th Order Polynomial

  15. Over-fitting Root-Mean-Square (RMS) Error:

  16. Polynomial Coefficients

  17. Data Set Size: 9th Order Polynomial

  18. Data Set Size: 9th Order Polynomial

  19. Regularization Penalize large coefficient values

  20. Regularization:

  21. Regularization:

  22. Regularization: vs.

  23. Polynomial Coefficients

  24. Probability Theory Apples and Oranges

  25. Probability Theory Marginal Probability Conditional Probability Joint Probability

  26. Probability Theory Sum Rule Product Rule

  27. The Rules of Probability Sum Rule Product Rule

  28. Bayes’ Theorem posterior  likelihood × prior

  29. Probability Densities

  30. Transformed Densities

  31. Expectations Conditional Expectation (discrete) Approximate Expectation (discrete and continuous)

  32. Variances and Covariances

  33. The Gaussian Distribution

  34. Gaussian Mean and Variance

  35. The Multivariate Gaussian

  36. Gaussian Parameter Estimation Likelihood function

  37. Maximum (Log) Likelihood

  38. Properties of and

  39. Curve Fitting Re-visited

  40. Maximum Likelihood Determine by minimizing sum-of-squares error, .

  41. Predictive Distribution

  42. MAP: A Step towards Bayes Determine by minimizing regularized sum-of-squares error, .

  43. Bayesian Curve Fitting

  44. Bayesian Predictive Distribution

  45. Model Selection Cross-Validation

  46. Curse of Dimensionality

  47. Curse of Dimensionality Polynomial curve fitting, M = 3 Gaussian Densities in higher dimensions

  48. Decision Theory Inference step Determine either or . Decision step For given x, determine optimal t.

  49. Minimum Misclassification Rate

  50. Minimum Expected Loss Example: classify medical images as ‘cancer’ or ‘normal’ Decision Truth

More Related