1 / 80

Computer Vision Chapter 4

Computer Vision Chapter 4. Statistical Pattern Recognition Presenter: 小彼得 E-mail: itspeter@gmail.com. Overview of this chapter. Introduction Bayes Decision Rules Economic Gain Matrix Maximin Decision Rule Decision Rule Error Reserving Judgment Nearest Neighbor Rule

rasul
Download Presentation

Computer Vision Chapter 4

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Computer VisionChapter 4 Statistical Pattern Recognition Presenter: 小彼得 E-mail: itspeter@gmail.com Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.

  2. Overview of this chapter • Introduction • Bayes Decision Rules • Economic Gain Matrix • Maximin Decision Rule • Decision Rule Error • Reserving Judgment • Nearest Neighbor Rule • Binary Decision Tree Classifier • Neural Networks

  3. Introduction • Units: Image regions and projected segments • Each unit has an associated measurement vector • Using decision rule to assign unit to class or category optimally DC & CV Lab. CSIE NTU

  4. Introduction (Cont.) • Feature selection and extraction techniques • Decision rule construction techniques • Techniques for estimating decision rule error DC & CV Lab. CSIE NTU

  5. Simple Pattern Discrimination • Also called pattern identification process • A unit is observed or measured • A category assignment is made that names or classifies the unit as a type of object • The category assignment is made only on observed measurement (pattern) • This is called Fair Game Assumption DC & CV Lab. CSIE NTU

  6. Overview of this chapter • Introduction • Bayes Decision Rules • Economic Gain Matrix • Maximin Decision Rule • Decision Rule Error • Reserving Judgment • Nearest Neighbor Rule • Binary Decision Tree Classifier • Neural Networks

  7. Simple Pattern Discrimination (cont.) • a: assigned category from a set of categories C • t: true category identification from C • d: observed measurement from a set of measurements D • (t, a, d): event of classifying the observed unit • P(t, a, d): probability of the event (t, a, b) DC & CV Lab. CSIE NTU

  8. Economic Gain Matrix • e(t, a): economic gain/utility with true category t and assigned category a • A mechanism to evaluate a decision rule • Identity gain matrix DC & CV Lab. CSIE NTU

  9. Identity Gain Matrix DC & CV Lab. CSIE NTU

  10. An Instance DC & CV Lab. CSIE NTU

  11. Another Instance P(g, g): probability of true good, assigned good, P(g, b): probability of true good, assigned bad, ... e(g, g): economic consequence for event (g, g), … e positive: profit consequence e negative: loss consequence DC & CV Lab. CSIE NTU

  12. Another Instance (cont.) DC & CV Lab. CSIE NTU

  13. Another Instance (cont.) DC & CV Lab. CSIE NTU

  14. Another Instance (cont.) • Fraction of good objects manufactured P(g) = P(g, g) + P(g, b) • Fraction of good objects manufactured P(b) = P(b, g) + P(b, b) • Expected profit per object E =P(g, g)*e(g,g) + P(g, b)*e(g,b) + P(b, g)*e(b,g) + P(b, b)*e(b,b) DC & CV Lab. CSIE NTU

  15. Conditional Probability DC & CV Lab. CSIE NTU

  16. Conditional Probability (cont.) • P(b|g): false-alarm rate • P(g|b): misdetection rate • Another formula for expected profit per object DC & CV Lab. CSIE NTU

  17. Example 4.1 P(g) = 0.95, P(b) = 0.05 DC & CV Lab. CSIE NTU

  18. Example 4.1 (cont.) DC & CV Lab. CSIE NTU

  19. Example 4.2 P(g) = 0.95, P(b) = 0.05 DC & CV Lab. CSIE NTU

  20. Example 4.2 (cont.) DC & CV Lab. CSIE NTU

  21. Decision Rule Construction • (t, a): summing (t, a, d) on every measurements d • Therefore, • Average economic gain DC & CV Lab. CSIE NTU

  22. Decision Rule Construction (cont.) DC & CV Lab. CSIE NTU

  23. Decision Rule Construction (cont.) • We can use identity matrix as the economic gain matrix to compute the probability of correct assignment: DC & CV Lab. CSIE NTU

  24. Fair Game Assumption • Decision rule uses only measurement data in assignment; the nature and the decision rule are not in collusion • In other words, P(a| t, d) = P(a| d) DC & CV Lab. CSIE NTU

  25. Fair Game Assumption (cont.) • From the definition of conditional probability DC & CV Lab. CSIE NTU

  26. Fair Game Assumption (cont.) • Derive P(t,a|d) = P(a|d)P(t|d) • By def., P(t,a|d) = P(t,a,d) / P(d) ----(1) • By def., P(a|t,d) = P(t,a,d) / P(t,d) ----(2) • By Fair Game Assumption P(a|t,d) = P(a|d) ----(3) • (1)+(2)可得 P(t,a|d)*P(d) = P(a|t,d)*P(t,d) • (3)代入得 P(t,a|d)*P(d) = P(a|d)*P(t,d) Q.E.D DC & CV Lab. CSIE NTU

  27. DC & CV Lab. CSIE NTU

  28. Deterministic Decision Rule • We use the notation f(a|d) to completely define a decision rule; f(a|d) presents all the conditional probability associated with the decision rule • A deterministic decision rule: • Decision rules which are not deterministic are called probabilistic/nondeterministic/stochastic DC & CV Lab. CSIE NTU

  29. Expected Value on f(a|d) • Previous formula • By and => DC & CV Lab. CSIE NTU

  30. Expected Value on f(a|d) (cont.) DC & CV Lab. CSIE NTU

  31. Bayes Decision Rules • Maximize expected economic gain • Satisfy • Constructing f DC & CV Lab. CSIE NTU

  32. Bayes Decision Rules (cont.) DC & CV Lab. CSIE NTU

  33. Bayes Decision Rules (cont.) + + DC & CV Lab. CSIE NTU

  34. Continuous Measurement • For the same example, try the continuous density function of the measurements: and • Measurement lie in the close interval [0,1] • Prove that they are indeed density function DC & CV Lab. CSIE NTU

  35. Continuous Measurement (cont.) • Suppose that the prior probability of is 1/3 and the prior probability of is 2/3 • When , a Bayes decision rule will assign an observed unit to t1, which implies => DC & CV Lab. CSIE NTU

  36. Continuous Measurement (cont.) • .805 > .68, the continuous measurement has larger expected economic gain than discrete DC & CV Lab. CSIE NTU

  37. Break DC & CV Lab. CSIE NTU

  38. Prior Probability • The Bayes rule: • Replace with • The Bayes rule can be determined by assigning any categories that maximizes DC & CV Lab. CSIE NTU

  39. Equal-probability of ignorance • Equal-probability of ignorance assumption • P(t) is likely equal. • Put identity gain matrix together. • Maximum likelihood decision rule. DC & CV Lab. CSIE NTU

  40. Overview of this chapter • Introduction • Bayes Decision Rules • Economic Gain Matrix • Maximin Decision Rule • Decision Rule Error • Reserving Judgment • Nearest Neighbor Rule • Binary Decision Tree Classifier • Neural Networks

  41. Economic Gain Matrix • Do they have different decision rules ? DC & CV Lab. CSIE NTU

  42. Economic Gain Matrix • Identity matrix • Incorrect loses 1 • A more balanced instance DC & CV Lab. CSIE NTU

  43. Economic Gain Matrix • Suppose are two different economic gain matrix with relationship • According to the construction rule. Given a measurement d, • Because • We then got DC & CV Lab. CSIE NTU

  44. Economic Gain Matrix • Summary • Under positive multiplicative constant and additive constantwill not change the optimal rule. DC & CV Lab. CSIE NTU

  45. Economic Gain Matrix • In some cases, non regular matrix can derive optimal rule. • Revisit the continuous example • Let the economic matric be arbitrary, say it is • Optimal rule is to assign t1 when • Substituting the density for P(t1,x) , P(t2,x) • Finally, reach assigns category t1 DC & CV Lab. CSIE NTU

  46. Overview of this chapter • Introduction • Bayes Decision Rules • Economic Gain Matrix • Maximin Decision Rule • Decision Rule Error • Reserving Judgment • Nearest Neighbor Rule • Binary Decision Tree Classifier • Neural Networks

  47. Maximin Decision Rule • Recall, we have done that… • Finding function f(a|d) under… • given d, pick “a” that maximize • What if we don’t know P(t,d) and P(t) is not reasonable to be assume even. • Equal-probability-of-ignorance assumption is not true DC & CV Lab. CSIE NTU

  48. Maximin Decision Rule • Maximizes average gain over worst prior probability DC & CV Lab. CSIE NTU

  49. Maximin Decision Rule • Simplify using the assumption that • Thus, reaching worst prior of • When and others have zero probability • Formula became , instead of DC & CV Lab. CSIE NTU

  50. Example 4.3 DC & CV Lab. CSIE NTU

More Related