Download
maximum likelihood n.
Skip this Video
Loading SlideShow in 5 Seconds..
Maximum Likelihood PowerPoint Presentation
Download Presentation
Maximum Likelihood

Maximum Likelihood

234 Views Download Presentation
Download Presentation

Maximum Likelihood

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Maximum Likelihood Benjamin Neale Boulder Workshop 2012

  2. We will cover • Easy introduction to probability • Rules of probability • How to calculate likelihood for discrete outcomes • Confidence intervals in likelihood • Likelihood for continuous data

  3. Starting simple • Let’s think about probability

  4. Starting simple • Let’s think about probability • Coin tosses • Winning the lottery • Roll of the die • Roulette wheel

  5. Starting simple • Let’s think about probability • Coin tosses • Winning the lottery • Roll of the die • Roulette wheel • Chance of an event occurring

  6. Starting simple • Let’s think about probability • Coin tosses • Winning the lottery • Roll of the die • Roulette wheel • Chance of an event occurring • Written as P(event) = probability of the event

  7. Simple probability calculations • To get comfortable with probability, let’s solve these problems: • Probability of rolling an even number on a six-sided die • Probability of pulling a club from a deck of cards

  8. Simple probability calculations • To get comfortable with probability, let’s solve these problems: • Probability of rolling an even number on a six-sided die ½ or 0.5 • Probability of pulling a club from a deck of cards ¼ or 0.25

  9. Simple probability rules • P(A and B) = P(A)*P(B)

  10. Simple probability rules • P(A and B) = P(A)*P(B) • E.g. what is the probability of tossing 2 heads in a row?

  11. Simple probability rules • P(A and B) = P(A)*P(B) • E.g. what is the probability of tossing 2 heads in a row? • A = Heads and B = Heads so,

  12. Simple probability rules • P(A and B) = P(A)*P(B) • E.g. what is the probability of tossing 2 heads in a row? • A = Heads and B = Heads so, • P(A) = ½, P(B) = ½,P(A and B) = ¼

  13. Simple probability rules • P(A and B) = P(A)*P(B) • E.g. what is the probability of tossing 2 heads in a row? • A = Heads and B = Heads so, • P(A) = ½, P(B) = ½, P(A and B) = ¼ *We assume independence

  14. Simple probability rules cnt’d • P(A or B) = P(A) + P(B) – P(A and B)

  15. Simple probability rules cnt’d • P(A or B) = P(A) + P(B) – P(A and B) • What is the probability of rolling a 1 or a 4?

  16. Simple probability rules cnt’d • P(A or B) = P(A) + P(B) – P(A and B) • What is the probability of rolling a 1 or a 4? • A = rolling a 1 and B = rolling a 4

  17. Simple probability rules cnt’d • P(A or B) = P(A) + P(B) – P(A and B) • What is the probability of rolling a 1 or a 4? • A = rolling a 1 and B = rolling a 4 • P(A) = , P(B) = , P(A or B) = 1 1 1 6 6 3

  18. Simple probability rules cnt’d • P(A or B) = P(A) + P(B) – P(A and B) • What is the probability of rolling a 1 or a 4? • A = rolling a 1 and B = rolling a 4 • P(A) = , P(B) = , P(A or B) = 1 1 1 6 6 3 *We assume independence

  19. Recap of rules • P(A and B) = P(A)*P(B) • P(A or B) = P(A) + P(B) – P(A and B) • Sometimes things are ‘exclusive’ such as rolling a 6 and rolling a 4. It cannot occur in the same trial implies P(A and B) = 0 Assuming independence

  20. Conditional probabilities • P(X | Y) = the probability of X occurring given Y.

  21. Conditional probabilities • P(X | Y) = the probability of X occurring given Y. • Y can be another event (perhaps that predicts X)

  22. Conditional probabilities • P(X | Y) = the probability of X occurring given Y. • Y can be another event (perhaps that predicts X) • Y can be a probability or set of probabilities

  23. Conditional probabilities • Roll two dice in succession

  24. 1 12 Conditional probabilities • Roll two dice in succession • P(total = 10) =

  25. 1 12 Conditional probabilities • Roll two dice in succession • P(total = 10) = • What is P(total = 10 | 1st die = 5)?

  26. 1 1 12 6 Conditional probabilities • Roll two dice in succession • P(total = 10) = • What is P(total = 10 | 1st die = 5)? • P(total = 10 | 1st die = 5) =

  27. Binomial probabilities • Used for two conditions such as coin toss • Determine the chance of any outcome:

  28. Binomial probabilities • Used for two conditions such as coin toss • Determine the chance of any outcome:

  29. Binomial probabilities • Used for two conditions such as coin toss • Determine the chance of any outcome: Probability # of k results # of trials

  30. Binomial probabilities • Used for two conditions such as coin toss • Determine the chance of any outcome: Number of combinations of n choose k ! = factorial; n! = n*(n-1)*(n-2)*…*2*1 and factorials are bad for big numbers

  31. Binomial probabilities • Used for two conditions such as coin toss • Determine the chance of any outcome: Probability of not k occurring Probability of k occurring

  32. Binomial probabilities • Used for two conditions such as coin toss • Determine the chance of any outcome: Probability of not k occurring Number of combinations of n choose k Probability # of positive results Probability of k occurring # of trials ! = factorial; n! = n*(n-1)*(n-2)*…*2*1 and factorials are bad for big numbers

  33. Combinations piece long way • Does it work? Let’s try: How many combinations for 3 heads out of 5 tosses?

  34. Combinations piece long way • Does it work? Let’s try: How many combinations for 3 heads out of 5 tosses? • HHHTT, HHTHT, HHTTH, HTHHT, HTHTH, HTTHH, THHHT, THHTH, THTHH, TTHHH = 10 possible combinations

  35. Combinations piece formula • Does it work? Let’s try: How many combinations for 3 heads out of 5 tosses? • We have 5 choose 3 = 5!/(3!)*(2!) • =(5*4)/2 • =10

  36. Probability roundup • We assumed the ‘true’ parameter values • E.g. P(Heads) = P(Tails) = ½

  37. Probability roundup • We assumed the ‘true’ parameter values • E.g. P(Heads) = P(Tails) = ½ • What happens if we have data and want to determine the parameter values?

  38. Probability roundup • We assumed the ‘true’ parameter values • E.g. P(Heads) = P(Tails) = ½ • What happens if we have data and want to determine the parameter values? • Likelihood works the other way round: what is the probability of the observed data given parameter values?

  39. Concrete example • Likelihood aims to calculate the range of probabilities for observed data, assuming different parameter values.

  40. Concrete example • Likelihood aims to calculate the range of probabilities for observed data, assuming different parameter values. • The set of probabilities is referred to as a likelihood surface

  41. Concrete example • Likelihood aims to calculate the range of probabilities for observed data, assuming different parameter values. • The set of probabilities is referred to as a likelihood surface • We’re going to generate the likelihood surface for a coin tossing experiment

  42. Concrete example • Likelihood aims to calculate the range of probabilities for observed data, assuming different parameter values. • The set of probabilities is referred to as a likelihood surface • We’re going to generate the likelihood surface for a coin tossing experiment • The set of parameter values with the best probability is the maximum likelihood

  43. Coin tossing • I tossed a coin 10 times and get 4 heads and 6 tails

  44. Coin tossing • I tossed a coin 10 times and get 4 heads and 6 tails • From this data, what does likelihood estimate the chance of heads and tails for this coin?

  45. Coin tossing • I tossed a coin 10 times and get 4 heads and 6 tails • From this data, what does likelihood estimate the chance of heads and tails for this coin? • We’re going to calculate: • P(4 heads out of 10 tosses| P(H) = *) • where star takes on a range of values

  46. Calculations • P(4 heads out of 10 tosses | P(H)=0.1) = We can make this easier, as it will be constant C* across all calculations

  47. Calculations • P(4 heads out of 10 tosses | P(H)=0.1) = Now all we do is change the values of p and q

  48. Calculations • P(4 heads out of 10 tosses | P(H)=0.1) =

  49. Table of likelihoods

  50. Table of likelihoods Largest probability observed = maximum likelihood