Probability interpretations, basic rules inverse probability, and Bayesian rule

1. Set 4 Probability interpretations, basic rules inverse probability, and Bayesian rule

2. Variable and outcomes • Variable • Quantitative (numerical) • Discrete (disconnected points): Number of items sold • Continuous (intervals): Income, price, proportion, average gain • Qualitative (categorical) • Type of products, gender, income level • Values of the variable (outcomes) • Number of items N = {n: n=0,1,2,…} • Income X = {x: x > 0} • Proportion of males in a class P = {p: 0 < p < 1} • Outcomes of tossing two coins S = {HH, HT, TH, TT} • Average gain

3. Uncertainty about unknown outcomes • Uncertainty exits when an outcome is unknown • Event =A, a set of unknown outcomes • P(A)= 1, certainly the outcome will be an element of A • P(A)= 0, A is an impossible event • P(A)= 0.50, maximum uncertainty about A • P(A) < 0.10, A is unlikely event? • P(A) < 0.05, A is very unlikely event? • P(A) < 0.01, A is very very unlikely event?

4. Interpretations of probability • Subjective • Degree of belief in occurrence of an event based on all available information • Knowledge of the subject matter, prior probability • All outcomes equally likely when no knowledge • Relative Frequency • Relative frequency of heads in n flips of a coin • Long-run under identical condition

5. Set terminology & probability • Complement of an event, Ac, A* • Set of all outcomes not inA • Null event f • Empty set, f= Sc • Intersection of two events, A and B • Set of outcomes for the occurrence ofboth AB • Disjoint events, AB = f • Union of two events, A or B • Set of outcomes for the occurrence ofat least one

6. Three axioms of probability • Probability is always between 0 and 1, inclusive 0 < P(A) < 1 • The set of all outcomes P(S)=1 • Addition rule for disjoint events P(A or B) = P(A) + P(B) • Easily extends to more than two disjoint events P(A1 or A2 or A3 or ... ) = P(A1) + P(A2) + P(A3)+ ...

7. Venn diagram • Two disjoint events, AB=f S A B

8. Rules for complement and null • The complement rule P(Ac) = 1 - P(A) S A Ac • The null event rule, P(f)= 0

9. General addition rule • Probability that at least one of a pair of events P(A or B) = P(A) + P(B) - P(AB) • More than two events • Complicated • Special case • For disjoint events P(AB)=0 P(A or B) = P(A) + P(B) A orB A B AB

10. Conditional probability • Probability of an event A as if another event B is going to occur • If B is going to occur, then only chance for A is the outcome will be common to both AandB • Conditional probability of Agiven B is B A AB

11. General multiplication rule • Cross multiply the two sides of the equation • For any pair of events A, B, the probability of their joint occurrence is P(AB) = P(A|B) P(B) • Similarly, conditioning on A gives P(AB) = P(B|A) P(A)

12. Independent events • Two events are independent if knowing that one will occur does not change the probability of the other: P(A|B) = P(A) • Similarly: P(B|A) = P(B) • Multiplication rule for two independent events P(AB) = P(A)P(B) • Use this to check for independence • Easily extends to more than two events P(A1A2 … An) = P(A1)P(A2) … P(An) • Two disjoint events with P(A)>0 and P(B)>0 can notbe independent because P(AB)=0 but P(A)P(B)>0

13. Application: Contingency Table • Cross-tabulation of individuals according to two characteristics • Example: Smoking and On-the-Job accident study • Table of frequencies (Observed counts) Accident Smoking

14. Table of proportions Accident • Relative frequencies Smoking • Conditional probability

15. Independence of events • Are heavy smoking and accident occurrence independent? • Check by multiplication rule: Is P(AB1)=P(A)P(B1)? P(AB1)=.18, P(A)=.52, P(B1)=.24 • Check by conditional probability: Is P(A)=P(A|B1)? • Are smoking and accident occurrence independent? • Independence of two variables • To be answered later in the course

16. Tree Diagram P(B)=.24 A .75 P(A|B)=.75 B .24 P(AB)=.75 x.24 Ac .25 P(Bc)=.76 A .44 .76 P(A|Bc)=.44 Bc P(ABc )=.44 x.76 Ac .56 P(A) = (.75 x.24) + (.44 x.76) = .52

17. Total probability rule • For any pair of events, P(A) = P(AB) + P(ABc) Bc B ABc AB A • Use product rule for P(AB) and P(ABc) P(A) = P(A|B) P(B)+ P(A|Bc)P(Bc) P(A) = (.75 x.24) + (.44 x.76) = .52

18. Inverse probability • An on-the-job accident has occurred. • What is the probability that the person is a heavy smoker? • Heavy smoking, B = B1 • Data, A • Find P(B|A) • Available information • Prior probability, P(B)=.24 • Likelihood of A, givenB, P(A|B)=.75 • Posterior probability P(B|A)?

19. Inverse probability (Bayes rule) • Given P(B),P(A|B), and P(A|Bc) • Compute the inverse conditional, P(B|A) • Bayes rule: • Direct applications • Important consequences for statistical analysis • Bayesian interpretation of data analysis results

20. Inverse probability • An on-the-job accident has occurred. • What is the probability that the person is heavy smoker? • Heavy smoking, B = B1 • Data, A • Find P(B|A) • Available information • Prior probability, P(B)=.24 • Likelihood of AgivenB = .75 • Posteriorprobability P(B|A)?