Chapter 3: Probability

1. Chapter 3: Probability One day there was a fire in the wastebasket in the Dean’s office. In rushed a physicist, a chemist, and a statistician. The physicist immediately started calculations to determine how much energy would have to be removed from the fire to stop combustion. The chemist tried to figure out what chemical reagent would have to be added to the fire to prevent oxidation. While they were doing this, the statistician set fires in all the other wastebaskets in the office. “What are you doing?” they demanded. “Well, to solve the problem, you obviously need a larger sample size,” the statistician replied.

2. The Monty Hall Problem • In the Game Show, “Let’s Make a Deal,” A contestant was presented with three doors. Behind one door was a prize (often a new car) and behind the other two were goats. The contestant would choose a door. Then Monty would open one of the other doors, exposing a goat. The contestant could then switch or stay. • What should the contestant do?

3. Dice Simulation • Do Excel Demo • Points to be made: • Randomness means unpredictable results • Probability means long run is predictable • We imagine a mechanism, rule, or law that produces results with definite probabilities • If we know the “probability law,” we can make meaningful predictions about the likelihood of various outcomes

4. Why we play silly games • In the study of probability, we need simple examples to learn from. Some of these may seem silly or unrealistic, but they are actually models of “real” problems. If we understand the examples, we can tackle real problems by formulating them in terms of simple examples like dice games, coin tosses, spinners, etc. Recognizing a familiar set-up is often the key to success.

5. Counting 3’s • Another dice game: Roll two dice and record the number of 3’s. • The possible outcomes are 0, 1, or 2. • We will count the frequency of each outcome as we repeat the process. • (Excel Simulation)

6. Properties of this Experiment If we continue this experiment indefinitely: • The frequencies will have approximately a 25:10:1 ratio (we need to find out why) • The relative frequencies will settle down. A computer simulation of experimental outcomes is a helpful tool that may lead to important insights regarding a probability problem. But, it is not a substitute for the theoretical development that we will begin now.

7. Definitions • Probability Experiment: A repeatable process that yields a result or observations. • Trial: One repetition (yielding one observation) • Outcome: One possible result of an experiment. • The language of set theory is used… • The set of all possible outcomes is the Sample Space, often denoted by S. • Events are subsets of the Sample Space; they contain one or more outcomes, and are often denoted by A, B, or E.

8. Some Examples • An Experiment: Select two students at random and ask them if they have cars on campus. Record Y if the answer is “yes” and N if the answer is “no.” • There are two trials, because each student yields one observation. Each trial has an outcome of Y or N • But the outcomes of the experiment are ordered pairs: • The Sample Space: S={(N,N), (N,Y), (Y,N), (Y,Y)} • One example event: both students have cars.A={(Y,Y)} • Another event: only one student has a car.B={(Y,N),(N,Y)} • Yet another event: at least one has a car.C={(Y,N),(N,Y),(Y,Y)}

9. More Examples • Toss one coin, then toss one die. • S={H1 H2 H3 H4 H5 H6 T1 T2 T3 T4 T5 T6} • (The notation has been simplified.) • A={The coin was a head} • B={The die toss was an even number} • Randomly select three voters and ask if they favor an increase in property taxes for road construction in the county. • S = {NNN, NNY, NYN, NYY, YNN, YNY, YYN, YYY} • C={At least one voter said yes} • Exercise: List the elements of A, B, and C.

10. More Terms • Outcomes are also called sample points. • n(S): the number of outcomes in the sample space. • Events containing only one outcome are called Simple Events. • Events containing two or more outcomes are called Compound Events.

11. Notes • The outcomes in a sample space can never overlap (they are mutually exclusive). • The sample space must contain all possible outcomes (relate to exhaustive, below). • Two events may or may not be mutually exclusive. • If two or more events together include all outcomes, they are called exhaustive. • In some cases a collection of events may be bothmutually exclusive and exhaustive.

12. When Events Occur • Remember, events can contain multiple outcomes. • An event occurs if it contains the actual outcome of the experiment. • More than one event can occur for a single trial (if not mutually exclusive). • Example: On the way to work, some employees at a certain company stop for a bagel and/or a cup of coffee. Possible outcomes for (bagel,coffee) are: • (n,n): Don’t stop • (b,n): Get only a bagel • (n,c): Get only coffee • (b,c): Get bagel and coffee

13. Example: Not Mutually Exclusive/Exhaustive • Define event B as “gets bagel” • Define event C as “gets coffee” • Then B={(b,n),(b,c)} and C={(n,c),(b,c)} • B∩C={(b,c)} so B and C are not mutually exclusive. • If the outcome is (b,c), then both B and C have occurred. • It is also true that B and C are not exhaustive, since BUC≠S.

14. The accompanying Venn diagram illustrates the choices of the employees for a randomly selected work day. Coffee Coffee Bagel

15. Exercises • For the three dice games that we simulated: • Give the Sample Space • Construct several events, demonstrating: • Simple events • Compound events • Mutually Exclusive events • Exhaustive events • Mutually Exclusive and Exhaustive events • Find n(S) and n(A) for several events.

16. Determining Probability • Probability of an Event: The expected relative frequency of the event • Three ways to determine the probability of an event: • Empirically • Theoretically • Subjectively

17. Empirical Probability • Based on counts of data. It is theobserved relative frequency. • Use prime notation: • n’(A): number of times the event A has occurred • n: number of trials or observations, or sample size. • The Law of Large Numbers says the larger the number of experimental trials n, the closer the empirical probability P’(A) is expected to be to the true probability P(A). • In symbols:As

18. Theoretical and Subjective • Theoretical Probability, P(A), is the expected relative frequency (long run) • P(A) is based on knowledge (or assumptions) of the fundamental properties of the experiment. • Subjective Probability is based on someone’s opinion and/or experience. It is usually just a guess and subject to bias.

19. Theoretical Probability • Toss a fair coin. Let event H be the occurrence of a head. What is P(H)? • In a single toss of the coin, there are two possible outcomes. • Since the coin is fair, each outcome is equally likely. • Therefore it follows that P(H) = 1/2. • This doesn’t mean one head occurs in every two tosses. • After many trials, the proportion of heads is expected to be close to half, not based on data, but by reasoning from the fundamental properties of the experiment.

20. Equally Likely Outcomes • The previous example of a coin toss is an example of an experiment in which all outcomes are equally likely. • Many common problems (coins, dice, cards, SRS) have this property. • If this property holds, the probability of an event A is the ratio of the number of outcomes in A to the number of outcomes in S.

21. Examples • A die toss has six equally likely outcomes. • S={1,2,3,4,5,6}, thus n(S)=6. • Define event E as E={2,4,6}. Then n(E)=3. • P(E)=3/6=1/2. • Toss two coins; there are 4 outcomes. • S={TT,TH,HT,HH}. • Define event E as E={at least one head}. • E={TH,HT,HH} • P(E)=3/4

22. Example: A fair coin is tossed 5 times, and a head (H) or a tail (T) is recorded each time. What is the probability of A = {exactly one head in 5 tosses}, and B = {exactly 5 heads}? The outcomes consist of a sequence of 5 H’s and T’s A typical outcome: HHTTH There are 32 possible outcomes, all equally likely. A = {HTTTT, THTTT, TTHTT, TTTHT, TTTTH} B = {HHHHH}

23. The Monty Hall Problem • In the Game Show, “Let’s Make a Deal,” A contestant was presented with three doors. Behind one door was a prize (often a new car) and behind the other two were goats. The contestant would choose a door. Then Monty would open one of the other doors, exposing a goat. The contestant could then switch or stay. • What should the contestant do?

24. Solve the Monty Hall Problem • List the elements of the sample space under each strategy • This is probably the part that makes the problem difficult • There are two parts to each outcome: choice of door and location of car. • Represent an outcome by an ordered pair (door chosen, door with car) • Determine the probability of winning under each strategy

25. Solution to Monty Hall • S={(1,1),(1,2),(1,3),(2,1),(2,2),(2,3),(3,1),(3,2),(3,3)} • Strategy=StayWin={(1,1),(2,2),(3,3)}P(Win)=3/9=1/3 • Strategy=SwitchWin={(1,2),(1,3),(2,1),(2,3),(3,1),(3,2)}P(Win)=6/9=2/3 • Note: The Win events under each strategy are complements of each other.

26. Revisit a Previous Example • An Experiment: Select two students at random and ask them if they have cars on campus. Record Y if the answer is “yes” and N if the answer is “no.” • We can use a tree diagram to enumerate the elements of the sample space.

27. Hmmm… • How many statisticians does it take to screw in a light bulb? • We don’t know yet…the entire sample was skewed to the left. • Hope that didn’t go right by you…

28. Tree Diagram of Sample Space Student 1 Student 2 Outcomes Y Y, Y Y N Y, N Y N, Y N N N, N -Tree diagrams start from a common point, or “root” -This tree has four branches (from root to ends) -There are 2 first- and 4 second-generation branches. -The path along each branch shows a possible outcome.

29. Example: An experiment consists of selecting electronic parts from an assembly line and testing each to see if it passes inspection (P) or fails (F). The experiment terminates as soon as one acceptable part is found or after three parts are tested. Construct the sample space. Outcome F FFF F F P FFP P FP P P S = { FFF, FFP, FP, P }

30. Laws or Axioms of Probability • The probability of any event A is between 0 and 1. • The sum of the probabilities of all outcomes is 1. • A probability of 0 means the event cannot occur. • A probability of 1 means the event is certain, it must occur every time.

31. Introducing Odds Example: On the way to work Bob’s personal judgment is that he is four times more likely to get caught in a traffic jam (J) than have an easy commute (E). What values should be assigned to P(J) and P(E)?

32. Definition of Odds • The complement of A is denoted by . • contains all outcomes in S not in A. • Two events are complementary if they are mutually exclusive and exhaustive. • Odds are a way of expressing probabilities for complementary events as a ratio of expected frequencies. • If the odds in favor of an event A are a to b then the odds against A are b to a. • Then the probability that A occurs is • The probability A does not occur is

33. Example: 1. The complement of the event “success” is “failure.” 2. The complement of the event “rain” is “no rain.” 3. The complement of the event “at least 3 patients recover” out of 5 patients is “2 or fewer recover.” Notes: 1. 2. 3. Every event A has a complementary event 4. Useful in calculations such as when the question asks for the probability of “at least one.” 5. The complement of S is Ø, the empty set. 6. Obviously, P(Ø)=1-P(S)=1-1=0.

34. Addition Rules • If A and B occur, the outcome is in both, i.e., A∩B has occurred. So P(A and B)=P(A∩B). • If A and B are mutually exclusive, A∩B=Ø so P(A and B)=P(Ø)=0. • If A or B occurs, the outcome is in at least one of them, i.e., AUB has occurred. So P(A or B)=P(AUB) =P(A)+P(B)–P(A∩B). • Note: If A and B are NOT mutually exclusive, just adding P(A)+P(B) would count the outcomes in the intersection twice, so we have to correct for this double-count. • But, if A and B ARE mutually exclusive, P(A∩B)=0 so P(A or B)=P(AUB) =P(A)+P(B).

35. Example This diagram shows the probability that a randomly selected consumer has tried a snack food (F) is .5, tried a new soft drink (D) is .6, and tried both the snack food and the soft drink is .2. .3 .4 .2 D F .1 S

36. Examples • Suppose A and B are mutually exclusive, and P(A)=.12 and P(B)=.34. Find P(AUB). • Suppose P(A)=.6, P(AUB)=.9, and P(B)=.5. Find P(A∩B). • Suppose A, B, and C are mutually exclusive and exhaustive. If P(A)=.2, P(B)=.4, find P(C).

37. Conditional Probability • Sometimes two events are related in such a way that the probability of one depends upon whether the other occurs. • Partial information about the outcome may alter our assessment of the probabilities. • The symbol P(A | B) represents the probability that A will occur given B is known (assured). This is called conditional probability. • Suppose I toss a die and show you that there is a 3 on the front face. What can you say about the probabilities for the top face? • What is P(1 on top|3 on front)? • What is P(4 on top|3 on front)?

38. Attention!! • It is crucial to realize we are not talking about two sequential events. This is for one outcome of one trial of an experiment, for which we have partial information, allowing us to remove some of the uncertainty. • When I show you the three on the front face, the toss has already occurred, but you don’t know the result. The “chance” involved is in your ability to guess the correct value, rather than in a particular value coming up.

39. Die Example • Normally, There are six possibilities with P=1/6 for each. • With the three showing on front, we eliminate two outcomes, restricting the sample space. • The four remaining numbers are equally likely, with P=1/4. 1 2 3 6 5 4 1 2 3 6 5 4 1 2 6 5

40. Calculating Conditional Probability Recall our definition of probability in terms of frequencies of equally likely outcomes: Given B has occurred, the numerator becomes the number of outcomes of A that are still in the sample space. Any outcomes in A that were not in B are eliminated now. The denominator is the number of outcomes in B, the new sample space. To relate this back to the original probabilities, divide the numerator and denominator by n(S). Though this formula was derived using the idea of equal probabilities for all outcomes, the final form works in general.

41. Independent Events • Two events, defined for one trial of an experiment, are independent iffP(A | B) = P(A) or P(B | A) = P(B). • This should be understood to mean that if A and B are independent, the occurrence of B does not affect the probability of A, and visa versa. • If A and B are independent, then so are:

42. Example of Independent Events Consider the experiment in which a single fair die is rolled: S = {1, 2, 3, 4, 5, 6 }. Define the following events: A = {1, 2} B = “an odd number occurs”

43. Example of non-Independent Events Consider the experiment in which a single fair die is rolled: S = {1, 2, 3, 4, 5, 6 }. Define the following events: A = {1} B = “an odd number occurs”

44. General Multiplication Rule • A little algebra gives this variation: • Which might be more usefully thought of as: Note: How to recognize phrasing that indicates intersections: • Both A and B: • A but not B: • Neither A nor B = Not A and Not B = Not (A or B): • Not (A and B)=Not A or Not B:

45. Special Multiplication Rule If A and B are independent events in S, then , so . Example: Suppose the event A is “Allen gets a cold this winter,” B is “Bob gets a cold this winter,” and C is “Chris gets a cold this winter.” P(A) = .15, P(B) = .25, P(C) = .3, and all three events are independent. Find the probability that: 1. All three get colds this winter. 2. Allen and Bob get a cold but Chris does not. 3. None of the three gets a cold this winter.

46. Solution:

47. Summary Notes • Independent and mutually exclusive are two very different concepts. • Mutually exclusive says the two events cannot occur together, that is, they have no intersection. • Independence says each event does not affect the other event’s probability. • P(A and B) = P(A) P(B) when A and B are independent. • Since P(A) and P(B) are not zero, P(A and B) is nonzero. • Thus, independent events have an intersection. • Events cannot be both mutually exclusive and independent. • If two events are independent, then they are not mutually exclusive. • If two events are mutually exclusive, then they are not independent.

48. Tree Diagrams Tree Diagrams can be used to calculate probabilities that involve the multiplication and addition rules. • A set of branches that initiate from a single point has a total probability 1. • Each outcome for the experiment is represented by a branch that begins at the common starting point and ends at the terminal points at the right.

49. Example: A certain company uses three overnight delivery services: A, B, and C. The probability of selecting service A is 1/2, of selecting B is 3/10, and of selecting C is 1/5. Suppose the event T is “on time delivery.” P(T|A) = 9/10, P(T|B) = 7/10, and P(T|C) = 4/5. A service is randomly selected to deliver a package overnight. Construct a tree diagram representing this experiment.

50. The resulting tree diagram ServiceDelivery T A T B T C