Lecture 12 More Probability (Section 0.2)

1. Lecture 12More Probability(Section 0.2) Theory of Information

2. Independent Events DEFINITION Two events E and F from the same sample space are independent if P(EF)=P(E)P(F). Intuition: If E and F are independent, then knowing that E happened does not change our expectation regarding whether F also happened. Example: Flipping fair coin twice: ({HH, HT, TH, TT}, ¼, ¼, ¼, ¼) Are the following events independent? E={HH, HT} and F={HT, TT} E={HH,TT} and F={HT,TH} E={HH} and F={HH,HT}

3. Multiple Independent Events DEFINITION Three events E, F and G from the same sample space are independent if P(EFG)=P(E)P(F)P(G), and also E and F, E and G, F and G are (pairwise) independent. DEFINITION n events E1,…,En from the same sample space are independent if the probability of the intersection of every subcollection of these events is equal to the product of the probabilities of these events. Intuition: If events are independent, then knowing that any subcollection of these events happened does not change our expectation regarding whether any other remaining events also happened. Example: {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT} with uniform probability. Are the following 3 events independent? {HHH,HHT,HTH,HTT}, {HHH,HHT,THH,THT}, {HHH,HTH,THH,TTH}

4. Example 0.2.8(1) Suppose that binary strings of length 5 are sent over a noisy communication line, where the probability of receiving a bit correctly is 0.75. Assume that the event that one bit is received correctly is independent of the event that another bit is received correctly. 1. What is the probability that a string will be received correctly? Let Ei be the event that the ith bit is received correctly. P(string received correctly) =

5. Example 0.2.8(2) Suppose that binary strings of length 5 are sent over a noisy communication line, where the probability of receiving a bit correctly is 0.75. Assume that the event that one bit is received correctly is independent of the event that another bit is received correctly. 2. What is the probability that exactly 3 bits will be received correctly? Let Ei be the event that the ith bit is received correctly. P(the first 3 bits received correctly) = How many choices are for 3 correct bits? P(exactly 3 bits received correctly) =

6. Example 0.2.9(1-2) Suppose that binary strings of length n are sent over a noisy communication line, where the probability of receiving a bit correctly is p. Assume that the event that one bit is received correctly is independent of the event that another bit is received correctly. Let Ei be the event that the ith bit is received correctly. 1. What is the probability that an entire string will be received correctly? P(string received correctly) = 2. What is the probability that a specified set of k bits (such as the first k bits) will be received correctly while the remaining bits received incorrectly?

7. Example 0.2.9(3-4) Suppose that binary strings of length n are sent over a noisy communication line, where the probability of receiving a bit correctly is p. Assume that the event that one bit is received correctly is independent of the event that another bit is received correctly. 3. What is the probability that exactly k bits (any k bits) will be received correctly? 4. What is the probability that at least k bits will be received correctly?

8. Conditional Probability DEFINITION Assume P(F)0. The conditional probability of E, given that F has occurred, is P(EF) P(E|F)= ----------- P(F) Note: P(EF) = P(E|F)  P(F)

9. Example 0.2.10 A certain surgery results in complete recovery 60% of the time, partial recovery 30% of the time, and death 10% of the time. What is the pro- bability of complete recovery, given that a patient survives the surgery? E --- complete recovery, F --- surviving the surgery. P(F) = What event is EF? EF = P(E|F)=

10. Partitions • Let S be a sample space. The events E1,…,En form a partition of S if the following • three conditions are satisfied: • The probability of each event Ei is >0; • The events are mutually exclusive; • The union of those events is S. S S S E1 35% E1 35% E1 35% E2 40% E2 40% E2 45% E3 25% E3 15% E3 25%

11. Theorem of Total Probabilities THEOREM 0.2.3 Let S be a sample space, A an event in S, and let E1,…,En form a partition of S. Then P(A) = k=1  P(A | Ek)P(Ek) n S A 10% 5% 5% E1 50% E2 30% E3 20% P(A|E1)= P(A|E2)= P(A|E1)= P(A)=

12. Suppose that bits are sent over a noisy communication line, where 0 is sent (event E1) 60% of the time, and 1 is sent (event E2) 40% of the time. Assume that the probability that 0 is received correctly is 80%, and the probability that 1 is received correctly is 90%. Let A be the event that an (arbitrary) bit is received correctly. What is P(A)? Example on Total Probabilities S= E1= E2= A= P(A|E1)= P(A|E2)= P(A)=

13. Homework • Consider the process of flipping a fair coin three times. Let E be the event that • all three flips result in the same outcome, and F the event that the first two flips • result in the same outcome. • What is the sample space S and its probability distribution? • List all elements of E, and all elements of F. • What is the probability of E? • What is the probability of F? • Are E and F independent? Why yes or no? • What is the probability of E, given that F occurred, i.e. P(E|F)=? • P(F|E)=? • Give two independent events from your sample space (if you answered “yes” to • question (e), then you should provide events different from E and F). • Let G1 be the event that the first flip results in heads, G2 the event that the second • flip results in heads, and G3 the event that the third flip results in heads. Do • G1,G2 and G3 form a partition of your sample space? Why yes or not?