Lesson 4:

1. Lesson 4: Discrete Probability Distributions

2. Outline

3. Random Variables and probability distribution • A random variable is a numerical value determined by the outcome of an experiment. A random variable is often denoted by a capital letter, e.g., X or Y. • A probability distribution is the listing of all possible outcomes of an experiment and the corresponding probability.

4. Types of Probability Distributions • A discrete probability distribution can assume only certain outcomes (need not be finite) – for random variables that take discrete values. • The number of students in a class. • The number of children in a family. • A continuous probability distribution can assume an infinite number of values within a given range – for random variables that take continuous values. • The distance students travel to class. • The time it takes an executive to drive to work. • The amount of money spent on your last haircut.

5. Types of Probability Distributions Probability distribution may be classified according to the number of random variables it describes.

6. Features of a Univariate Discrete Distribution • Let x1,…,xN be the list of all possible outcomes (N of them). • The main features of a discrete probability distribution are: • The probability of a particular outcome, P(xi), is between 0 and 1.00. • The sum of the probabilities of the various outcomes is 1.00. That is, P(x1) + … + P(xN) = 1 • The outcomes are mutually exclusive. That is, P(x1and x2) = 0 and P(x1or x2) = P(x1)+ P(x2) • Generally, for all i not equal to k. P(xi and xk) = 0. P(xi or xk) = P(xi)+ P(xk)

7. Features of a Univariate Discrete Distribution Can the following be a probability distribution of a random variable?

8. If the coin is fair EXAMPLE: Univariate probability distribution • Consider a random experiment in which a coin is tossed three times. Let x be the number of heads. Let H represent the outcome of a head and T the outcome of a tail. • The possible outcomes for such an experiment will be: TTT, TTH, THT, THH, HTT, HTH, HHT, HHH. • Thus the possible values of x (number of heads) are • From the definition of a random variable, x as defined in this experiment, is a random variable. P(x=0) =1/8 P(x=1) =3/8 P(x=2) =3/8 P(x=3) =1/8 x=0: TTTx=1: TTH, THT, HTT x=2: THH, HTH, HHTx=3: HHH

9. Features of a Bivariate Discrete Distribution • If X and Y are discrete random variables, we may define their joint probability function asPXY(xi,yi) • Let (x1,…,xR) and (y1,…,yS) be the list of all possible outcomes for X and Y respectively. • The main features of a bivariate discrete probability distribution are: • The probability of a particular outcome, PXY(xi,yi) is between 0 and 1. • The sum of the probabilities of the various outcomes is 1.00. That is, PXY(x1,y1) + PXY(x2,y1) +…+ PXY(xR,y1) + + … + PXY(xR,yS) = 1 • The outcomes are mutually exclusive. That is, if xi not equal to xk, or yi not equal to yk PXY((xi,yi) and (xk,yk)) = 0 and PXY((xi,yi) or (xk,yk)) = PXY(xi,yi) + PXY(xk,yk)

10. Example: Bivariate Discrete Distribution X takes 3 possible values and Y takes 4 possible values.

11. EXAMPLE: Bivariate distribution The joint distribution of the movement of Hang Seng Index (HSI) and weather is shown in the following table.

12. EXAMPLE: Bivariate distribution The joint distribution of the movement of Hang Seng Index (HSI) and weather is shown in the following table. Suppose ….. PX|Y(x | y) = P(X = x | Y = y)=P(x,y)/P(y) if P(Y = y) > 0 PX|Y(x | y) =0 if P(Y = y) = 0 P(HSI falls|Rainy) = P(HSI falls, Rainy) / P(Rainy)= 0/0 Forcing P(HSI falls|Rainy) in the definition eliminates the difficulty in interpreting 0/0.

13. Marginal Distributions • The marginal probability function of X. PX(x) = yPXY(x, y) = PXY(x, y1) +PXY(x, y2) +…+ PXY(x, yn) P(HSI falls)= P(HSI falls and rainy) + P(HSI falls and not rainy) P(HSI rises)= P(HSI rises and rainy) + P(HSI rises and not rainy) • The double sum xyPXY(x, y) = P(HSI falls and rainy) + P(HSI falls and not rainy)+ P(HSI rises and rainy) + P(HSI rises and not rainy) = P(HSI falls)+P(HSI rises)= 1

14. Marginal Distributions • The marginal probability function of X. yPXY(x, y) = PX(x). • The marginal probability function of Y. xPXY(x, y) = PY(y). • The double sum yxPXY(x, y) = 1

15. Conditional Distributions • The conditional probability function of X given Y: PX|Y(x | y) = P(X = x | Y = y) = PXY(x,y)/PY(y) if P(Y = y) > 0 PX|Y(x | y) =0 if P(Y = y) = 0 Note that PX|Y(x | y) when P(Y = y) = 0 is undefined using the top formula.

16. Conditional Distributions • For each fixed y this is a probability function for X, i.e. the conditional probability function is non-negative and XPX|Y(x | y) = PX|Y(x1 | y)+ PX|Y(x2 | y) = PX,Y(x1, y)/ PY(y) + PX,Y(x2, y)/ PY(y) =[PX,Y(x1, y) + PX,Y(x2, y)]/ PY(y) =1.

17. Conditional Distributions • The conditional probability function of X given Y: PX|Y(x | y) = P(X = x | Y = y) if P(Y = y) > 0 PX|Y(x | y) =0 if P(Y = y) = 0 • For each fixed y this is a probability function for X, i.e. the conditional probability function is non-negative and XPX|Y(x | y) = 1. • By the definition of conditional probability, PX|Y(x | y) = PX,Y(x, y)/ PY(y). E.g., P(HSI rises| Rainy) = 0.2/0.35. • When X and Y are independent, PX|Y(x | y) is equal to PX(x).

18. Example: Conditional Distributions PX|Y(x | y) = PX,Y(x, y)/ PY(y). PY|X(y | x) = PX,Y(x, y)/ PX(x).

19. Transformation of Random variables • A transformation of random variable(s) results in a new random variable. • For example, if X and Y are random variables, the following are also random variables: • Z=2X • Z=3+2X • Z=X2 • Z=log(X) • Z=X+Y • Z=X2+Y2

20. The Expectation (mean) of a Discrete Probability Distribution • The expectation (mean): • reports the central location of the data. • is the long-run average value of the random variable. That is, the average of the outcomes of many experiments. • is also referred to as its expected value, E(X), in a probability distribution. • Is also known as first moment of a random variable. • is a weighted average.

21. Moments of a random variable The n-th moment is defined as the expectation of the n-th power of a random variable: E(Xn) The n-th centralized moment is defined as: E[X-E(X)]n

22. The Expectation (Mean) of Discrete Probability Distribution • For univariate probability distribution, the expectation or mean E(X) is computed by the formula: • For bivariate probability distribution, the the expectation or mean E(X) is computed by the formula:

23. Conditional Mean of Bivariate Discrete Probability Distribution • For bivariate probability distribution, the conditional expectation or conditional mean E(X|Y) is computed by the formula: • Unconditional expectation or mean of X, E(X)

24. Expectation of a linear transformed random variable • If a and b are constants and X is a random variable, then E(a) = a E(bX) = bE(X) E(a+bX) = a+bE(X)

25. The Variance of a Discrete Probability Distribution • The variance measures the amount of spread (variation) of a distribution. • The variance of a discrete distribution is denoted by the Greek letter 2 (sigma squared). • The standard deviation is the square root of 2.

26. The Variance of a Discrete Probability Distribution • For univariate discrete probability distribution • For bivariate discrete probability distribution

27. Variance of a linear transformed random variable • If a and b are constants and X is a random variable, then V(a) = 0 V(bX) = b2V(X) V(a+bX) = b2V(X)

28. The Covariance of a Bivariate Discrete Probability Distribution Covariance measures how two random variables co-vary.

29. Covariance of linear transformed random variables • If a and b are constants and X is a random variable, then C(a,b) = 0 C(a,bX) = 0 C(a+bX,Y) = bC(X,Y)

30. Variance of a sum of random variables • If a and b are constants and X and Y are random variables, then V(X+Y) = V(X) + V(Y) + 2C(X,Y) V(aX+bY) =a2V(X) + b2V(Y) + 2abC(X,Y)

31. Correlation coefficient • The strength of the dependence between X and Y is measured by the correlation coefficient:

32. EXAMPLE • Dan Desch, owner of College Painters, studied his records for the past 20 weeks and reports the following number of houses painted per week:

33. EXAMPLE • Compute the mean and variance of the number of houses painted per week and:

34. Binomial Probability Distribution • The binomial distribution has the following characteristics: • An outcome of an experiment is classified into one of two mutually exclusive categories, such as a success or failure. • The data collected are the results of counts in a series of trials. • The probability of success stays the same for each trial. • The trials are independent. • For example, tossing an unfair coin three times. • H is labeled success and T is labeled failure. • The data collected are number of H in the three tosses. • The probability of H stays the same for each toss. • The results of the tosses are independent.

35. Binomial Probability Distribution • To construct a binomial distribution, let • n be the number of trials • x be the number of observed successes • be the probability of success on each trial • The formula for the binomial probability distribution is: P(x) = nCxx(1- )n-x

36. The density functions of binomial distributions with n=20 and different success rates p

37. EXAMPLE x = number of patients who will experience nausea following treatment with Phe-Mycin , q = 1 – p = 1 - 0.1 = 0.9 , p = 0.1 n = 4 Find the probability that 2 of the 4 patients treated will experience nausea.

38. Binomial Probability Distribution • The formula for the binomial probability distribution is: P(x) = nCxx(1- )n-x TTT, TTH, THT, THH, HTT, HTH, HHT, HHH. • X=number of heads • The coin is fair, i.e., P(head) = 1/2. • P(x=0) = 3C00.50(1- 0.5)3-0 =3!/(0!3!) (1) (1/8)=1/8 • P(x=1) = 3C10.51(1- 0.5)3-1 =3!/(1!2!) (1) (1/8)= 3/8 • P(x=2) = 3C20.52(1- 0.5)3-2 =3!/(2!1!) (1) (1/8)= 3/8 • P(x=3) = 3C30.53(1- 0.5)3-3 =3!/(3!0!) (1) (1/8)= 1/8 When the coin is not fair, simple counting rule will not work.

39. Mean & Variance of the Binomial Distribution • The mean is found by: • The variance is found by:

40. EXAMPLE The Alabama Department of Labor reports that 20% of the workforce in Mobile is unemployed. From a sample of 14 workers, calculate the following probabilities: • Exactly three are unemployed. • At least three are unemployed. • At least one are unemployed.

41. EXAMPLE • The probability of exactly 3: • The probability of at least 3 is: • The probability of at least one being unemployed:

42. EXAMPLE • Since  =.2 and n=14. • Hence, the meanis: =n = 14(.2) = 2.8. • The varianceis: 2= n(1-  ) = (14)(.2)(.8) =2.24.

43. Finite Population • A finite population is a population consisting of a fixed number of known individuals, objects, or measurements. Examples include: • The number of students in this class. • The number of cars in the parking lot.

44. Hypergeometric Distribution • The hypergeometric distribution has the following characteristics: • There are only 2 possible outcomes. • The probability of a success is not the same on each trial. • It results from a count of the number of successes in a fixed number of trials.

45. EXAMPLE In a bag containing 7 red chips and 5 blue chips you select 2 chips one after the other without replacement. 6/11 R2 R1 7/12 B2 5/11 R2 7/11 B1 5/12 4/11 B2 The probability of a success (red chip) is not the same on each trial.

46. Hypergeometric Distribution • The formula for finding a probability using the hypergeometric distribution is: where Nis the size of the population, S is the number of successes in the population,xis the number of successes in a sample of n observations.

47. Hypergeometric Distribution • Use the hypergeometric distribution to find the probability of a specified number of successes or failures if: • the sample is selected from a finite population without replacement (recall that a criteria for the binomial distribution is that the probability of success remains the same from trial to trial) • the size of the sample n is greater than 5% of the size of the population N .

48. The density functions of hypergeometric distributions with N=100, n=20 and different success rates p (=S/N).

49. EXAMPLE: Hypergeometric Distribution • The National Air Safety Board has a list of 10 reported safety violations. Suppose only 4 of the reported violations are actual violations and the Safety Board will only be able to investigate five of the violations. What is the probability that three of five violations randomly selected to be investigated are actually violations?

50. Poisson Probability Distribution The formula for the binomial probability distribution is: P(x) = nCxx(1- )n-x • The binomial distribution becomes more skewed to the right (positive) as the probability of success become smaller. • The limiting form of the binomial distribution where the probability of success  is small and n is large is called the Poisson probability distribution.