Convergence concepts in probability theory

1. Convergence concepts in probability theory • Definitions and relations between convergence concepts • Sufficient conditions for almost sure convergence • Convergence via transforms • The law of large numbers and the central limit theorem Probability theory 2011

2. Coin-tossing: relative frequency of heads Convergence of each trajectory? Convergence in probability? Probability theory 2011

3. Convergence to a constant The sequence {Xn} of random variables converges almost surely to the constantcif and only if P({ ; Xn()  cas n  }) = 1 The sequence {Xn} of random variables converges in probability to the constantcif and only if, for all  > 0, P({ ; | Xn() – c| > })  0 as n  Probability theory 2011

4. An (artificial) example Let X1, X2,… be a sequence of independent binary random variables such that P(Xn = 1) =1/n and P(Xn= 0) = 1 – 1/n Does Xn converge to 0 in probability? Does Xn converge to 0 almost surely? Common exception set? Probability theory 2011

5. The law of large numbers for random variableswith finite variance Let {Xn} be a sequence of independent and identically distributed random variables with mean  and variance 2, and set Sn = X1 + … + Xn Then Proof: Assume that = 0. Then . Probability theory 2011

6. Convergence to a random variable: definitions The sequence {Xn} of random variables converges almost surely to the random variable Xif and only if P({ ; Xn()  X()asn  }) = 1 Notation: The sequence {Xn} of random variables converges in probability to the random variable Xif and only if, for all  > 0, P({ ; | Xn() – X()| > })  0 asn  Notation: Probability theory 2011

7. Convergence to a random variable: an example Assume that the concentration of NO in air is continuously recorded and let Xt, be the concentration at timet. Consider the random variables: Does Yn converge to Y in probability? Does Yn converge to Y almost surely? Probability theory 2011

8. Convergence in distribution: an example Let XnBin(n, c/n). Then the distribution of Xn converges to a Po(c) distribution as n  . p = 0.1) Probability theory 2011

9. Convergence in distribution and in norm The sequence Xnconverges in distribution to the random variable X as n  iff for all x where FX(x) is continuous. Notation: The sequence Xnconverges in quadratic mean to the random variable X as n  iff Notation: Probability theory 2011

10. Relations between the convergence concepts Almost sure convergence Convergence in probability Convergence in distribution Convergence in r-mean Probability theory 2011

11. Convergence in probability impliesconvergence in distribution Note that, for all  > 0, Probability theory 2011

12. Convergence almost surely -convergence in r-mean Consider a branching process in which the offspring distribution has mean 1. Does it converge to zero almost surely? Does it converge to zero in quadratic mean? Let X1, X2,… be a sequence of independent random variables such that P(Xn = n2) =1/n2and P(Xn= 0) = 1 – 1/n2 Does Xn converge to 0 in probability? Does Xn converge to 0 almost surely? Does Xn converge to 0 in quadratic mean? Probability theory 2011

13. Relations between different types ofconvergence to a constant Almost sure convergence Convergence in probability Convergence in distribution Convergence in r-mean Probability theory 2011

14. Convergence via generating functions Let X, X1,X2, … be a sequence of nonnegative, integer-valued random variables, and suppose that Then Is the limit function of a sequence of generating functions a generating function? Probability theory 2011

15. Convergence via moment generating functions Let X, X1,X2, … be a sequence of random variables, and suppose that Then Is the limit function of a sequence of moment generating functions a moment generating function? Probability theory 20101

16. Convergence via characteristic functions Let X, X1,X2, … be a sequence of random variables, and suppose that Then Is the limit function of a sequence of characteristic functions a characteristic function? Probability theory 2011

17. Convergence to a constantvia characteristic functions Let X1,X2, … be a sequence of random variables, and suppose that Then Probability theory 2011

18. The law of large numbers(for variables with finite expectation) Let {Xn} be a sequence of independent and identically distributed random variables with expectation , and set Sn = X1 + … + Xn Then . Probability theory 2011

19. The strong law of large numbers(for variables with finite expectation) Let {Xn} be a sequence of independent and identically distributed random variables with expectation , and set Sn = X1 + … + Xn Then . Probability theory 2011

20. The central limit theorem Let {Xn} be a sequence of independent and identically distributed random variables with mean  and variance 2, and set Sn = X1 + … + Xn Then Proof: If  = 0, we get . Probability theory 2011

21. Rate of convergence in the central limit theorem Example: XU(0,1) . Probability theory 2011

22. Sums of exponentially distributed random variables Probability theory 2011

23. Convergence of empirical distribution functions Proof: Write Fn(x) as a sum of indicator functions Bootstrap techniques: The original distribution is replaced with the empirical distribution Probability theory 2011

24. Resampling techniques- the bootstrap method Resampled data Observed data 60 22 88 41 Sampling with replacement 34 58 67 88 79 88 79 90 62 39 41 41 73 22 34 44 90 58 44 70 60 70 60 85 85 Probability theory 2011

25. Characteristics of infinite sequences of events Let {An, n = 1, 2, …} be a sequence of events, and define Example: Consider a queueing system and let An = {the queueing system is empty at time n} Probability theory 2011

26. The probability that an event occurs infinitely often - Borel-Cantelli’s first lemma Let {An, n = 1, 2, …} be an arbitrary sequence of events. Then Example: Consider a queueing system and let An = {the queueing system is empty at time n} Is the converse true? Probability theory 2011

27. The probability that an event occurs infinitely often- Borel-Cantelli’s second lemma Let {An, n = 1, 2, …} be a sequence of independent events. Then Probability theory 2011

28. Necessary and sufficient conditions for almost sure convergence of independent random variables Let X1, X2, … be a sequence of independent random variables. Then Probability theory 2011

29. Exercises: Chapter VI 6.1, 6.6, 6.9, 6.10, 6.17, 6.21, 6.25, 6.49 Probability theory 2011