Stochastic Process - Electronics & Telecommunication Engineering

1. Digital Communication Unit-III StochasticProcess Ashok N Shinde ashok.shinde0349@gmail.com International Institute of Information Technology HinjawadiPune July 26,2017 Ashok NShinde 1/28

2. Introduction to StochasticProcess Signals Deterministic: can be reproduced exactly with repeated measurements. Ashok NShinde 2/28

3. Introduction to StochasticProcess Signals Deterministic: can be reproduced exactly with repeated measurements. s(t)=Acos(2πfct+θ) Ashok NShinde 2/28

4. Introduction to StochasticProcess Signals Deterministic: can be reproduced exactly with repeated measurements. s(t)=Acos(2πfct+θ) where A,fc and θ areconstant. Ashok NShinde 2/28

5. Introduction to StochasticProcess Signals Deterministic: can be reproduced exactly with repeated measurements. s(t)=Acos(2πfct+θ) where A,fc and θ areconstant. Random: signal that is not repeatable in a predictablemanner. Ashok NShinde 2/28

6. Introduction to StochasticProcess Signals Deterministic: can be reproduced exactly with repeated measurements. s(t)=Acos(2πfct+θ) where A,fc and θ areconstant. Random: signal that is not repeatable in a predictablemanner. s(t)=Acos(2πfct+θ) Ashok NShinde 2/28

7. Introduction to StochasticProcess Signals Deterministic: can be reproduced exactly with repeated measurements. s(t)=Acos(2πfct+θ) where A,fc and θ areconstant. Random: signal that is not repeatable in a predictablemanner. s(t)=Acos(2πfct+θ) where A,fc and θ arevariable. Ashok NShinde 2/28

8. Introduction to StochasticProcess Signals Deterministic: can be reproduced exactly with repeated measurements. s(t)=Acos(2πfct+θ) where A,fc and θ areconstant. Random: signal that is not repeatable in a predictablemanner. s(t)=Acos(2πfct+θ) where A,fc and θ arevariable. Unwanted signals:Noise Ashok NShinde 2/28

9. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Ashok NShinde 3/28

10. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Mathematically relationship between probability theory and stochastic processes is asfollows- Ashok NShinde 3/28

11. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Mathematically relationship between probability theory and stochastic processes is asfollows- Sample point →SampleFunction Ashok NShinde 3/28

12. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Mathematically relationship between probability theory and stochastic processes is asfollows- Sample point →Sample Function Sample space→Ensemble Ashok NShinde 3/28

13. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Mathematically relationship between probability theory and stochastic processes is asfollows- Sample point →Sample Function Sample space→Ensemble Random Variable→ RandomProcess Ashok NShinde 3/28

14. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Mathematically relationship between probability theory and stochastic processes is asfollows- Sample point →Sample Function Sample space→Ensemble Random Variable→ Random Process Sample point s is function of time: Ashok NShinde 3/28

15. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Mathematically relationship between probability theory and stochastic processes is asfollows- Sample point →Sample Function Sample space→Ensemble Random Variable→ RandomProcess Sample point s is function of time: X(s, t),−T ≤ t ≤ T Ashok NShinde 3/28

16. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Mathematically relationship between probability theory and stochastic processes is asfollows- Sample point →Sample Function Sample space→Ensemble Random Variable→ RandomProcess Sample point s is function of time: X(s, t),−T ≤ t ≤ T Sample function denotedas: Ashok NShinde 3/28

17. StochasticProcess Definition: Astochasticprocessisasetofrandomvariablesindexedintime. Mathematically: Mathematically relationship between probability theory and stochastic processes is asfollows- Sample point →Sample Function Sample space→Ensemble Random Variable→ RandomProcess Sample point s is function of time: X(s, t),−T ≤ t ≤ T Sample function denotedas: xj(t)=X(t,sj),−T≤t≤T Ashok NShinde 3/28

18. StochasticProcess A random process is defined as the ensemble(collection) of time functions together with a probabilityrule Ashok NShinde 4/28

19. StochasticProcess A random process is defined as the ensemble(collection) of time functions together with a probabilityrule |xj(t)|,j=1,2,...,n Ashok NShinde 4/28

20. StochasticProcess StochasticProcess Each sample point in S is associated with a sample functionx(t) Ashok NShinde 5/28

21. StochasticProcess StochasticProcess Each sample point in S is associated with a sample function x(t) X(t, s) is a randomprocess Ashok NShinde 5/28

22. StochasticProcess StochasticProcess Each sample point in S is associated with a sample functionx(t) X(t, s) is a randomprocess isanensembleofalltimefunctionstogetherwithaprobabilityrule Ashok NShinde 5/28

23. StochasticProcess StochasticProcess Each sample point in S is associated with a sample functionx(t) X(t, s) is a randomprocess isanensembleofalltimefunctionstogetherwithaprobabilityrule X(tk , sj ) is a realization or sample function of the randomprocess {x1(tk),x2(tk),...,xn(tk)=X(tk,s1),X(tk,s2),...,X(tk,sn)} Ashok NShinde 5/28

24. StochasticProcess StochasticProcess Each sample point in S is associated with a sample functionx(t) X(t, s) is a randomprocess isanensembleofalltimefunctionstogetherwithaprobabilityrule X(tk , sj ) is a realization or sample function of the randomprocess {x1(tk),x2(tk),...,xn(tk)=X(tk,s1),X(tk,s2),...,X(tk,sn)} Probability rules assign probability to any meaningful event associated with an observation An observation is a sample function of the randomprocess Ashok NShinde 5/28

25. StochasticProcess StochasticProcess Each sample point in S is associated with a sample functionx(t) X(t, s) is a randomprocess isanensembleofalltimefunctionstogetherwithaprobabilityrule X(tk , sj ) is a realization or sample function of the randomprocess {x1(tk),x2(tk),...,xn(tk)=X(tk,s1),X(tk,s2),...,X(tk,sn)} Probability rules assign probability to any meaningful event associated with an observation An observation is a sample function of the randomprocess A stochastic process X(t, s) is represented by time indexed ensemble (family) of random variables {X(t,s)} Ashok NShinde 5/28

26. StochasticProcess StochasticProcess Each sample point in S is associated with a sample functionx(t) X(t, s) is a randomprocess isanensembleofalltimefunctionstogetherwithaprobabilityrule X(tk , sj ) is a realization or sample function of the randomprocess {x1(tk),x2(tk),...,xn(tk)=X(tk,s1),X(tk,s2),...,X(tk,sn)} Probability rules assign probability to any meaningful event associated with an observation An observation is a sample function of the randomprocess A stochastic process X(t, s) is represented by time indexed ensemble (family) of random variables {X(t,s)} Represented compactly by :X(t) Ashok NShinde 5/28

27. StochasticProcess StochasticProcess Each sample point in S is associated with a sample functionx(t) X(t, s) is a randomprocess isanensembleofalltimefunctionstogetherwithaprobabilityrule X(tk , sj ) is a realization or sample function of the randomprocess {x1(tk),x2(tk),...,xn(tk)=X(tk,s1),X(tk,s2),...,X(tk,sn)} Probability rules assign probability to any meaningful event associated with an observation An observation is a sample function of the randomprocess A stochastic process X(t, s) is represented by time indexed ensemble (family) of random variables {X(t,s)} Represented compactly by :X(t) “A stochastic process X(t) is an ensemble of time functions, which, together with a probability rule, assigns a probability to any meaningful event associated with an observation of one of the sample functions of the stochasticprocess”. Ashok NShinde 5/28

28. StochasticProcess: Stationary Vs Non-StationaryProcess StationaryProcess: Ashok NShinde 6/28

29. StochasticProcess: Stationary Vs Non-StationaryProcess StationaryProcess: If a process is divided into a number of time intervalsexhibiting same statistical properties, is called asStationary. Ashok NShinde 6/28

30. StochasticProcess: Stationary Vs Non-StationaryProcess StationaryProcess: If a process is divided into a number of time intervalsexhibiting same statistical properties, is called asStationary. It is arises from a stable phenomenon that has evolved into a steady-state mode ofbehavior. Ashok NShinde 6/28

31. StochasticProcess: Stationary Vs Non-StationaryProcess StationaryProcess: If a process is divided into a number of time intervalsexhibiting same statistical properties, is called asStationary. It is arises from a stable phenomenon that has evolved into a steady-state mode ofbehavior. Non-StationaryProcess: Ashok NShinde 6/28

32. StochasticProcess: Stationary Vs Non-StationaryProcess StationaryProcess: If a process is divided into a number of time intervalsexhibiting same statistical properties, is called asStationary. It is arises from a stable phenomenon that has evolved into a steady-state mode ofbehavior. Non-StationaryProcess: If a process is divided into a number of time intervalsexhibiting different statistical properties, is called asNon-Stationary. Ashok NShinde 6/28

33. StochasticProcess: Stationary Vs Non-StationaryProcess StationaryProcess: If a process is divided into a number of time intervalsexhibiting same statistical properties, is called asStationary. It is arises from a stable phenomenon that has evolved into a steady-state mode ofbehavior. Non-StationaryProcess: If a process is divided into a number of time intervalsexhibiting different statistical properties, is called as Non-Stationary. It is arises from an unstablephenomenon. Ashok NShinde 6/28

34. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary The Stochastic process X(t) initiated at t = −∞ is said to be Stationary in the strict sense, or strictly stationaryif, Ashok NShinde 7/28

35. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary The Stochastic process X(t) initiated at t = −∞ is said to be Stationary in the strict sense, or strictly stationaryif, FX(t1+τ),X(t2+τ),...,X(tk+τ)(x1,x2,...,xk)= FX(t1),X(t2),...,X(tk)(x1,x2,...,xk)Where, Ashok NShinde 7/28

36. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary The Stochastic process X(t) initiated at t = −∞ is said to be Stationary in the strict sense, or strictly stationaryif, FX(t1+τ),X(t2+τ),...,X(tk+τ)(x1,x2,...,xk)= FX(t1),X(t2),...,X(tk)(x1,x2,...,xk)Where, X(t1),X(t2),...,X(tk)denotesRVsobtainedbysamplingprocess X(t)att1,t2,...,tkrespectively. Ashok NShinde 7/28

37. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary The Stochastic process X(t) initiated at t = −∞ is said tobe Stationary in the strict sense, or strictly stationary if, FX(t1+τ),X(t2+τ),...,X(tk+τ)(x1,x2,...,xk)= FX(t1),X(t2),...,X(tk)(x1,x2,...,xk)Where, X(t1),X(t2),...,X(tk) denotesRVsobtainedbysamplingprocess X(t)att1,t2,...,tkrespectively. FX(t1),X(t2),...,X(tk)(x1,x2,...,xk)denotesJointdistribution function ofRVs. Ashok NShinde 7/28

38. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary The Stochastic process X(t) initiated at t = −∞ is said tobe Stationary in the strict sense, or strictly stationaryif, FX(t1+τ),X(t2+τ),...,X(tk+τ)(x1,x2,...,xk)= FX(t1),X(t2),...,X(tk)(x1,x2,...,xk)Where, X(t1),X(t2),...,X(tk) denotesRVsobtainedbysamplingprocess X(t)att1,t2,...,tkrespectively. FX(t1),X(t2),...,X(tk)(x1,x2,...,xk)denotesJointdistribution function ofRVs. X(t1+τ),X(t2+τ),...,X(tk+τ)denotesnewRVsobtainedby samplingprocessX(t)att1+τ,t2+τ,...,tk+τrespectively.Here τ is fixed timeshift. Ashok NShinde 7/28

39. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary The Stochastic process X(t) initiated at t = −∞ is said tobe Stationary in the strict sense, or strictly stationaryif, FX(t1+τ),X(t2+τ),...,X(tk+τ)(x1,x2,...,xk)= FX(t1),X(t2),...,X(tk)(x1,x2,...,xk)Where, X(t1),X(t2),...,X(tk) denotesRVsobtainedbysamplingprocess X(t)att1,t2,...,tkrespectively. FX(t1),X(t2),...,X(tk)(x1,x2,...,xk)denotesJointdistribution function ofRVs. X(t1+τ),X(t2+τ),...,X(tk+τ)denotesnewRVsobtainedby samplingprocessX(t)att1+τ,t2+τ,...,tk+τrespectively.Here τ is fixed timeshift. FX(t1+τ),X(t2+τ),...,X(tk+τ)(x1,x2,...,xk)denotesJoint distribution function of newRVs. Ashok NShinde 7/28

40. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Ashok NShinde 8/28

41. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Ashok NShinde 8/28

42. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Ashok NShinde 8/28

43. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Weakly StationaryProcess: Ashok NShinde 8/28

44. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Weakly StationaryProcess: A stochastic process X(t) is said to be weakly stationary(Wide-Sense Stationary) if its second-order momentssatisfy: Ashok NShinde 8/28

45. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Weakly StationaryProcess: A stochastic process X(t) is said to be weakly stationary(Wide-Sense Stationary) if its second-order momentssatisfy: The mean of the process X(t) is constant for all timet. Ashok NShinde 8/28

46. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Weakly StationaryProcess: A stochastic process X(t) is said to be weakly stationary(Wide-Sense Stationary) if its second-order momentssatisfy: The mean of the process X(t) is constant for all timet. The autocorrelation function of the process X(t) depends solely on thedifferencebetweenanytwotimesatwhichtheprocessissampled. Ashok NShinde 8/28

47. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Weakly StationaryProcess: A stochastic process X(t) is said to be weakly stationary(Wide-Sense Stationary) if its second-order momentssatisfy: The mean of the process X(t) is constant for all timet. The autocorrelation function of the process X(t) depends solely on thedifferencebetweenanytwotimesatwhichtheprocessissampled. Summary of RandomProcesses Ashok NShinde 8/28

48. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Weakly StationaryProcess: A stochastic process X(t) is said to be weakly stationary(Wide-Sense Stationary) if its second-order momentssatisfy: The mean of the process X(t) is constant for all timet. The autocorrelation function of the process X(t) depends solely on thedifferencebetweenanytwotimesatwhichtheprocessissampled. Summary of RandomProcesses Wide-Sense Stationaryprocesses Ashok NShinde 8/28

49. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Weakly StationaryProcess: A stochastic process X(t) is said to be weakly stationary(Wide-Sense Stationary) if its second-order momentssatisfy: The mean of the process X(t) is constant for all timet. The autocorrelation function of the process X(t) depends solely on thedifferencebetweenanytwotimesatwhichtheprocessissampled. Summary of RandomProcesses Wide-Sense Stationaryprocesses Strictly StationaryProcesses Ashok NShinde 8/28

50. Classes of StochasticProcess: Strictly Stationary and Weakly Stationary Properties of Strictly StationaryProcess: Fork=1,wehaveFX(t)(x)=FX(t+τ)(x)=FX(x)foralltandτ. First-order distribution function of a strictly stationary stochastic process is independent of timet. Fork=2,wehaveFX(t1),X(t2)(x1,x2)=FX(0),X(t1−t2)(x1,x2)for all t1 and t2. Second-order distribution function of a strictly stationary stochastic process depends only on the time difference between the sampling instants and not on timesampled. Weakly StationaryProcess: A stochastic process X(t) is said to be weakly stationary(Wide-Sense Stationary) if its second-order momentssatisfy: The mean of the process X(t) is constant for all timet. The autocorrelation function of the process X(t) depends solely on thedifferencebetweenanytwotimesatwhichtheprocessissampled. Summary of RandomProcesses Wide-Sense Stationaryprocesses Strictly Stationary Processes ErgodicProcesses Ashok NShinde 8/28