1 / 111

Statistical Inference and Regression Analysis: Stat-GB.3302.30, Stat-UB.0015.01

Statistical Inference and Regression Analysis: Stat-GB.3302.30, Stat-UB.0015.01. Professor William Greene Stern School of Business IOMS Department Department of Economics. Part 2 – A Expectations of Random Variables. 2-A Expectations of Random Variables 2-B Covariance and Correlation

ailani
Download Presentation

Statistical Inference and Regression Analysis: Stat-GB.3302.30, Stat-UB.0015.01

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistical Inference and Regression Analysis: Stat-GB.3302.30, Stat-UB.0015.01 Professor William Greene Stern School of Business IOMS Department Department of Economics

  2. Part 2 – AExpectations of Random Variables 2-A Expectations of Random Variables 2-B Covariance and Correlation 2-C Limit Results for Sums

  3. Expected Value of a Random Variable Weighted average of the values taken by the variable

  4. Discrete Uniform • X = 1,2,…,J • Prob(X = x) = 1/J • E[X] = 1/J + 2/J + … + J/J = J(J+1)/2 * 1/J = (J+1)/2 • Expected toss of a die = 3.5 (J=6) • Expected sum of two dice = 7. Proof?

  5. Poisson ()

  6. Poisson (5)

  7. The St. Petersburg Paradox • Coin toss game. If first heads comes up on nth toss, you win $2n • Entry fee to play a game is $C • Expected value of the game = E[Win] -C + (½)21 + (½)222 + … + (½)k2k  Game has infinite value. Noone would pay very much to play. Why not?

  8. Continuous Random Variable

  9. Gamma Random Variable

  10. Gamma Function: (1/2)=

  11. Expected Value of a Linear Translation • Z = aX+b • E[Z] = aE[X] + b • Proof is trivial using the definition of the expected value and the fact that the density integrates to 1 to have E[b]=b.

  12. Normal(,) Variable • From the definition of the random variable,  is the mean. • Proof in Rice (119) uses the linear translation. • If X ~ N[0,1], X +  ~ N(,)

  13. Cauchy Random Variables • f(x)=(1/) 1/(1+x2) • Mean does not exist. No higher moments exist. • If X~N[0,1] and Y ~ N[0,1] then X/Y has the Cauchy distribution. • Many applications obtain estimates of interesting quantities as ratios of estimators that are normally distributed.

  14. Cauchy Random Sample

  15. Expected Value of a Function of X • Y=g(X) • One to one case • E[Y] = expected value of Y(X) – find the distribution of the new variable • E[g(X)] = x g(x)f(x) will equal E[Y] • Many to one case – similar argument. Proceed without the transformation of the random variable. • E[g(X)] is generally not equal to g(E[X]) if g(X) is not linear

  16. Linear Translation • Z = aX+b • E[Z] = E[aX+b] • E[Z] = aE[X] + b • Proof is trivial using the definition of the expected value and the fact that the density integrates to 1 to E[b]=b.

  17. Powers of x - Moments • Moment = E[Xk] for positive integer x • Raw moment: E[Xk] • Central moment: E[(X – E[X])k] • Standard notation • E[Xk] = k • E[(X – E[X])k] = k • Mean = 1 = 

  18. Variance as a g(X) • Variance = E[(X – E[X])2] • Standard deviation = square root of variance is usually more interesting

  19. Variance of a Translation: Y = a + bX • Var[a] = 0 • Var[bX] = b2Var[X] • Standard deviation of Y = |b|S.D.(X)

  20. Shortcut • Var[X] = E[X2] - {E[X]}2

  21. Bernoulli • Prob(X=1)=; Prob(X=0)=1- • E[X] = 0(1- ) + 1 =  • E[X2] = 02(1- ) + 12 =  • Var[X] =  - 2 = (1-)

  22. Poisson: Factorial Moment

  23. Normal Moments

  24. Gamma Random Variable

  25. Chi Squared [1] • Chi squared [1] = Gamma(½, ½) P = ½ ,  = ½ • Mean = P/ = (½)/(½) = 1 • Variance = P/2= (½)/[(½)2] = 2

  26. Higher Moments • Skewness: 3. • 0 for all symmetric distributions (not just the normal) • Standardized measure 3/3 • Kurtosis: 4. • Standardized 4/4. • Compare to normal, 3 • Degree of excess = 4/4 – 3.

  27. Symmetric and Skewed Distributions

  28. Kurtosis: t[5] vs. Normal Kurtosis of normal(0,1) = 3, Excess = 0 Excess Kurtosis of t[k] = 6/(k-4); for t[5] = 6/(5-4) = 6.

  29. Approximations for g(X) • g(X) = continuous function • g() exists • Continuous first derivative not equal to zero at  • Taylor series approximation around mu • g(X) = g() + g’()(X- ) + ½ g’’()(X- )2 (+ higher order terms)

  30. Approximation to the Mean • g(X) ~ g() + g’()(X- ) + ½ g’’()(X -)2 • E[g(X)] ~ E[approximation] = g() + 0 + ½ g’’() E[(X -)2] = g() + ½ g’’()2

  31. Example: N[, ]. g(X)=exp(X). True mean = exp( +  2/2). Approximation: = exp() + ½ exp()  2 Example:  =0, s = 1, True mean = exp(.5) = 1.6487 Approximation = exp(0) + .5*exp(0)*1 = 1.5000

  32. Delta method: Var[g(X)] • Use linear approximation • g(X) ~ g() + g’()(X - ) • Var[g(X)] ~ Var[approximation] = [g’()]22 • Example: Var[X2] ~ (2)22

  33. Delta Method – x ~ N[, 2] • y = g(x) = exp(x) ~ lognormal • Exact • E[y] = exp( + ½ 2) • Var[y] = exp(2 + 2)[exp(2) – 1] • Approximate • E*[y] = exp() + ½ exp() 2 • V*[y] = [exp()]2 2 • N[0,1], exact mean and variance are exp(.5) =1.648 and exp(1)(exp(1)-1) = 4.671. Approximations are 1.5 and 1 (!)

  34. Moment Generating Function • Let g(X) = exp(tX) • M(t) = E[exp(tX)] = the moment generating function for random variable X.

  35. MGF Bernoulli • P(x) = (1-) for x=0 and  for x=1 • E[exp(tX)] = (1- )exp(0t) + exp(1t) = (1 - ) + exp(t).

  36. MGF Poisson

  37. MGF Gamma

  38. MGF Normal • MX(t) for X ~ N[0,1] is exp(½ t2) • MY(t) for Y = X +  isexp(t)MX(t) = exp[t + ½ 2t2] • This is the moment generating function for N[,2]

  39. Generating the Moments rth derivative of M(t) evaluated at t = 0 gives the rth raw moment, r’ M(r)(t) = drM(t)/dtr |t=0 = equals rth raw moment.

  40. Poisson MGF • M(t) = exp((exp(t) – 1)); M(0)=1 • M’(t) = M(t) * exp(t); M’(0)=  •  = M’(0)=1  1 =  • 2’ = E[X2] = M’’(0) = M’(0) exp(0) + exp(0)M(0) = 2 +  • Variance = 2’ - 2 = 

  41. Useful Properties • MGF of X = MX(t) and y = a+bX then • MY(t) for y is exp(at)MX(bt) • For independent X and Y, MX+Y (t) = is MX(t)MY(t) • The sequence of moments does not uniquely define the distribution

  42. Side Results • MGF MX(t) = E[exp(tx)] does not always exist. • Characteristic function E[exp(itx)] always exists. Used to prove central limit theorems • Cumulant generating function logMX(t)is sometimes useful. Cumulants are functions of moments. First cumulant is the mean, second is the variance.

  43. Part 2 – BCovariance and Correlation

  44. Covariance • Random variables X,Y with joint discrete distribution p(X,Y) or continuous density f(x,y). • Covariance = E({X – E[X]}{Y-E[Y]}) = E[XY] – E[X] E[Y]. • (Note, Covariance of X,X = Var[X]. • Connection to joint distribution and covariation

  45. Correlation and Covariance

  46. Correlated Populations

  47. Correlated Variables • X1 and X2 are independent with means 0 and standard deviations 1. • Y = aX1 + bX2. Choose a and b such that • X1 and Y have means 0, standard deviation 1 and correlation rho. • Var[Y] = a2 + b2 = 1 • Cov[X1,Y] = a = . b = sqr(1 – 2)

  48. Conditional Distributions • f(y|x) = f(y,x) / f(x) • Conditional distribution of y given a realization of x • Conditional mean = mean of the conditional random variable = regression function • Conditional variance = variance of conditional random variable = scedastic function

  49. Joint Normal Random Variables

  50. Conditional Normal

More Related