 Download Presentation Week 4

Loading in 2 Seconds...

# Week 4 - PowerPoint PPT Presentation

Week 4. Bivariate Regression, Least Squares and Hypothesis Testing. Lecture Outline. Method of Least Squares Assumptions Normality assumption Goodness of fit Confidence Intervals Tests of Significance alpha versus p. Recall . . . I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation ## Week 4

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
1. Week 4 Bivariate Regression, Least Squares and Hypothesis Testing

2. Lecture Outline • Method of Least Squares • Assumptions • Normality assumption • Goodness of fit • Confidence Intervals • Tests of Significance • alpha versus p IS 620 Spring 2006

3. Recall . . . • Regression curve as “line connecting the mean values” of y for a given x • No necessary reason for such a construction to be a line • Need more information to define a function IS 620 Spring 2006

4. Method of Least Squares • Goal:describe the functional relationship between y and x • Assume linearity (in the parameters) • What is the best line to explain the relationship? • Intuition:The line that is “closest” or “fits best” the data IS 620 Spring 2006

5. “Best” line, n = 2 IS 620 Spring 2006

6. “Best” line, n = 2 IS 620 Spring 2006

7. “Best” line, n > 2 ? IS 620 Spring 2006

8. “Best” line, n > 2 IS 620 Spring 2006

9. u2 u3 u1 Least squares: intuition IS 620 Spring 2006

10. Least squares, n > 2 IS 620 Spring 2006

11. Why sum of squares? • Sum of residuals may be zero • Emphasize residuals that are far away from regression line • Better describes spread of residuals IS 620 Spring 2006

12. Least-squares estimates Intercept Residuals Effect of x on y (slope) IS 620 Spring 2006

13. Gauss-Markov Theorem • Least-squares method produces best, linear unbiased estimators (BLUE) • Also most efficient (minimum variance) • Provided classic assumptions obtain IS 620 Spring 2006

14. Classical Assumptions • Focus on #3, #4, and #5 in Gujarati • Implications for estimators of violations • Skim over #1, #2, #6 through #10 IS 620 Spring 2006

15. #3: Zero mean value of ui • Residuals are randomly distributed around the regression line • Expected value is zero for any given observation of x • NOTE: Equivalent to assuming the model is fully specified IS 620 Spring 2006

16. #3: Zero mean value of ui IS 620 Spring 2006

17. #3: Zero mean value of ui IS 620 Spring 2006

18. #3: Zero mean value of ui IS 620 Spring 2006

19. #3: Zero mean value of ui IS 620 Spring 2006

20. #3: Zero mean value of ui IS 620 Spring 2006

21. #3: Zero mean value of ui IS 620 Spring 2006

22. #3: Zero mean value of ui IS 620 Spring 2006

23. #3: Zero mean value of ui IS 620 Spring 2006

24. Violation of #3 • Estimated betas will be • Unbiased but • Inconsistent • Inefficient • May arise from • Systematic measurement error • Nonlinear relationships (Phillips curve) IS 620 Spring 2006

25. #4: Homoscedasticity • The variance of the residuals is the same for all observations, irrespective of the value of x • “Equal variance” • NOTE: #3 and #4 imply (see “Normality Assumption”) IS 620 Spring 2006

26. #4: Homoscedasticity IS 620 Spring 2006

27. #4: Homoscedasticity IS 620 Spring 2006

28. #4: Homoscedasticity IS 620 Spring 2006

29. #4: Homoscedasticity IS 620 Spring 2006

30. #4: Homoscedasticity IS 620 Spring 2006

31. Violation of #4 • Estimated betas will be • Unbiased • Consistent but • Inefficient • Arise from • Cross-sectional data IS 620 Spring 2006

32. #5: No autocorrelation • The correlation between any two residuals is zero • Residual for xi is unrelated to xj IS 620 Spring 2006

33. #5: No autocorrelation IS 620 Spring 2006

34. #5: No autocorrelation IS 620 Spring 2006

35. #5: No autocorrelation IS 620 Spring 2006

36. #5: No autocorrelation IS 620 Spring 2006

37. Violations of #5 • Estimated betas will be • Unbiased • Consistent • Inefficient • Arise from • Time-series data • Spatial correlation IS 620 Spring 2006

38. Other Assumptions (1) • Assumption 6: zero covariance between xi and ui • Violations cause of heteroscedasticity • Hence violates #4 • Assumption 9: model correctly specified • Violations may violate #1 (linearity) • May also violate #3: omitted variables? IS 620 Spring 2006

39. Other Assumptions (2) • #7: n must be greater than number of parameters to be estimated • Key in multivariate regression • King, Keohane and Verba’s (1996) critique of small n designs IS 620 Spring 2006

40. Normality Assumption • Distribution of disturbance is unknown • Necessary for hypothesis testing of I.V.s • Estimates a function of ui • Assumption of normality is necessary for inference • Equivalent to assuming model is completely specified IS 620 Spring 2006

41. Normality Assumption • Central Limit Theorem: M&Ms • Linear transformation of a normal variable itself is normal • Simple distribution (mu, sigma) • Small samples IS 620 Spring 2006

42. Assumptions, Distilled • Linearity • DV is continuous, interval-level • Non-stochastic: No correlation between independent variables • Residuals are independently and identically distributed (iid) • Mean of zero • Constant variance IS 620 Spring 2006

43. If so, . . . • Least-squares method produces BLUE estimators IS 620 Spring 2006

44. Goodness of Fit • How “well” the least-squares regression line fits the observed data • Alternatively: how well the function describes the effect of x on y • How much of the observed variation in y have we explained? IS 620 Spring 2006

45. Coefficient of determination • Commonly referred to as “r2” • Simply, the ratio of explained variation in y to the total variation in y IS 620 Spring 2006

46. Components of variation explained total residual IS 620 Spring 2006

47. Components of variation • TSS: total sum of squares • ESS: explained sum of squares • RSS: residual sum of squares IS 620 Spring 2006

48. Hypothesis Testing • Confidence Intervals • Tests of significance • ANOVA • Alpha versus p-value IS 620 Spring 2006

49. Confidence Intervals • Two components • Estimate • Expression of uncertainty • Interpretation: • Gujarati, p. 121: “The probability of constructing an interval that contains Beta is 1-alpha” • NOT: “The p that Beta is in the interval is 1-alpha” IS 620 Spring 2006

50. C.I.s for regression • Depend upon our knowledge or assumption about the sampling distribution • Width of interval proportional to standard error of the estimators • Typically we assume • The t distribution for Betas • The chi-square distribution for variances • Due to unknown true standard error IS 620 Spring 2006