Regression Analysis

1. Regression Analysis • The contents in this chapter are from Chapters 20-23 of the textbook. • The cntry15.sav data will be used. The data collected 15 countries’ information • lifeexpf: female life expectancy • Birthrat: births per 1000 population • Both are scale variables.

2. Linear regression model

3. Linear regression model • It is obviously, the points are not randomly scattered over the grid. Instead, there appears to be a pattern. • As birthrate increases, life expectancy decreases. • How to choose the “best” line? • The least squares principle is recommended.

4. Least squares principle

5. Least squares principle • Dependent variable: the variable you wish to predict • Independent variable: variables used to make the prediction • Simple linear regression: in which a single numerical independent variable X is used to predict the numerical dependent variable Y. where

6. Least squares principle

7. Least squares principle

8. Least squares principle

9. Linear regression model

10. Linear regression model • The regression model becomes • life expectancy=90-(0.70 x birthrate) That tells us that for an increase of 1 in birthrate, there is a decrease in life expectancy of 0.70 years.

11. Prediction and residuals

12. Coefficient of Correlation • It measures the strength of the linear relationship between two numerical variables.

13. Coefficient of Correlation • Coefficient of correlation • -1=< r >= 1

14. Prediction and residuals • The coefficient determination

15. ANOVA

16. Testing hypotheses about the assumptions • Independence: all of the observations are independent • The variance homogeneity: the variance of the distribution of the dependent variable must be the same for all values of the independent variable. • Normality: for each value of the independent variable, the distribution of the related dependent variable follows a normal distribution.

17. Testing hypotheses

18. Testing hypotheses • Testing that the slope is zero • In this example, the sample slope is about -0.70 and its standard error is 0.05, so the value for the t statistics is -0.70/0.05=-14, related p-value is less that 0.0005. We should reject the hypothesis. There appears to be a linear relationship between 1992 female life expectancy and birthrate. • The 95% confidence interval for the population slope is (-0.805, -0.590).

19. Prediction • The regression equation obtained can be used for predict the life expectancy based on birthrates. • For a country with a birthrate of 30 per 1000 population • Predicted life expectancy =89.99-0.697 x 30=69.08 years

20. Predicting means and individual observations • The plot on the next page gives the standard error of the predicted mean life expectancy for different values of birthrate. • The vertical line at 32.9 is the average birthrate for all cases. • The farther birthrates are from the sample mean, the larger the standard error of the predicted means.

21. Plot of standard error of predicted mean

22. The 95% fitting confidence region

23. Statistical diagnostics  Is the model correct?  Are there any outliers?  Is the variance constant?  Is the error normally distributed?

24. Statistical diagnostics • Residuals can provide many useful information for the above four issues in statistical diagnostics. • You can’t judge the related size of a residual by looking at its value alone as it depends on the unit of the dependent variable and are not convenient to use. • Standardized residuals: divide the residual by the estimated standard deviation of the residuals.

25. Statistical diagnostics • If the distribution of residuals is approximately normal, about 95% of the standardized residuals should be between -2 and 2; 99% should be between -2.58 and 2.58. It is easy to see whether there are some outliers.

26. Statistical diagnostics • When you compute a standardized residuals, all of the observed residuals are divided by the same number. • The variability of the dependent variable is not constant for all points, but depends on the value of the independent variable. • The studentized residual takes into account the differences in variability from point to point. • We calculate it by dividing the residual by an estimate of the standard deviation of the residual at that point.

27. Statistical diagnostics • A residual divided by an estimate of the standard deviation of the residual at that point is called its studentized residual. • The studentized residuals make it easier to see violations of the regression assumptions.

28. Statistical diagnostics

29. Standardized Residuals Standardized Residual Stem-and-Leaf Plot Frequency Stem & Leaf 3.00 -1 . 019 4.00 -0 . 0148 7.00 0 . 0466669 1.00 1 . 7 Stem width: 1.00000 Each leaf: 1 case(s)

30. Checking for normality

31. Checking for normality • If the data are a sample from a normal distribution, you expect the points to fall more or less on a straight line. • You can see the two largest residuals in absolute value (Thailand and Namibia) are stragglers from the line. • Next page is a detrended normality plot. If the data are from a normal, the points in the detrended normal plot should fall randomly in a band abound 0.

32. Checking for normality

33. Testing for normality • Many statistical tests for normality have been proposed, one of them is the Kolmogorov-Smirnov test.

34. Residual plot: plot of studentized residuals against the estimated values. From the residual plot you can see whether there are some pattern. For a normal case, the residuals appears to be randomly scattered around a horizontal line through 0. Checking for constant variance

35. Checking for constant variance

36. When the relationship between two variables is not linear, you can sometimes transform the variables to make the relationship linear, for example, take logarithm, sine, exponential, etc. Scale plot of female life expectancy against natural log of phones per 100. Checking linearity

37. Multiple Regression Models • Considering the country.sav data, you are interesting to predict female life expectancy from • Urban: percentage of the population living in urban areas • Docs: number of doctors per 10,000 people • Beds: number of hospital beds per 10,000 people • Gdp: per capita gross domestic product in dollars • Radios: radios per people

38. Multiple Regression Models • A linear regression model is • Scatterplot matrix is useful.

39. Scatterplot matrix

40. Scatterplot matrix • The relationship between female life expectancy and the percentage of the population living urban areas appears to be more or less linear. • The other four independent variables appear to be related to female life expectancy, but the relation is not linear. • We take log of the values of the four independent variables.

42. Correlation matrix

43. Regression coefficients • The estimated regression model Y=40.78-0.007 urban + 3.96 lndocs + 1.17 lnbeds +1.63 lngdp +1.54 lnradio

44. SPSS output: model summary statistics

45. SPSS output: ANOVA • This regression is meaningful as the significance level is less than 0.0005. • The residual variance is 22.489