1 / 32

MULTIPLE REGRESSION

MULTIPLE REGRESSION. OVERVIEW. What Makes it Multiple? Additional Assumptions Methods of Entering Variables Adjusted R 2 Using z-Scores Categorical Predictors. WHAT MAKES IT MULTIPLE?. Predict from a combination of two or more predictor (X) variables

hada
Download Presentation

MULTIPLE REGRESSION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MULTIPLE REGRESSION

  2. OVERVIEW • What Makes it Multiple? • Additional Assumptions • Methods of Entering Variables • Adjusted R2 • Using z-Scores • Categorical Predictors

  3. WHAT MAKES IT MULTIPLE? • Predict from a combination of two or more predictor (X) variables • The regression model may account for more variance with more predictors • Look for predictor variables with low inter-correlations

  4. Multiple Regression Equation • Like simple regression, use a linear equation to predict Y scores. • Use the least squares solution.

  5. Assumptions for Regression • Quantitative data (or dichotomous) • Independent observations • Predict for same population that was sampled • Linear relationship

  6. Assumptions for Regression • Homoscedasticity • Independent errors • Normality of errors

  7. ADDITIONAL ASSUMPTIONS • Large ratio of sample size to number of predictor variables • Minimum 15 subjects per predictor variable • Predictor variables are not strongly intercorrelated (no multicollinearity) • Examine VIF – should be close to 1

  8. Multicollinearity • When predictor variables are highly intercorrelated with each other, prediction accuracy is not as good • Be cautious about determining which predictor variable is predicting the best when there is high collinearity among the predictors

  9. METHODS OF ENTERING VARIABLES • Simultaneous • Hierarchical/Block Entry • Stepwise • Forward • Backward • Stepwise

  10. Simultaneous Multiple Regression • All predictor variables are entered into the regression at the same time • Allows you to determine portion of variance explained by each predictor with the others statistically controlled (part correlation)

  11. Hierarchical Multiple Regression • Enter variables in a particular order based on a theory or on prior research • Can be done with blocks of variables

  12. Stepwise Multiple Regression • Enter or remove predictor variables one at a time based on explaining significant portions of variance in the criterion • Forward • Backward • Stepwise

  13. Forward Stepwise • begin with no predictor variables • add predictors one at a time according to which one will result in the largest increase in R2 • stop when R2 will not be significantly increased

  14. Backward Stepwise • begin with all predictor variables • remove predictors one at a time according to which one will result in the smallest decrease in R2 • stop when R2 would be significantly decreased • may uncover suppressor variables

  15. Suppressor Variable • Predictor variable which, when entered into the equation, increases the amount of variance explained by another predictor variable • In backward regression, removing the suppressor would likely result in a significant decrease in R2, so it will be left in the equation

  16. Suppressor Variable Example • Y = Job Performance Rating • X1 = College GPA • X2 = Writing Test Score

  17. Suppressor Variable Example • Let’s say Writing Score is not correlated with Job Performance, because the job doesn’t require much writing • Let’s say GPA is only a weak predictor of Job Performance, but it seems like it should be a good predictor

  18. Suppressor Variable Example • Let’s say GPA is “contaminated” by differences in writing ability – really good writers can fake and get higher grades • So, if Writing Score is in the equation, the contamination is removed, and we get a better picture of the GPA-Job Performance relationship

  19. Stepwise • begin with no predictor variables • add predictors one at a time according to which one will result in the largest increase in R2 • at each step remove any variable that does not explain a significant portion of variance • stop when R2 will not be significantly increased

  20. Choosing a Stepwise Method • Forward • Easier to conceptualize • Provides efficient model for predicting Y • Backward • Can uncover suppressor effects • Stepwise • Can uncover suppressor effects • Tends to be unstable with smaller N’s

  21. ADJUSTED R2 • R2 may overestimate the true amount of variance explained • Adjusted R2 compensates by reducing the R2 according to the ratio of subjects per predictor variable

  22. BETA WEIGHTS • The regression weights can be standardized into beta weights • Beta weights do not depend on the scales of the variables • A beta weight indicates the amount of change in Y in units of SD for each SD change in the predictor

  23. Example of Reporting Results of Multiple Regression We performed a simultaneous multiple regression with vocabulary score, abstraction score, and age as predictors and preference for intense music as the dependent variable. The equation accounted for a significant portion of variance, F(3,66) = 4.47, p = .006. As shown in Table 1, the only significant predictor was abstraction score.

  24. Regression with a Categorical Predictor Variable • Recall that predictor variables must be quantitative or dichotomous • Categorical variables that are not dichotomous can be used, but first they must be recoded to be either quantitative or dichotomous

  25. Ways to Code a Categorical Variable • Dummy Coding • Effect Coding • Orthogonal Coding • Criterion Coding

  26. Dummy Coding • Test for the overall effect of the predictor variable • 1 indicates being in that category and 0 indicates not being in that category; need one fewer dummy variables than categories

  27. Dummy Coding Example • We are trying to predict happiness rating using region of the country as a predictor variable • Three regions: • Northeast • Southeast • West

  28. Dummy Coding Example • Dummy Variable 1: Northeast = 1 Southeast = 0 West = 0 • Dummy Variable 2: Northeast = 0 Southeast = 1 West = 0 • We don’t need a third dummy variable, because West is indicated by 0’s on both dummy variables

  29. Effect Coding • Compare specific categories to each other • Use weights to indicate the intended contrast

  30. Orthogonal Coding • Same as Effect Coding, except that the contrasts are orthogonal to each other • You can do a maximum of k-1 orthogonal contrasts, where k is the number of categories

  31. Criterion Coding • Overall relationship between predictor and criterion variable • Each individual is assigned the mean score of the category

More Related