1 / 9

Extension to Multiple Regression

Extension to Multiple Regression. Simple regression. With simple regression, we have a single predictor and outcome, and in general things are straightforward though issues may arise regarding outliers and violations of assumptions The basic model is Y = b 0 + b 1 X + e.

Download Presentation

Extension to Multiple Regression

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Extension to Multiple Regression

  2. Simple regression • With simple regression, we have a single predictor and outcome, and in general things are straightforward though issues may arise regarding outliers and violations of assumptions • The basic model is Y = b0 + b1X + e

  3. Multiple Regression • Adding more predictors sounds simple enough, and, true enough, the basic model doesn’t change much Y = b0 + b1X + b2X + b3X … + bnX + e • The key idea here is that we are getting a linear combination of the predictors1and assessing the correlation of that combination with the outcome • Squaring that correlation provides our model R2 • However there is much more to deal with

  4. Prediction and Explanation • Focus of the analysis can be seen as falling somewhere on two continuums of prediction and explanation • Low Prediction-----------------------------------High Prediction • Low Explanation-----------------------------------High Explanation • Example of high prediction/low explanation • Model R2 must be very strong, exclusive focus on raw coefficients, little concern about variable importance, and actual prediction on a new set of data • Example of high explanation/low prediction • Model R2 can even be weak but at least statistically significant, focus on standardized coefficients and variable importance, little if any validation • Most of psych research tends to fall in low prediction, high explanation • This is not a good thing as it leads to satisfaction with models that may be marginally useful at best

  5. Macro Analysis • Model fit • Statistical significance • Amount of variance accounted for in the DV (R2) • Standard error of estimate • The interpretation is no different with the addition of predictors • However, do know that as we have noted, there is never a zero correlation between variables unless forced (e.g. via experimental design) • As such, in practice adding a variable will always increase R2 • A bias-adjusted R2 becomes even more important to report as model complexity increases.

  6. Micro Analysis • Raw coefficients • Standardized coefficients • As if we ran the model after standardizing our predictors first • This puts them and their subsequent coefficients on the same scale for easier interpretation • However, just because one is larger than another doesn’t mean it is statistically or meaningfully so • Interval estimates for coefficients • Measures of variable importance • E.g. semi-partial correlation

  7. Problems • Issues with outliers, violations of the assumptions, and overfitting remain and the integrity of the model must be thoroughly examined • Furthermore, one must be on the look out for things like collinearity (high correlations among predictors) and suppression (unusual coefficients due to predictor-DV relationships)

  8. Multiple Approaches • Sequential (hierarchical) regression • Exploratory (stepwise) regression • Tests of interactions (moderation) • Mediating effects • Model comparison

  9. Summary • Most of the regressions you see will exhibit these types of problems: • Focus on explanation at the expense of prediction • Lack of or inadequately tested assumptions • Lack of bias-adjusted model fit indices • Lack of comparison of results to robust regression nor validation of their own model (much less comparison it to other theoretically motivated possible models) • Inadequately examine differences among predictors • Poorly perform exploratory approaches when doing them • Are a product of someone not doing more appropriate and complex analysis (e.g. path analysis) • In short, while commonly used, MR is also usually poorly performed • Try and do better!

More Related