Lecture 12 multiple regression analysis
This presentation is the property of its rightful owner.
Sponsored Links
1 / 15

LECTURE 12 Multiple regression analysis PowerPoint PPT Presentation


  • 111 Views
  • Uploaded on
  • Presentation posted in: General

LECTURE 12 Multiple regression analysis. Epsy 640 Texas A&M University. Multiple regression analysis. The test of the overall hypothesis that y is unrelated to all predictors, equivalent to H 0 :  2 y  123… = 0 H 1 :  2 y  123… = 0 is tested by

Download Presentation

LECTURE 12 Multiple regression analysis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Lecture 12 multiple regression analysis

LECTURE 12Multiple regression analysis

Epsy 640

Texas A&M University


Multiple regression analysis

Multiple regression analysis

  • The test of the overall hypothesis that y is unrelated to all predictors, equivalent to

  • H0: 2y123… = 0

  • H1: 2y123… = 0

  • is tested by

  • F = [ R2y123… / p] / [ ( 1 - R2y123…) / (n – p – 1) ]

  • F = [ SSreg / p ] / [ SSe / (n – p – 1)]


Multiple regression analysis1

Multiple regression analysis

SOURCEdfSum of SquaresMean SquareF

x1, x2…pSSregSSreg / pSSreg/ 1

SSe /(n-p-1)

e (residual) n-p-1SSeSSe / (n-p-1)

total n-1SSySSy / (n-1)

  • Table 8.2: Multiple regression table for Sums of Squares


Multiple regression analysis predicting depression

Multiple regression analysis predicting Depression

LOCUS OF CONTROL, SELF-ESTEEM, SELF-RELIANCE


Lecture 12 multiple regression analysis

SSreg

ssx1

SSy

SSe

ssx2

Fig. 8.4: Venn diagram for multiple regression with two predictors and one outcome measure


Lecture 12 multiple regression analysis

Type I

ssx1

SSx1

SSy

SSe

SSx2

Type III

ssx2

Fig. 8.5: Type I contributions


Lecture 12 multiple regression analysis

Type III

ssx1

SSx1

SSy

SSe

SSx2

Type III

ssx2

Fig. 8.6: Type IIII unique contributions


Multiple regression anova table

Multiple Regression ANOVA table

SOURCEdfSum of SquaresMean SquareF

(Type I)

  • Model2SSregSSreg / 2SSreg / 2

  • SSe / (n-3)

  • x11 SSx1SSx1 / 1SSx1/ 1

  • SSe /(n-3)

  • x21 SSx2  x1SSx2  x1SSx2  x1/ 1

  • SSe /(n-3)

  • e n-3SSeSSe / (n-3)

  • total n-1SSySSy / (n-3)

  • Table 8.3: Multiple regression table for Sums of Squares of each predictor


Lecture 12 multiple regression analysis

PATH DIAGRAM FOR REGRESSION

 = .5

X1

.387

r = .4

Y

e

X2

 = .6

R2 = .742 + .82

- 2(.74)(.8)(.4)

 (1-.42)

= .85


Depression

Depression

e

.471

.4

LOC. CON.

-.317

DEPRESSION

SELF-EST

R2 = .60

-.186

SELF-REL


Shrinkage r 2

Shrinkage R2

  • Different definitions: ask which is being used:

    • What is population value for a sample R2?

    • R2s = 1 – (1- R2)(n-1)/(n-k-1)

    • What is the cross-validation from sample to sample?

    • R2sc = 1 – (1- R2)(n+k)/(n-k)


Estimation methods

Estimation Methods

  • Types of Estimation:

    • Ordinary Least Squares (OLS)

      • Minimize sum of squared errors around the prediction line

    • Generalized Least Squares

      • A regression technique that is used when the error terms from an ordinary least squares regression display non-random patterns such as autocorrelation or heteroskedasticity.

    • Maximum Likelihood


Maximum likelihood estimation

Maximum Likelihood Estimation

  • Maximum likelihood estimation

  • There is nothing visual about the maximum likelihood method - but it is a powerful method and, at least for large samples, very preciseMaximum likelihood estimation begins with writing a mathematical expression known as the Likelihood Function of the sample data. Loosely speaking, the likelihood of a set of data is the probability of obtaining that particular set of data, given the chosen probability distribution model. This expression contains the unknown model parameters. The values of these parameters that maximize the sample likelihood are known as theMaximum Likelihood Estimatesor MLE's.  Maximum likelihood estimation is a totally analytic maximization procedure.

  • MLE's and Likelihood Functions generally have very desirable large sample properties: 

    • they become unbiased minimum variance estimators as the sample size increases

    • they have approximate normal distributions and approximate sample variances that can be calculated and used to generate confidence bounds

    • likelihood functions can be used to test hypotheses about models and parameters 

  • With small samples, MLE's may not be very precise and may even generate a line that lies above or below the data pointsThere are only two drawbacks to MLE's, but they are important ones: 

    • With small numbers of failures (less than 5, and sometimes less than 10 is small), MLE's can be heavily biased and the large sample optimality properties do not apply

  • Calculating MLE's often requires specialized software for solving complex non-linear equations. This is less of a problem as time goes by, as more statistical packages are upgrading to contain MLE analysis capability every year.


Outliers

Outliers

  • Leverage (for a single predictor):

  • Li = 1/n + (Xi –Mx)2 / x2 (min=1/n, max=1)

  • Values larger than 1/n by large amount should be of concern

  • Cook’s Di = (Y – Yi) 2 / [(k+1)MSres]

    • the difference between predicted Y with and without Xi


Outliers1

Outliers

  • In SPSS Regression, under the SAVE option, both leverage and Cook’s D will be computed and saved as new variables with values for each case


  • Login