Scatterplots regression
Download
1 / 60

Scatterplots & Regression - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

Scatterplots & Regression. Week 3 Lecture MG461 Dr. Meredith Rolfe. Key Goals of the Week. What is regression? When is regression used? Formal statement of linear model equation Identify components of linear model Interpret regression results:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Scatterplots & Regression' - jeanne


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Scatterplots regression

Scatterplots & Regression

Week 3 Lecture

MG461

Dr. Meredith Rolfe


Key goals of the week
Key Goals of the Week

  • What is regression?

  • When is regression used?

  • Formal statement of linear model equation

  • Identify components of linear model

  • Interpret regression results:

    • decomposition of variance and goodness of fit

    • estimated regression coefficients

    • significance tests for coefficients

MG461, Week 3 Seminar


Who has studied regression or linear models before
Who has studied regression or linear models before?

Taken before

Not taken before


Which group are you in
Which group are you in?

Group 1

Group 2

Group 3

Group 4

Group 5

Group 6

Group 7

Group 8

Which group are you in?


Regression is a set of statistical tools to model the conditional expectation
Regression is a set of statistical tools to model the conditional expectation…

of one variable on another variable.

of one variable on one or more other variables.


Linear model background
Linear Model Background conditional expectation…

MG461, Week 3 Seminar


Recap theoretical system
Recap: Theoretical System conditional expectation…

Y

X

MG461, Week 3 Seminar


What is regression
What is regression? conditional expectation…

Regression is the study of relationships between variables

It provides a framework for testing models of relationships between variables

Regression techniques are used to assess the extent to which the outcome variable of interest, Y, changes dependent on changes in the independent variable(s), X

MG461, Week 3 Seminar


Conditional dependence correct answer vs prior exposure
Conditional Dependence: conditional expectation…Correct Answer vs. Prior Exposure


When to use regression
When to use Regression conditional expectation…

  • We want to know whether the outcome, y, varies depending on x

    • We can use regression to study correlation (not causation) or make predictions

  • Continuous variables (but many exceptions)

MG461, Week 3 Seminar


Questions we might care about
Questions we might care about conditional expectation…

Does an change in X lead to an change in Y

Do higher paid employees contribute more to organizational success?

Do large companies earn more? Do they have lower tax rates?

MG461, Week 3 Seminar


How to answer the questions
How to answer the questions? conditional expectation…

Observational (Field) Study

Experimental Study

Random assignment to different contribution levels or levels of pay (?)

? Random assignment to country and/or tax rate, quasi-experiment?

  • Collect data on income and a measure of contributions to the organization

  • Collect data on corporate profits in various regions (which companies, which regions)

MG461, Week 3 Seminar


When to use regression1
When to use Regression conditional expectation…

  • We want to know whether the outcome, y, varies depending on x

  • Continuous variables (but many exceptions)

  • Observational data (mostly)

MG461, Week 3 Seminar


Example 1 pay and performance
Example 1: Pay and Performance conditional expectation…

Pay

Performance

Runs

Yearly Salary

Y

X

MG461, Week 3 Seminar


Scatterplot salaries vs runs
Scatterplot: conditional expectation…Salaries vs. Runs

Y

X

MG461, Week 3 Seminar


Scatterplot salaries vs runs1
Scatterplot: conditional expectation…Salaries vs. Runs

Δy

Δx

MG461, Week 3 Seminar


What is the equation for a line
What is the equation for a line? conditional expectation… What is the equation for a line?

y=ax2

y=ax

y=ax+b

y=x+b


Equation of a regression line
Equation of a (Regression) Line conditional expectation…

Population Parameters

Intercept

Slope

But… x and y are random variables, we need an equation that accounts for noise and signal

MG461, Week 3 Seminar


Simple linear model
Simple conditional expectation…Linear Model

Observation or data point, i, goes from 1…n

Error

Intercept

Coefficient

(Slope)

Dependent

Variable

Independent

Variable

MG461, Week 3 Seminar


The relationship must be linear
The relationship must be LINEAR conditional expectation…

  • The linear model assumes a LINEAR relationship

  • You can get results even if the relationship is not linear

  • LOOK at the data!

  • Check for linearity

MG461, Week 3 Seminar


When to use regression2
When to use Regression conditional expectation…

We want to know whether the outcome, y, varies depending on x

Continuous variables (but many exceptions)

Observational data (mostly)

The relationship between x and y is linear

MG461, Week 3 Seminar


Understanding the key points
Understanding the key points conditional expectation…

What is regression?

When is regression used?

Formal statement of linear model equation

MG461, Week 3 Seminar


Understand what regression is
Understand what regression is… conditional expectation…

Strongly Agree

Agree

Disagree

Strongly Disagree

Mean =

Median =


Understand when to use regression
Understand when to use regression conditional expectation…

Strongly Agree

Agree

Disagree

Strongly Disagree

Mean =

Median =


Know how to make formal statement of linear model
Know how to make conditional expectation…formal statement of linear model

Strongly Agree

Agree

Disagree

Strongly Disagree

Mean =

Median =


Discussion
Discussion conditional expectation…

What is regression?

When is regression used?

Formal statement of linear model equation

MG461, Week 3 Seminar

26


Model estimation
Model Estimation conditional expectation…

MG461, Week 3 Seminar


Which model parameter do we not need to estimate
Which model parameter conditional expectation…do we NOT need to estimate?

β0

β1

xi

σ2


Goal estimate the relationship between x and y
Goal: Estimate the Relationship between X and Y conditional expectation…

("beta hat one")

("beta hat zero")

  • We can also estimate the error variance σ2 as

Estimate the population parameters

β0 and β1

MG461, Week 3 Seminar


What would good estimates do
What would “good” estimates do? conditional expectation…

Minimize explained variance

Minimize distance to outliers

Minimize unexplained variance


Finding the best line unexplained variance
Finding the Best Line: conditional expectation…Unexplained Variance

ei

MG461, Week 3 Seminar


Ordinary least squares ols criteria
Ordinary Least Squares (OLS) Criteria conditional expectation…

("beta hat one")

("beta hat zero")

Minimize “noise” (unexplained variance) defined as residual sum of squares (RSS)

MG461, Week 3 Seminar


Ols estimates of beta hat
OLS conditional expectation…Estimates of Beta-hat

MG461, Week 3 Seminar


Mean of x and y
Mean of conditional expectation…x and y

MG461, Week 3 Seminar


Variance of x
Variance of conditional expectation…x

MG461, Week 3 Seminar


Variance of y
Variance of conditional expectation…y

MG461, Week 3 Seminar


Covariance of x and y
Covariance of conditional expectation…x and y

MG461, Week 3 Seminar


Ols estimates of beta hat1
OLS conditional expectation…Estimates of Beta-hat

Note the similarity between ß1 and the slope of a line: change in y over change in x (rise over run)

MG461, Week 3 Seminar


Why squared residuals
Why squared residuals? conditional expectation…

Y

X

Geometric intuition: X and Y are vectors, find the shortest line between them:

MG461, Week 3 Seminar


Decomposing the variance
Decomposing the Variance conditional expectation…

  • As in Anova, we now have:

    • Explained Variation

    • Unexplained Variation

    • Total Variation

  • This decomposition of variance provides one way to think about how well the estimated model fits the data

MG461, Week 3 Seminar


Total squared residuals syy
Total conditional expectation…Squared Residuals (SYY)

MG461, Week 3 Seminar


Explained vs unexplained squared residuals
Explained vs. Unexplained conditional expectation…Squared Residuals

MG461, Week 3 Seminar


R 2 and goodness of fit
R conditional expectation…2 and goodness of fit

MG461, Week 3 Seminar

RSS (residual sum of squares) =

unexplained variation

TSS (total sum of squares) =

SYY, total variation of dependent variable y

ESS (explained sum of squares) =

explained variation (TSS-RSS)


Examples of high and low r 2
Examples of high and low conditional expectation…R2

Graph 1

Graph 2

MG461, Week 3 Seminar


Which graph had a high r 2
Which graph had a high conditional expectation…R2?

Graph 1

Graph 2


Recall that r 2 ess tss or 1 rss tss what values can r 2 take on
Recall that R conditional expectation…2= ESS/TSS or 1-(RSS/TSS). What values can R2 take on?

Can be any number

Any number between -1 and 1

Any number between 0 and 1

Any number between 1 and 100


Examples of high and low r 21
Examples of high and low R conditional expectation…2

R2=0.29

R2=0.87

MG461, Week 3 Seminar


Interpretation of hats
Interpretation of conditional expectation…ß-hats

ß-hat0 : intercept, value of yi when xi is 0

ß-hat1: average or expected change in yi for every 1 unit change in xi

MG461, Week 3 Seminar


Visualization of coefficients
Visualization of Coefficients conditional expectation…

Δy=β1

Δx=1

β0

MG461, Week 3 Seminar


Ols estimates pay for runs
OLS estimates: Pay for Runs conditional expectation…

Salaryi = Beta-hat0 + Beta-hat1 * Runsi + errori

MG461, Week 3 Seminar


Ols estimates of regression line
OLS estimates of Regression Line conditional expectation…

Salary = -34 + 27.47*Runs

MG461, Week 3 Seminar


Interpretation of hats1
Interpretation of conditional expectation…ß-hats

y (Salary) = -34 + 27.47x (Runs)

ß-hat0 : players with no runs don’t get paid (not really – come back to this next week!)

ß-hat1: Each additional run translates into $27,470in salary per year

OR: a difference of almost $1 million/year between a player with an average (median) and an above average (80%) number of runs

MG461, Week 3 Seminar


Significance of results
Significance of Results conditional expectation…

Model Significance

Coefficient Significance

H0: ß1=0, there is no relationship (covariation) between x and y

HA: ß1≠0, there is a relationship (covariation) between x and y

Application: a single estimated coefficient

Test: t-test

**assumes errors (ei) are normally distributed

  • H0: None of the 1 (or more) independent variables covary with the dependent variable

  • HA: At least one of the independent variables covaries with d.v.

  • Application: compare two fitted models

  • Test: Anova/F-Test

  • **assumes errors (ei) are normally distributed

MG461, Week 3 Seminar


Ols estimates pay for runs1
OLS estimates: Pay for Runs conditional expectation…

Assuming normality, we can derive estimated standard errors for the coefficients

MG461, Week 3 Seminar


Ols estimates pay for runs2
OLS estimates: Pay for Runs conditional expectation…

And using these, calculate a

t-statistic and test for whether or not the coefficients are equal to zero

MG461, Week 3 Seminar


Ols estimates pay for runs3
OLS estimates: Pay for Runs conditional expectation…

And finally, the probability of being wrong (Type 1) if we reject H0

MG461, Week 3 Seminar


Plotting confidence intervals
Plotting Confidence Intervals conditional expectation…

MG461, Week 3 Seminar


Agree or disagree the lecture was clear and easy to follow
Agree or Disagree, “The lecture conditional expectation…was clear and easy to follow”

Strongly Agree

Agree

Somewhat Agree

Neutral

Somewhat Disagree

Disagree

Strongly Disagree


Next time
Next time.. conditional expectation…

Multiple independent variable

OLS assumptions

What to do when OLS assumptions are violated

MG461, Week 3 Seminar


Team scores
Team Scores conditional expectation…


ad