Loading in 5 sec....

Linear Regression/CorrelationPowerPoint Presentation

Linear Regression/Correlation

- 79 Views
- Uploaded on
- Presentation posted in: General

Linear Regression/Correlation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

- Quantitative Explanatory and Response Variables
- Goal: Test whether the level of the response variable is associated with (depends on) the level of the explanatory variable
- Goal: Measure the strength of the association between the two variables
- Goal: Use the level of the explanatory to predict the level of the response variable

- Notation:
- Y: Response (dependent, outcome) variable
- X: Explanatory (independent, predictor) variable

- Linear Function (Straight-Line Relation):
- Y = a + b X (Plot Y on vertical axis, X horizontal)
- Slope (b): The amount Y changes when X increases by 1
- b > 0 Line slopes upward (Positive Relation)
- b = 0 Line is flat (No linear Relation)
- b < 0 Line slopes downward (Negative Relation)

- Y-intercept (a): Y level when X=0

- Slope (b): The amount Y changes when X increases by 1

- Internet History Resources (New South Wales Family History Document Service)
- Membership fee: $20A
- 20¢ ($0.20A) per image viewed
- Y = Total cost of service
- X = Number of images viewed
- a = Cost when no images viewed
- b = Incremental Cost per image viewed
- Y = a + b X = 20+0.20X

- In practice, the relationship between Y and X is not “perfect”. Other sources of variation exist. We decompose Y into 2 components:
- Systematic Relationship with X: a + bX
- Random Error: e

- Random respones can be written as the sum of the systematic (also thought of as the mean) and random components: Y = a + bX + e
- The (conditional on X) mean response is:
- E(Y) = a + bX

- Problem: a, b are unknown parameters, and must be estimated and tested based on sample data.
- Procedure:
- Sample n individuals, observing X and Y on each one
- Plot the pairs Y (vertical axis) versus X (horizontal)
- Choose the line that “best fits” the data.
- Criteria: Choose line that minimizes sum of squared vertical distances from observed data points to line. Least Squares Prediction Equation:

- Response (Y) - Math score (mean among 5 volunteers)
- Predictor (X) - LSD tissue concentration (mean of 5 volunteers)
- Raw Data and scatterplot of Score vs LSD concentration:

Source: Wagner, et al (1968)

(Column totals given in bottom row of table)

- U.S. SMSA’s
- Y = Per Capita Retail Sales
- X = Females per 100 Males

- Residuals (aka Errors): Difference between observed values and predicted values:
- Error sum of squares:
- Estimate of (conditional) standard deviation of Y:

- Data: Y = a + b X + e
- Mean: E(Y) = a + b X
- Conditional Standard Deviation: s
- Error terms (e) are assumed to be independent and normally distributed

- Slope of the regression describes the direction of association (if any) between the explanatory (X) and response (Y). Problems:
- The magnitude of the slope depends on the units of the variables
- The slope is unbounded, doesn’t measure strength of association
- Some situations arise where interest is in association between variables, but no clear definition of X and Y

- Population Correlation Coefficient: r
- Sample Correlation Coefficient: r

- Pearson Correlation: Measure of strength of linear association:
- Does not delineate between explanatory and response variables
- Is invariant to linear transformations of Y and X
- Is bounded between -1 and 1 (higher values in absolute value imply stronger relation)
- Same sign (positive/negative) as slope

- Using formulas for standard deviation from beginning of course: sX = 1.935 and sY = 18.611
- From previous calculations: b = -9.01

This represents a strong negative association between math scores and LSD tissue concentration

- Measure of the variation in Y that is “explained” by X
- Step 1: Ignoring X, measure the total variation in Y (around its mean):
- Step 2: Fit regression relating Y to X and measure the unexplained variation in Y (around its predicted values):
- Step 3: Take the difference (variation in Y “explained” by X), and divide by total:

TSS

SSE

- Parameter: Slope in the population model(b)
- Estimator: Least squares estimate: b
- Estimated standard error:
- Methods of making inference regarding population:
- Hypothesis tests (2-sided or 1-sided)
- Confidence Intervals

2-Sided Test

H0: b = 0

HA: b 0

1-sided Test

H0: b = 0

HA+: b> 0 or

HA-: b< 0

- Conclude positive association if entire interval above 0
- Conclude negative association if entire interval below 0
- Cannot conclude an association if interval contains 0
- Conclusion based on interval is same as 2-sided hypothesis test

- Testing H0: b = 0 vs HA: b 0

- 95% Confidence Interval for b :

t.025,5

- Goal: Partition the total variation in y into variation “explained” by x and random variation

- These three sums of squares and degrees of freedom are:
- Total (TSS) dfTotal = n-1
- Error (SSE) dfError = n-2
- Model (SSR) dfModel = 1

- Analysis of Variance - F-test
- H0: b = 0 HA: b 0

F represents the F-distribution with 1 numerator and n-2 denominator degrees of freedom

- Total Sum of squares:

- Error Sum of squares:

- Model Sum of Squares:

- Analysis of Variance - F-test
- H0: b = 0 HA: b 0

- Test identical (mathematically) to t-test for b, but more appropriate when no clear explanatory and response variable
- H0: r = 0 Ha: r 0 (Can do 1-sided test)
- Test Statistic:
- P-value: 2P(t|tobs|)

- Linearity: Many relations are not perfectly linear, but can be well approximated by straight line over a range of X values
- Extrapolation: While we can check validity of straight line relation within observed X levels, we cannot assume relationship continues outside this range
- Influential Observations: Some data points (particularly ones with extreme X levels) can exert a large influence on the predicted equation.