Least Squares Regression Chapter 17

1 / 30

# Least Squares Regression Chapter 17 - PowerPoint PPT Presentation

Least Squares Regression Chapter 17. Linear Regression Fitting a straight line to a set of paired observations: (x 1 , y 1 ), (x 2 , y 2 ),…,(x n , y n ). y=a 0 +a 1 x+e a 1 - slope a 0 - intercept e- error, or residual, between the model and the observations.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Least Squares Regression Chapter 17' - wyatt-leon

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Least Squares RegressionChapter 17

Linear Regression

• Fitting a straight line to a set of paired observations: (x1, y1), (x2, y2),…,(xn, yn).

y=a0+a1x+e

a1- slope

a0- intercept

e- error, or residual, between the model and the observations

Chapter 17

Criteria for a “Best” Fit/
• Minimize the sum of the residual errors for all available data:

n = total number of points

• However, this is an inadequate criterion, so is the sum of the absolute values

Chapter 17

Figure 17.2

Chapter 17

Best strategy is to minimize the sum of the squares of the residuals between the measured y and the y calculated with the linear model:
• Yields a unique line for a given set of data.

Chapter 17

List-Squares Fit of a Straight Line/

Normal equations, can be solved simultaneously

Mean values

Figure 17.3

Chapter 17

Figure 17.4

Chapter 17

Figure 17.5

Chapter 17

“Goodness” of our fit/

If

• Total sum of the squares around the mean for the dependent variable, y, is St
• Sum of the squares of residuals around the regression line is Sr
• St-Sr quantifies the improvement or error reduction due to describing data in terms of a straight line rather than as an average value.

r2-coefficient of determination

Sqrt(r2) – correlation coefficient

For a perfect fit

Sr=0 and r=r2=1, signifying that the line explains 100 percent of the variability of the data.

• For r=r2=0, Sr=St, the fit represents no improvement.

Chapter 17

Polynomial Regression
• Some engineering data is poorly represented by a straight line. For these cases a curve is better suited to fit the data. The least squares method can readily be extended to fit the data to higher order polynomials (Sec. 17.2).

Chapter 17

General Linear Least Squares

Minimized by taking its partial derivative w.r.t. each of the coefficients and setting the resulting equation equal to zero

Chapter 17

InterpolationChapter 18
• Estimation of intermediate values between precise data points. The most common method is:
• Although there is one and only one nth-order polynomial that fits n+1 points, there are a variety of mathematical formats in which this polynomial can be expressed:
• The Newton polynomial
• The Lagrange polynomial

Chapter 17

Figure 18.1

Chapter 17

Newton’s Divided-Difference Interpolating Polynomials

Linear Interpolation/

• Is the simplest form of interpolation, connecting two data points with a straight line.
• f1(x) designates that this is a first-order interpolating polynomial.

Slope and a finite divided difference approximation to 1st derivative

Linear-interpolation formula

Chapter 17

Figure

18.2

Chapter 17

• If three data points are available, the estimate is improved by introducing some curvature into the line connecting the points.
• A simple procedure can be used to determine the values of the coefficients.
General Form of Newton’s Interpolating Polynomials/

Bracketed function evaluations are finite divided differences

Chapter 18

Errors of Newton’s Interpolating Polynomials/
• Structure of interpolating polynomials is similar to the Taylor series expansion in the sense that finite divided differences are added sequentially to capture the higher order derivatives.
• For an nth-order interpolating polynomial, an analogous relationship for the error is:
• For non differentiable functions, if an additional point f(xn+1) is available, an alternative formula can be used that does not require prior knowledge of the function:

x Is somewhere containing the unknown and he data

Chapter 17

Lagrange Interpolating Polynomials
• The Lagrange interpolating polynomial is simply a reformulation of the Newton’s polynomial that avoids the computation of divided differences:

Chapter 17

Figure 18.10

Chapter 17

Coefficients of an Interpolating Polynomial
• Although both the Newton and Lagrange polynomials are well suited for determining intermediate values between points, they do not provide a polynomial in conventional form:
• Since n+1 data points are required to determine n+1 coefficients, simultaneous linear systems of equations can be used to calculate “a”s.

Chapter 17

Figure 18.13

Chapter 17

Spline Interpolation
• There are cases where polynomials can lead to erroneous results because of round off error and overshoot.
• Alternative approach is to apply lower-order polynomials to subsets of data points. Such connecting polynomials are called spline functions.

Chapter 17

Figure 18.14

Chapter 17

Figure 18.15

Chapter 17

Figure 18.16

Chapter 17

Figure 18.17

Chapter 17