1 / 15

Linear Least-Squares Approximation

Linear Least-Squares Approximation. Ellen and Jason. Problem the Algorithm Solves. Finds a function that most cl early passes through a set of points Algorithm is used for Summarizing data Predicting data. Standard variable names & expressions. p = S X k q = S Y k r = S X k Y k

etana
Download Presentation

Linear Least-Squares Approximation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Linear Least-Squares Approximation Ellen and Jason

  2. Problem the Algorithm Solves • Finds a function that most clearly passes through a set of points • Algorithm is used for • Summarizing data • Predicting data

  3. Standard variable names & expressions • p = S Xk • q = S Yk • r = S XkYk • s = S (Xk)2 • d = (m + 1) s - (p)2 • a = [(m + 1) r - pq] / d • b =[sq – pr] / d • y = ax + b

  4. Standard Terms & Definitions • l1 Approximation: S | axk + b – yk | • l2 Approximation: j(a, b) = S (axk+ b – yk) 2

  5. Principle behind algorithm/How it was derived • S(yi – axi- b)2 This equation gives us the error.

  6. Principle behind algorithm/How it was derived df/dm = S 2(yi - mxi - b)(-xi) = S -2xiyi+ 2mxi2 + 2bxi Now we take the derivative with respect tom, and with respect to b. df/db = S 2(Yi - mxi - b)(-1) = S -2yi + 2mxi+ 2b

  7. Principle behind algorithm/How it was derived Now we want to set the equations to 0 because we are trying to find the minimum error. On the graph you can see that the minimum is at a point where the derivative would equal 0. • = S -2xiyi+ 2mxi2 + 2bxi • = S (-2xiyi) + m S (2xi2) + b S (2xi) • = S -2yi + 2mxi+ 2b • = S (-2yi) + m S (2xi) + b S (2)

  8. Principle behind algorithm/How it was derived • (-2xiyi) + m S (2xi2) + b S (2xi) = 0 • m S (2xi2) + b S (2xi) = S(2xiyi) • m S(xi2) + b S(xi) = S (xiyi) • (-2yi) + m S (2xi) + b S (2) = 0 • m S(2xi) + b S(2) = S(2yi) • m S(xi) + b = S(yi)

  9. Principle behind algorithm/How it was derived To get rid of variables A, C and m we multiply the first equation by C and the second equation by -A Am + Bb = E Cm + Db = F C(Am +Bb = E) -A(Cm + Db = F) => ACm + BCb= EC -ACm - ADb= -AF ----------------------- (BC – AD)b = EC – AF b = (EC-AF)/(BC-AD)

  10. Principle behind algorithm/How it was derived Now, in order to get rid of b, we multiply the first equation by D and the second equation by (-B) Am + Bb = E Cm + Db = F D(Am +Bb = E) -B(Cm + Db = F) => ADm+ BDb= ED -BCm - BDb= -BF ----------------------- (AD- BC)m = ED -BF m= (AD - BC)/(ED - BF)

  11. Example • Find the linear least-squares solution for the table of values p = 1 + 3 + 4 + 9 = 17 q = 2 + 6 + 9 + 8 = 25 r = (1*2) + (3*6) + (4*9) + (9*8) = 128 s = (12) + (32) + (42) + (92) = 107 d =(3+1)*107 - (172) = 139 a = [(3+1)*128- (17*25)] / 139 = 87/139 b = [(107*25) - (17*128)] / 139 = 499/139 Y = ax+b Y = (87/139)x + (499/139)

  12. Example: Graph

  13. Example: Error Point y - (ax +b) Point 1: [2 - (0.6259*1 +3.5900)] 2 = 4.9102 Point 2: [6 - (0.6259*3 + 3.5900)] 2 = 0.2833 Point 3: [9 - (0.6259*4 + 3.5900)] 2 = 8.4472 Point 4: [8 - (0.6259*9 + 3.5900)]2 = 1.4960 Error: 15.1367

  14. Advantages

  15. Disadvantages • Limitations in shapes • May not be effective for data that is nonlinear. For example, a linear function would not represent the sets of points in this graph very well.

More Related