Gauss and the Method of Least Squares
Download
1 / 25

Gauss and the Method of Least Squares - PowerPoint PPT Presentation


  • 675 Views
  • Updated On :

Gauss and the Method of Least Squares. Teddy Petrou Hongxiao Zhu. Outline. Who was Gauss? Why was there controversy in finding the method of least squares? Gauss ’ treatment of error Gauss ’ derivation of the method of least squares Gauss ’ derivation by modern matrix notation

Related searches for Gauss and the Method of Least Squares

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Gauss and the Method of Least Squares' - bono


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Gauss and the Method of Least Squares

Teddy Petrou Hongxiao Zhu


Outline

  • Who was Gauss?

  • Why was there controversy in finding the method of least squares?

  • Gauss’ treatment of error

  • Gauss’ derivation of the method of least squares

  • Gauss’ derivation by modern matrix notation

  • Gauss-Markov theorem

  • Limitations of the method of least squares

  • References


Johann carl friedrich gauss
Johann Carl Friedrich Gauss

Born:1777 Brunswick, Germany

Died: February 23, 1855, Göttingen, Germany

By the age of eight during arithmetic class he astonished his teachers by being able to instantly find the sum of the first hundred integers.


Facts about gauss
Facts about Gauss

  • Attended Brunswick College in 1792, where he discovered many important theorems before even reaching them in his studies

  • Found a square root in two different ways to fifty decimal places by ingenious expansions and interpolations

  • Constructed a regular 17 sided polygon, the first advance in this matter in two millennia. He was only 18 when he made the discovery


Ideas of gauss
Ideas of Gauss

  • Gauss was a mathematical scientist with interests in so many areas as a young man including theory of numbers, to algebra, analysis, geometry, probability, and the theory of errors.

  • His interests grew, including observational astronomy, celestial mechanics, surveying, geodesy, capillarity, geomagnetism, electromagnetism, mechanism optics, and actuarial science.


Intellectual personality and controversy
Intellectual Personality and Controversy

  • Those who knew Gauss best found him to be cold and

    uncommunicative.

  • He only published half of his ideas and found no one to share his most valued thoughts.

  • In 1805 Adrien-Marie Legendre published a paper on the method of least squares. His treatment, however, lacked a ‘formal consideration of probability and it’s relationship to least squares’, making it impossible to determine the accuracy of the method when applied to real observations.

  • Gauss claimed that he had written colleagues concerning the use of least squares dating back to 1795


Formal arrival of least squares
Formal Arrival of Least Squares

  • Gauss

    • Published ‘The theory of the Motion of Heavenly Bodies’in 1809. He gave a probabilistic justification of the method,which was based on the assumption of a normal distribution of errors. Gauss himself later abandoned the use of normal error function.

    • Published‘Theory of the Combination of Observations Least Subject to Errors’in 1820s. He substituted the root mean square error for Laplace’s mean absolute error.

  • Laplace Derived the method of least squares (between1802 and 1820) from the principle that the best estimate should have the smallest ‘mean error’ -the mean of the absolute value of the error.


Treatment of Errors

  • Using probability theory to describe error

  • Error will be treated as a random variable

  • Two types of errors

    • Constant-associated with calibration

    • Random error


Error assumptions
Error Assumptions

  • Gauss began his study by making two assumptions

    • Random errors of measurements of the same type lie within fixed limits

    • All errors within these limits are possible, but not necessarily with equal likelihood



Mean and Variance

  • Define . In many cases assume k=0

  • Define mean square error as

  • If k=0 then the variance will equal


Reasons for
Reasons for

  • is always positive and is simple.

  • The function is differentiable and integrable unlike the absolute value function.

  • The function approximates the average value in cases where large numbers of observations are being considered,and is simple to use when considering small numbers of observations.


More on variance
More on Variance

If then variance equals .

Suppose we have independent random variables with standard deviation 1 and expected value 0. The linear function of total errors is given by

Now the variance of E is given as

This is assuming every error falls within standard deviations from the mean


Gauss derivation of the method of least squares
Gauss’ Derivation of the Method of Least Squares

  • Suppose a quantity, V=f(x), where V, x are unknown. We estimate V by an observation L.

    • If x is calculated by L, L~f(x), error will occur.

    • But if several quantities V,V’,V’’…depend on the same unknown x and they are determined by inexact observations, then we can recover x by some combinations of the observations.

  • Similar situations occur when we observe several quantities that depend on several unknowns.


Gauss derivation of the method of least squares1
Gauss’ Derivation of the Method of Least Squares


Gauss derivation of the method of least squares2
Gauss’ Derivation of the Method of Least Squares


Gauss’ Derivation of the Method of Least Squares

  • Solutions:

  • It’s still not obvious:

    How do these results relate with the least squares estimation?


Gauss derivation of the method of least squares3
Gauss’ Derivation of the Method of Least Squares

  • It can be proved that


Gauss derivation by modern matrix notation
Gauss’ derivation by modern matrix notation:


Gauss derivation by modern matrix notation1
Gauss’ derivation by modern matrix notation:

Gauss’ results are equivalent to the following lemma:



Limitation of the Method of Least Squares

  • Nothing is perfect:

    • This method is very sensitive to the presence of unusual data points. One or two outliers can sometimes seriously skew the results of a least squares analysis.


References
References

  • Gauss, Carl Friedrich, Translated by G. W. Stewart. 1995. Theory of the Combination of Observations Least Subject to Errors: Part One, Part Two, Supplement. Philadelphia: Society for Industrial and Applied Mathematics.

  • Plackett, R. L. 1949. A Historical Note on the Method of Least Squares. Biometrika. 36:458–460.

  • Stephen M. Stiger, Gauss and the Invention of Least Squares. The Annals of Statistics, Vol.9, No.3(May,1981),465-474.

  • Plackett, Robin L. 1972. The Discovery of the Method of Least Squares. Plackett, Robin L. 1972. The Discovery of the Method of Least Squares.

  • Belinda B.Brand, Guass’ Method of Least Squares: A historically-based introduction. August 2003

  • http://www.infoplease.com/ce6/people/A0820346.html

  • http://www.stetson.edu/~efriedma/periodictable/html/Ga.html


ad