Numerical Analysis

1. Numerical Analysis EE, NCKU Tien-Hao Chang (Darby Chang)

2. In the previous slide • Accelerating convergence • linearly convergent • Newton’s method on a root of multiplicity >1 • (exercises) • Proceed to systems of equations • linear algebra review • pivoting strategies

3. In this slide • Error estimation in system of equations • vector/matrix norms • LU decomposition • split a matrix into the product of a lower and a upper triangular matrices • efficient in dealing with a lots of right-hand-side vectors • Direct factorization • as an systems of n2+n equations • Crout decomposition • Dollittle decomposition

4. 3.3 Vector and Matrix Norms

5. Vector and matrix norms • Pivoting strategies are designed to reduce the impact roundoff error • The size of a vector/matrix is necessary to measure the error

6. Vector norm

7. The two most commonly used norms in practice

10. Vector normEquivalent • One of the other uses of norms is to establish the convergence • Two trivial questions: • converge or diverge in different norms? • converge to different limit values in different norms? • The answer to both is no • all vector norms are equivalent

12. The Euclidean norm and the maximum norm are equivalent

13. Matrix norms • Similarly, there are various matrix norms, here we focus on those norms related to vector norms • natural matrix norms

14. Matrix normsNatural matrix norms

16. Natural matrix normsComputing maximum norm

19. Natural matrix normsComputing Euclidean norm • Is, unfortunately, not as straightforward as computing maximum matrix norms • Requires knowledge of the eigenvalues of the matrix

20. Eigenvalue review later

22. Eigenvalue review

24. http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpghttp://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg In action

27. 3.3 Vector and Matrix Norms

28. 3.4 Error Estimates and Condition Number

29. Error estimation • A linear system Ax=b, and x’ is an approximate solution • The error, e=x’-x,cannot be directly computed (x is never known) • The residue vector, r=Ax’-b, can be easily computed • r=0  x’=x  e=0

31. Is ||r|| a good estimation of ||e||? • Construct the relationship between r and e • From the definition • r=Ax’-b=Ax’-Ax=A(x’-x)=Ae hint#1 hint#2 hint#3 hint#4 answer

36. Condition number

38. Perturbations (skipped) . . .

39. 3.4 Error Estimates and Condition Number

40. 3.5 LU Decomposition

41. LU decompositionMotivation • Gaussian elimination solve a linear system, Ax=b, with n unknowns • (2/3)n3 + (3/2)n2 – (7/6)n • with back substitution • the minimum number of operations • If there are a lots of right-hand-side vectors • how many operations for a new RHS? • with Gaussian elimination, all operations are also carried out on the RHS

42. LU decomposition • Given a matrix A, a lower triangular matrix L and an upper triangular matrix U for which LU=A are said to form an LU decomposition of A • Here we replace mathematical descriptions with an example to show how use Gaussian elimination to obtain an LU decomposition

48. Is there any other LU decompositions in addition to using modified Gaussian elimination? • degree of freedoms (number of unknowns) • An2, LUn2+n • Direct factorization (3.6) • as an systems of n2+n equations hint answer

49. Solving a linear system • A LU • When a new RHS comes • Ax=bPAx=PbLUx=Pb • with z=Ux, actually to solveLz=Pb and Ux=z • both steps are easy • notice that Pb does not require real matrix-vector multiplication