LU Decomposition, Pivoting, and Orthogonalization: A Comprehensive Lecture Overview

1. Lecture 3 LU Decomposition Pivoting Diagonalization Gram-Shcmidt Orthogonalization Lecture 3

2. LU Decomposition A=LU Ax=b LUx=b Define Ux=y Ly=b Solve y by forward substitution ERO’s must be performed on b as well as A The information about the ERO’s are stored in L Indeed y is obtained by applying ERO’s to b vector Ux=y Solve x by backward substitution Lecture 3

3. LU Decomposition by Gaussian elimination U=Gaussian eliminated matrix L=Multipliers used for elimination Compact storage: The diagonal entries of L matrix are all 1’s, they don’t need to be stored. LU is stored in a single matrix. Lecture 3

4. Pivoting • Computer uses finite-precision arithmetic • A small error is introduced in each arithmetic operation, error propagates • When the pivotal element is very small, the multipliers will be large. • Adding numbers of widely differening magnitude can lead to loss of significance. • To reduce error, row interchanges are made to maximise the magnitude of the pivotal element Lecture 3

5. Example: Without Pivoting 4-digit arithmetic Loss of significance Lecture 3

6. Example: With Pivoting Lecture 3

7. Pivoting procedures Pivotal row Eliminated part Pivotal column Lecture 3

8. Row pivoting • Most commonly used partial pivoting procedure • Search the pivotal column • Find the largest element in magnitude • Then switch this row with the pivotal row Lecture 3

9. Row pivoting Interchange these rows Largest in magnitude Lecture 3

10. Column pivoting Largest in magnitude Interchange these columns Lecture 3

11. Complete pivoting Interchange these rows Largest in magnitude Interchange these columns Lecture 3

12. Row Pivoting in LU Decomposition • When two rows of A are interchanged, those rows of b should also be interchanged. • Use a pivot vector. Initial pivot vector is integers from 1 to n. • When two rows (i and j) of A are interchanged, apply that to pivot vector. Lecture 3

13. Modifying the b vector • When LU decomposition of A is done, the pivot vector tells the order of rows after interchanges • Before applying forward substitution to solve Ly=b, modify the order of b vector according to the entries of pivot vector Lecture 3

14. Gauss-Jordan Diagonalization • The elements above the diagonal are made zero at the same time that zeros are created below the diagonal Lecture 3

15. Gauss-Jordan Diagonalization Lecture 3

16. Gauss-Jordan Diagonalization Lecture 3

17. Gram-Schmidt procedure for vector orthogonalization • The purpose we can construct a set of orthonormal vectors uifrom a set of n-dimensional vectors vi. 1≤i≤m • And vican be represented by the linear combination of ui Lecture 3

18. G-S procedure (cont.) • Select a vector from vi arbitrarily, say v1 • By normalizing its length, we obtain the first vector, say • Select v2 and subtract the projection of v2 ontou1, we get w2=v2– (v2• u1)u1 Lecture 3

19. G-S procedure (cont.) • And normalizing • Now we can check, that Lecture 3

20. G-S procedure (cont.) • The procedure continues by selecting v3 and subtract its projection on u1 and u2, we have w3=v3– (v3• u1)u1– (v3• u2)u2 7. Then, the orthonormal vector u3 is 8. By continuing this procedure, we shall construct the set of orthonormal vectors ui Lecture 3

21. G-S procedure: Example 1/3 Consider v1=(0,1,1); v2=(1,-1,0); v3=(1,0,1) They are not orthogonal: (vi,vj)≠0. Following G-S: Lecture 3

22. Example 2/3 We can check, that: Lecture 3

23. Example 3/3 Why? Lecture 3