Scientific Computing

1. Scientific Computing QR Factorization Part 1 – Orthogonal Matrices, Reflections and Rotations

2. Orthogonal Matrices Matrix Factoring: So far, we have seen that we can factor a non-singular matrix A as A = LU, where L is lower triangular and U is upper triangular. QR Factoring: Similar idea to factor A = QR, where R is upper triangular and Q is orthogonal. Definition: A square matrix Q is orthogonal if Qt Q = I

3. Orthogonal Matrix Properties • Multiplication of a vector by an orthogonal matrix preserves Euclidean norm • In general, multiplication by an orthogonal matrix Q preserves inner products, angles, and distances.

4. Rotations • Rotation matrices are one example of orthogonal matrices. In R2 the matrix applied to a vector x rotates x counter-clockwise about the origin. θ Qv v

5. Rotations • In general, the matrix acts as a rotation in the (xi,xj) plane j i

6. Reflections • Let w be a unit vector (column vector). The matrix acts as a reflection across the line (plane) perpendicular to w. Qv v O w

7. Householder Transformations • A Householder transformation(also known as Householder reflectionor elementary reflector) is a linear transformation that describes a reflection about a plane (or hyperplane ) containing the origin. The Householder transformation was introduced in 1958 by Alston Scott Householder. (Wikipedia)

8. Effect of Householder Transformations • Householder transformations can be used to zero out entries of a matrix below the diagonal. • Let A = [A1 A2 … An] be a square matrix, with columns Ai. • Let Let v1 = A1 – α1 e1 where e1 =(1,0,….,0) Then,

9. Effect of Householder Transformations • Now, let Then, w1 is a unit vector. Thus, H1 = I – 2 w1 w1t is a reflection (Householder transf) The first column of H1 A will be H1 A1 .

10. Effect of Householder Transformations • Thus,

11. QR Factorization Theorem: If A is any square matrix of size n, then, there is an orthogonal matrix Q, and an upper triangular matrix R, such that A = QR. Proof: The construction above shows that there is a Householder matrix H1 such that the first column of H1 A has zeros below the first entry. Proceed by induction. Suppose, after step k-1 we have found a series of matrices H1, H2, …, Hk-1 such that Hk-1 …H2 H1 A has all zeroes below the diagonal to position (k-1,k-1).

12. QR Factorization

13. QR Factorization Proof:(continued) . We will zero out the column below the (k,k) entry as follows: Let Then, Let Note: Thus, where Ik is a kxk identity matrix.

14. QR Factorization Proof:(continued) We see that Hk will only affect the lower (n-k)x(n-k) portion of the current matrix. Thus, as before, Hk(Hk-1 …H2 H1 A ) will zero out the column below the (k,k) entry. Thus, we have by induction, that we can find a sequence fo Householder transformations such that (Hn-1 …H2 H1 A) = R, where R is upper triangular. Then, A = H1-1H2-1… Hn-1-1 R. Since Hi-1 =Hi , we have A = H1 H2 … Hn-1 R. Let Q = H1 H2 … Hn-1 . Since Q is a product of orthogonal matrics, it is also orthogonal. QED.

15. QR Example To derive H1 for the given matrix A : We have : Thus, So,

16. Example con’t Then, k=2:

17. Example con’t Then, So, A = H1 H2 R = Q R where

18. Example con’t From Matlab: >> A=[4 3 0; 2 1 2; 4 7 0]; >> [q r] = qr(A) q = 0.6667 -0.5270 -0.5270 0.3333 -0.4216 0.8433 0.6667 0.7379 0.1054 r = 6.0000 7.0000 0.6667 0 3.1623 -0.8433 0 0 1.6865 Note the slight difference between matlab’s result and our algorithm result. Matlab always uses αk values so that the diagonal of R has non-negative values.

19. QR Factorization Note: The factorization of A=QR is not unique!