Lecture 17 Introduction to Eigenvalue Problems

1. Lecture 17Introduction to Eigenvalue Problems Shang-Hua Teng

2. Eigenvalue Problems • Eigenvalue problems occur in many areas of science and engineering • E.g., Structure analysis • It is important for analyzing numerical and linear algebra algorithms • Impact of roundoff errors and precision requirement • It is widely used in information management and web-search • It is the key ingredient for the analysis of Markov process, sampling algorithms, and various approximation algorithms in computer science

3. Eigenvalues and Eigenvectors • Standard Eigenvalue Problem: Given an n by n matrix A, find a scalar l and nonzero vector x such that A x = lx • l is eigenvalue, and x is corresponding eigenvector

4. Spectrum of Matrices • Spectrum(A) = l(A) = set of all eigenvalues of A • Spectral radius (A) = r (A) = max {|l|: l in l(A)} • Spectral analysis • Spectral methods

5. Geometric Interpretation • Matrix expands or shrinks any vector lying in direction of eigenvector by scalar factor • Expansion of contraction factor given by corresponding eigenvalue l • Eigenvalues and eigenvectors decompose complicated behavior of general linear transformation into simpler actions

6. Examples: Eigenvalues and Eigenvectors Note: x1 and x2are perpendicular to each other

7. Examples: Eigenvalues and Eigenvectors Note: x1 and x2are not perpendicular to each other

8. Examples: Eigenvalues and Eigenvectors Note: x1 and x2are perpendicular to each other

9. Examples: Eigenvalues and Eigenvectors Note: x1 and x2are perpendicular to each other

10. Examples: Eigenvalues and Eigenvectors Note: x1 and x2are not perpendicular to each other : eigenvalues or eigenvectors may not be real!!!

11. Simple Facts of Eigenvalue Problem • If (l,x) is a eigenvalue-eigenvector pair of A, then for any k, (lk, x) is a eigenvalue-eigenvector pair of Ak. • If (l,x) is a eigenvalue-eigenvector pair of A, then for any c, (cl, x) is a eigenvalue-eigenvector pair of cA.

12. Algebraic Interpretation:Equation for the Eigenvalues A x = lx (A - lI ) x = 0 • The eigenvectors make up the nullspace of (A – lI ) if we know l.

13. Eigenvalue First • If (A - lI ) x = 0 hasa nonzero solution, then • A - lI is not invertible • The determinant of A - lI mustbe zero.

14. Characteristic Equation for Eigenvalues • The number l is an eigenvalue of A if and only if (A - lI ) is singular: det( A - lI ) = 0

15. Characteristic Polynomial for Eigenvalues • det (A - lI ) = 0 is a polynomial in l of degree at most n. • The spectrum of A is the set of roots of this characteristic polynomial: • Fundamental Theorem of Algebra implies that n by n matrix A always has n eigenvalues, but they need be neither distinct nor real

16. Examples: Characteristic Polynomial So Spectrum(A) = {1,2}

17. Examples: Characteristic Polynomial So Spectrum(A) = {2, 4}

18. Examples: Characteristic Polynomial So Spectrum(A) = {i, -i}

19. A Possible Methods for Solving the Eigenvalue Problems • Compute the characteristic polynomial of A in l by expanding det(A – lI) = 0 • Find the roots of the characteristic polynomial • For each eigenvalue l, solve (A – lI) x=0 to find an eigenvector x.

20. Practical Difficulties • Computing eigenvalues using characteristic polynimial is not recommended or used because • Roots of polynomial of degree > 4 cannot always be computed in finite number of steps • A lot of work is needed in computing coefficients of the characteristic polynomial • Computer has round-off errors

21. Examples: Characteristic Polynomial So Spectrum(A) = {1+e, 1-e} But in machine, e2 < emachine is equal to 0 So, the algorithm will return Spectrum(A) = {1,1}

22. Theory and Practice • Characteristic polynomial is a powerful theoretical tool but usually is not useful computationally.

23. Special Matrix • What is Spectrum( I )? • Multiplicity is the number of times root appears when polynomial written as product of linear factors • det(I – l I ) = (1-l)n • What is Spectrum( upper or lower triangular matrix )?

24. Bad News • Elimination does not preserve the l’s.

25. Diagonalizing A Matrix • Suppose the n by n matrix A has n linearly independent eigenvectors x1, x2,…, xn. • Eigenvector matrix S: x1, x2,…, xn are columns of S. • Then L is the eigenvalue matrix

26. Matrix Power Ak • S-1AS = L implies A = S LS-1 • implies A2 = S LS-1 S LS-1 = S L2S-1 • implies Ak = S LkS-1

27. Random walks How long does it take to get completely lost?

28. 1 2 6 3 4 5 Random walks Transition Matrix

29. Matrix Powers • If A is diagonalizable as A = S LS-1 then for any vector u, we can compute Akuefficiently • Solve S c = u • Aku =S LkS-1 S c = S Lkc • As if A is a diagonal matrix!!!!

30. Independent Eigenvectors from Different Eigenvalues • Eigenvectors x1, x2,…, xkthat correspond to distinct (all different) eigenvalues are linear independent. • An n by n matrix that has n different eigenvalues (no repeated l’s) must be diagonalizable Proof: Show that implies all ci = 0