Math 2999 presentation

1. Math 2999 presentation Singular Value Decomposition and its applications Yip Ka Wa (2009137234)

2. 1. Mathematics foundation

3. Singular Value Decomposition • Suppose that we are given a matrix, it is not known that whether it has any special properties like diagonal or normal, but we can always decompose it into a product of 3 matrices. This is called the singular value decomposition:

4. The statement:

5. The picture:

8. General idea of construction (proof omitted)

9. Positive semidefinite matrix

10. Singular Value

11. Spectral Decomposition of Hermitian matrices

13. Recall:

16. Compact form

17. Example

18. 2. Image compression

19. Image compression

20. Outer product expansion

24. Truncation of terms • To ensure that the memory used is reduced, however, we can even do a bit more. We can delete some of the nonzero singular values and keep only the first k largest singular values out of the r nonzero singular values. r terms in the outer product expansion are reduced to k terms.

27. Memory reduction

28. Block-based SVD • Divide the image matrix into blocks and apply SVD compression to blocks.

30. Result without block-based

34. Special topic Investigation of the kind of images for whichlow rank SVD approximations can give very good results

35. Quality of image: Error ratio:

36. Complex? • Simple? • How do I know whether I can delete some of the singular values and still preserve quality?

37. Some suggestions • 1. The difference (Euclidean norm) between two columns/rows is small compared to the Frobenius norm of the matrix. • 2. The difference (Euclidean norm) between columns/rows with zero columns/rows is small compared to the Frobenius norm of the matrix. • 3. The columns/rows are near to another columns/row in term of the cosine angle. The cosine angle is near to 1.

45. An example illustrating the Eckart-Young theorem:

46. Best rank 2 approximation

47. Obvious rank 2 matrix:

48. Frobenius norm difference

49. Upper bound of singular values

50. Sometimes we can easily spot a matrix of which the rank is 1 lower A, and observe that the Frobenius norm of the difference matrix between them is small, for example, a column is near to a multiple of another column. As that Frobenius norm is an upper bound for the last nonzero singular value of A, we know the last nonzero singular value of A is very small. Therefore it can be truncated to do SVD approximation without much loss of accuracy.