320 likes | 520 Views
Matrix Factorizations: Singular Value Decomposition. Presented by Nik Clark MTH 421. Introduction. In the exciting world of numerical analysis, one may wonder “Why? Why do I study matrices and their factorizations?”
E N D
Matrix Factorizations:Singular Value Decomposition Presented by Nik Clark MTH 421
Introduction • In the exciting world of numerical analysis, one may wonder “Why? Why do I study matrices and their factorizations?” • Such a simple answer, “Because matrices and their decompositions can take you anywhere!”
Choleski Jordan LU Polar Proper Orthogonal QR Schur Singular Value Spectral (Eigendecomposition) Matrix Decompositions Some interesting ways to decompose a matrix
Singular Value Decomposition We’d like to more formally introduce you to Singular Value Decomposition (SVD) and some of its applications.
What is SVD? • SVD is a type of factorization for a rectangular, real or complex, matrix. • THEOREM: All matrices Amxn have a singular value decomposition.
Why should we care? • All math is awesome. • Also because SVD can be applied to several situations: • Data Compression • Analyzing DNA Gene Expression Data • Solving Least Squares Problems • Information Retrieval
More Caring • Image Processing • De-blurring • Seismology • Digital Signal Processing • Noise Reduction • Data Hiding • Cryptography • Watermarks • Researching Databases
Back to the SVD • Each mxn matrix A decomposes into the product of three matrices A=UΣVT • U is orthogonal and mxm. Its columns span col(A). • V is also orthogonal, but is nxn. Its columns span row(A) • is a diagonal matrix where the singular values of A are along the main diagonal, and all other values are zero.
More on SVD • The m columns of matrix U are the eigenvectors of AAT • The n columns of matrix V are the eigenvectors of ATA • The entries of the main diagonal of matrix are the singular values of A, denoted by i
Another Way to Write • An additional way to write A, apart from A=UΣVT : A = 1u1vT1 +2u2vT2 + …+rurvTr , Where r = rank(A), and is defined to be the number of linearly independent columns of A.
Importance • Each term of the expansion is already in order of importance. • 1≥ 2 ≥ … ≥ k ≥ 0
Rank k Approximation • Ak=UkΣkVTk • Where Ak is the first k terms of the SVD factorization of A • i.e. A = 1u1vT1 +2u2vT2 + …+kukvTk
More on Rank • A = 1u1vT1 +2u2vT2 + …+kukvTk • Where each term of the expansion is a rank 1 matrix.
An Example: Code breaking • Cipher- a method for encrypting a message. • Cryptography – The art of creating a coded message using a cipher. • Cryptanalysis – The art of breaking a code by finding its weakness. • Cryptology – the study of the aforementioned definitions.
How do we decode the code? • Most commonly, a cryptogram is created by substituting one letter for another. • When decoding a code (in english) it is easiest to first decode the vowels. • Vowel pairs are less frequent than consonant-vowel pairs. • More frequently, vowels follow consonants, vfc.
vcf • When vowels follow consonants, there is a mathematical proportion: number of vowel pairs number of vowels number consonant-vowel pairs number of consonants <
The matrix • We define matrix A to be a digram frequency of the letter combinations of our text. • aij is the number of times the ith letter is followed by the jth letter. • We define vector wi to equal 1 when the letter is a vowel and 0 otherwise. • Vector ci is defined to equal 1 when the letter is a consonant and 0 otherwise.
Thus • Our vectors w and c are orthogonal. • wTAw is the number of vowel pairs. • cTAw is the number of consonant vowel pairs. • We can now rewrite our equation: wTAw wTA(w+c) cTAw cTA(w+c) <
Singular Vectors • A ~ A1 = 1u1vT1 • f = k1* u1 • f = k2* v1 • Where f represents the frequency of the words in the text.
Rank Two • A ~ A2 = 1u1vT1 +2u2vT2 • Rank two is the frequency of vowel pairs. • Each vowel (except for u) corresponds to a (+, -) vector pair. • Each consonant corresponds to a (-, +) vector pair. • There are some exceptions
Neuter Letters • Those letters that correspond to a (+, +) vector pair or a (-, -) vector pair are called neuter. • These letter patterns correspond to any text we consider.
An Encrypted Example • The following matrix is the digram matrix of an encrypted text (cryptogram) similar to the sentence below. • What does this mean?? • Gsviv rh ml zkkorw nzgsvnzgrxk ru gsviv rh ml nzgsvnzgrxh gl zkkob.