Matrix factorizations singular value decomposition
1 / 30

Matrix Factorizations: Singular Value Decomposition - PowerPoint PPT Presentation

  • Uploaded on

Matrix Factorizations: Singular Value Decomposition. Presented by Nik Clark MTH 421. Introduction. In the exciting world of numerical analysis, one may wonder “Why? Why do I study matrices and their factorizations?”

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Matrix Factorizations: Singular Value Decomposition' - mindy

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Matrix factorizations singular value decomposition

Matrix Factorizations:Singular Value Decomposition

Presented by Nik Clark

MTH 421


  • In the exciting world of numerical analysis, one may wonder “Why? Why do I study matrices and their factorizations?”

  • Such a simple answer, “Because matrices and their decompositions can take you anywhere!”

Matrix decompositions





Proper Orthogonal



Singular Value

Spectral (Eigendecomposition)

Matrix Decompositions

Some interesting ways to decompose a matrix

Singular value decomposition
Singular Value Decomposition

We’d like to more formally introduce you to Singular Value Decomposition (SVD) and some of its applications.

What is svd
What is SVD?

  • SVD is a type of factorization for a rectangular, real or complex, matrix.


    All matrices Amxn have a singular value decomposition.

Why should we care
Why should we care?

  • All math is awesome.

  • Also because SVD can be applied to several situations:

    • Data Compression

    • Analyzing DNA Gene Expression Data

    • Solving Least Squares Problems

    • Information Retrieval

More caring
More Caring

  • Image Processing

    • De-blurring

  • Seismology

  • Digital Signal Processing

    • Noise Reduction

  • Data Hiding

    • Cryptography

    • Watermarks

  • Researching Databases

Back to the svd
Back to the SVD

  • Each mxn matrix A decomposes into the product of three matrices A=UΣVT

  • U is orthogonal and mxm. Its columns span col(A).

  • V is also orthogonal, but is nxn. Its columns span row(A)

  •  is a diagonal matrix where the singular values of A are along the main diagonal, and all other values are zero.

More on svd
More on SVD

  • The m columns of matrix U are the eigenvectors of AAT

  • The n columns of matrix V are the eigenvectors of ATA

  • The entries of the main diagonal of matrix  are the singular values of A, denoted by i

Another way to write
Another Way to Write

  • An additional way to write A, apart from A=UΣVT :

    A = 1u1vT1 +2u2vT2 + …+rurvTr ,

    Where r = rank(A), and is defined to be the number of linearly independent columns of A.


  • Each term of the expansion is already in order of importance.

  • 1≥ 2 ≥ … ≥ k ≥ 0

Rank k approximation
Rank k Approximation

  • Ak=UkΣkVTk

  • Where Ak is the first k terms of the SVD factorization of A

  • i.e. A = 1u1vT1 +2u2vT2 + …+kukvTk

More on rank
More on Rank

  • A = 1u1vT1 +2u2vT2 + …+kukvTk

  • Where each term of the expansion is a rank 1 matrix.

An example code breaking
An Example: Code breaking

  • Cipher- a method for encrypting a message.

  • Cryptography – The art of creating a coded message using a cipher.

  • Cryptanalysis – The art of breaking a code by finding its weakness.

  • Cryptology – the study of the aforementioned definitions.

How do we decode the code
How do we decode the code?

  • Most commonly, a cryptogram is created by substituting one letter for another.

  • When decoding a code (in english) it is easiest to first decode the vowels.

  • Vowel pairs are less frequent than consonant-vowel pairs.

  • More frequently, vowels follow consonants, vfc.

Matrix factorizations singular value decomposition

  • When vowels follow consonants, there is a mathematical proportion:

number of vowel pairs

number of vowels

number consonant-vowel pairs

number of consonants


The matrix
The matrix

  • We define matrix A to be a digram frequency of the letter combinations of our text.

  • aij is the number of times the ith letter is followed by the jth letter.

  • We define vector wi to equal 1 when the letter is a vowel and 0 otherwise.

  • Vector ci is defined to equal 1 when the letter is a consonant and 0 otherwise.

Matrix factorizations singular value decomposition

  • Our vectors w and c are orthogonal.

  • wTAw is the number of vowel pairs.

  • cTAw is the number of consonant vowel pairs.

  • We can now rewrite our equation:






Singular vectors
Singular Vectors

  • A ~ A1 = 1u1vT1

  • f = k1* u1

  • f = k2* v1

  • Where f represents the frequency of the words in the text.

Rank two
Rank Two

  • A ~ A2 = 1u1vT1 +2u2vT2

  • Rank two is the frequency of vowel pairs.

  • Each vowel (except for u) corresponds to a (+, -) vector pair.

  • Each consonant corresponds to a (-, +) vector pair.

  • There are some exceptions

Neuter letters
Neuter Letters

  • Those letters that correspond to a (+, +) vector pair or a (-, -) vector pair are called neuter.

  • These letter patterns correspond to any text we consider.

An encrypted example
An Encrypted Example

  • The following matrix is the digram matrix of an encrypted text (cryptogram) similar to the sentence below.

  • What does this mean??

  • Gsviv rh ml zkkorw nzgsvnzgrxk ru gsviv rh ml nzgsvnzgrxh gl zkkob.