Matrix factorizations singular value decomposition
This presentation is the property of its rightful owner.
Sponsored Links
1 / 30

Matrix Factorizations: Singular Value Decomposition PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on
  • Presentation posted in: General

Matrix Factorizations: Singular Value Decomposition. Presented by Nik Clark MTH 421. Introduction. In the exciting world of numerical analysis, one may wonder “Why? Why do I study matrices and their factorizations?”

Download Presentation

Matrix Factorizations: Singular Value Decomposition

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Matrix factorizations singular value decomposition

Matrix Factorizations:Singular Value Decomposition

Presented by Nik Clark

MTH 421


Introduction

Introduction

  • In the exciting world of numerical analysis, one may wonder “Why? Why do I study matrices and their factorizations?”

  • Such a simple answer, “Because matrices and their decompositions can take you anywhere!”


Matrix decompositions

Choleski

Jordan

LU

Polar

Proper Orthogonal

QR

Schur

Singular Value

Spectral (Eigendecomposition)

Matrix Decompositions

Some interesting ways to decompose a matrix


Singular value decomposition

Singular Value Decomposition

We’d like to more formally introduce you to Singular Value Decomposition (SVD) and some of its applications.


What is svd

What is SVD?

  • SVD is a type of factorization for a rectangular, real or complex, matrix.

  • THEOREM:

    All matrices Amxn have a singular value decomposition.


Why should we care

Why should we care?

  • All math is awesome.

  • Also because SVD can be applied to several situations:

    • Data Compression

    • Analyzing DNA Gene Expression Data

    • Solving Least Squares Problems

    • Information Retrieval


More caring

More Caring

  • Image Processing

    • De-blurring

  • Seismology

  • Digital Signal Processing

    • Noise Reduction

  • Data Hiding

    • Cryptography

    • Watermarks

  • Researching Databases


Back to the svd

Back to the SVD

  • Each mxn matrix A decomposes into the product of three matrices A=UΣVT

  • U is orthogonal and mxm. Its columns span col(A).

  • V is also orthogonal, but is nxn. Its columns span row(A)

  •  is a diagonal matrix where the singular values of A are along the main diagonal, and all other values are zero.


More on svd

More on SVD

  • The m columns of matrix U are the eigenvectors of AAT

  • The n columns of matrix V are the eigenvectors of ATA

  • The entries of the main diagonal of matrix  are the singular values of A, denoted by i


Another way to write

Another Way to Write

  • An additional way to write A, apart from A=UΣVT :

    A = 1u1vT1 +2u2vT2 + …+rurvTr ,

    Where r = rank(A), and is defined to be the number of linearly independent columns of A.


Importance

Importance

  • Each term of the expansion is already in order of importance.

  • 1≥ 2 ≥ … ≥ k ≥ 0


Rank k approximation

Rank k Approximation

  • Ak=UkΣkVTk

  • Where Ak is the first k terms of the SVD factorization of A

  • i.e. A = 1u1vT1 +2u2vT2 + …+kukvTk


More on rank

More on Rank

  • A = 1u1vT1 +2u2vT2 + …+kukvTk

  • Where each term of the expansion is a rank 1 matrix.


An example code breaking

An Example: Code breaking

  • Cipher- a method for encrypting a message.

  • Cryptography – The art of creating a coded message using a cipher.

  • Cryptanalysis – The art of breaking a code by finding its weakness.

  • Cryptology – the study of the aforementioned definitions.


How do we decode the code

How do we decode the code?

  • Most commonly, a cryptogram is created by substituting one letter for another.

  • When decoding a code (in english) it is easiest to first decode the vowels.

  • Vowel pairs are less frequent than consonant-vowel pairs.

  • More frequently, vowels follow consonants, vfc.


Matrix factorizations singular value decomposition

vcf

  • When vowels follow consonants, there is a mathematical proportion:

number of vowel pairs

number of vowels

number consonant-vowel pairs

number of consonants

<


The matrix

The matrix

  • We define matrix A to be a digram frequency of the letter combinations of our text.

  • aij is the number of times the ith letter is followed by the jth letter.

  • We define vector wi to equal 1 when the letter is a vowel and 0 otherwise.

  • Vector ci is defined to equal 1 when the letter is a consonant and 0 otherwise.


Digram matrix

Digram Matrix


Matrix factorizations singular value decomposition

Thus

  • Our vectors w and c are orthogonal.

  • wTAw is the number of vowel pairs.

  • cTAw is the number of consonant vowel pairs.

  • We can now rewrite our equation:

wTAw

wTA(w+c)

cTAw

cTA(w+c)

<


Singular vectors

Singular Vectors

  • A ~ A1 = 1u1vT1

  • f = k1* u1

  • f = k2* v1

  • Where f represents the frequency of the words in the text.


Digram matrix1

Digram Matrix


Rank two

Rank Two

  • A ~ A2 = 1u1vT1 +2u2vT2

  • Rank two is the frequency of vowel pairs.

  • Each vowel (except for u) corresponds to a (+, -) vector pair.

  • Each consonant corresponds to a (-, +) vector pair.

  • There are some exceptions


Neuter letters

Neuter Letters

  • Those letters that correspond to a (+, +) vector pair or a (-, -) vector pair are called neuter.

  • These letter patterns correspond to any text we consider.


An encrypted example

An Encrypted Example

  • The following matrix is the digram matrix of an encrypted text (cryptogram) similar to the sentence below.

  • What does this mean??

  • Gsviv rh ml zkkorw nzgsvnzgrxk ru gsviv rh ml nzgsvnzgrxh gl zkkob.


  • Login