Loading in 5 sec....

Review of Linear AlgebraPowerPoint Presentation

Review of Linear Algebra

- 116 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Review of Linear Algebra' - pearl-jensen

**An Image/Link below is provided (as is) to download presentation**
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

Presentation Transcript

Today’s Topics

Announcements

- TA: Shiyao Wang
- Office hours: 3:30-5:00 pm Tu/Th
- Office Location: 201C
- Homework-1 is available on ICON

Today’s Topics

- Vectorand Matrix
- Operation on Matrices/Vectors
- Singular value decomposition
- Norms
- An Application in Text Analysis

Vector

- Scalar
- a real number: 7

- Vector
- one dimensional array
- representation: column vector
- representation: row vector

Vector

- Dimensionality or size:
- number of scalars

- Vector Space
- all vectors of the same dimension

Matrix

- Two dimensional array
- Representation
- (i,j)-th element:
- A set of vectors
- vector: a special matrix

rows

columns

Matrix

- Dimensionality or size
- m*n (m rows and n columns)

- Matrix Space:

Today’s Topics

- Vectorand Matrix
- Operation on Matrices/Vectors
- Singular value decomposition
- Norms
- An Application in Text Analysis

Operations

- Matrix addition:
- two matrices of the same size
- (i,j)-th element:

- Scalar multiplication:
- results in the same size

- Matrix subtraction:

Operations

- Multiplication of a row vector and a column vector
- Matrix Multiplication
- ,

Operations

- Transpose:
- (i,j)-the element:
- transpose of a column vector: row vector

- Rules:

Special Matrices

- Square matrix:
- Symmetric matrix:
- Zero matrix
- all elements are zeros

- Identity Matrix:
- each column (or row) standard basis
- :

Operations

- (Square) Matrix Inverse
- similar to inverse of a scalar:
- inverse of a square matrix:
- if there exists:

Non-singular

Operations

- Trace of a square matrix:
- definition
- rules

Today’s Topics

- Vectorand Matrix
- Operation on Matrices/Vectors
- Singular value decomposition
- Norms
- An Application in Text Analysis

mm

mm

mn

mn

V is nn

V is nn

Singular Value Decomposition- A matrix:
- Singular Value Decomposition (SVD)
- The columns of are left singular vectors
- The columns of are right singular vectors
- is a diagonal matrix with singular values (positive values)

Singular Value Decomposition

- Rank of a Matrix
- organize singular values in descending order
- the largest index that is non-zero

Eigen-value Decomposition

- Eigenvectors(for a square mm matrix S)
- Example

(right) eigenvector

eigenvalue

Eigen-value Decomposition

S = U * * UT

Eigen-value Decomposition

S = U * * UT

Eigen-value Decomposition

- This is generally true for symmetric square matrix
- Columns of U are eigenvectors of S
- Diagonal elements of are eigenvalues of S

S = U * * UT

mm

mn

V is nn

nn

nn

nn

Eigen-value Decomposition- A symmetric matrix:
- Eigen-value Decomposition
- The columns of are eigen-vectors
- is a diagonal matrix with real eigen-values

mm

mn

V is nn

nn

nn

nn

Positive (Semi-)Definite Matrix- A symmetric matrix:
- Eigen-value Decomposition
- The columns of are eigen-vectors
- is a diagonal matrix with Positive eigen-values
- is a diagonal matrix with Non-negative eigen-values

- Vectorand Matrix
- Operation on Matrices/Vectors
- Singular value decomposition
- Norms
- An Application in Text Analysis

Inner Product

- inner product between two vectors
- Norm of a Vector: (Euclidean Norm, norm)

Inequalities

- Cauchy-Schwarz Inequality
- Triangle Inequality

p-Norm of a Vector

- p-norm
- p = 1 norm
- p = 2 norm
- p = norm

Norm of a Matrix

- Inner Product between two matrices
- Norm of a Matrix (Frobenius norm)

Other Matrix Norms

- Induced Norm (operator norm):
- p=2, spectral norm: maximum singular value
- p=1, maximum absolute column sum
- p= , maximum absolute row sum

Other Matrix Norms

- Schatten Norm:
- p=1, trace norm (or nuclear norm)
- p=2, Frobenius norm
- p= , Spectral norm

Today’s Topics

- Vectorand Matrix
- Operation on Matrices/Vectors
- Singular value decomposition
- Norms
- An Application in Search Engine

Search Engine

- A database of Webpages
- A user-typed query
- generate a list of relevant webpages
- A ranking problem

https://www.facebook.com/

contain query words (LSI)

a lot of links to them (PageRank)

Representation of documents

- webpage is a document
- document contains many terms (words)
- To represent a document
- collect all meaningful terms
- count the occurrence of each term in a document

Representation of documents

- Term-Document Matrix

Search Engine

- Represent the query in the same way
- e.g. query: “computer system”

Query

0

0

1

0

1

0

0

0

0

0

0

0

Search Engine

- Retrieve Similar Documents
- Query
- Similarity
- inner product
- normalized inner product (cosine similarity)
- Assume A is column normalized and q is normalized

Concept

Rep. of Concepts in term space

Rep. of concepts in document space

Search Engine- Latent Semantic Indexing
- SVD

Search Engine

- Low rank approximation:
- approximate matrix with the largest singular values and singular vectors

Rank-k approximation

Search Engine

- Why Low rank approximation:
- data compression: billions to thousands
- filter out noise

Rank-k approximation

LSI: Example: m=3

Top three left singular vectors

-0.2214 -0.1132 0.2890

-0.1976 -0.0721 0.1350

-0.2405 0.0432 -0.1644

-0.4036 0.0571 -0.3378

-0.6445 -0.1673 0.3611

-0.2650 0.1072 -0.4260

-0.2650 0.1072 -0.4260

-0.3008 -0.1413 0.3303

-0.2059 0.2736 -0.1776

-0.0127 0.4902 0.2311

-0.0361 0.6228 0.2231

-0.0318 0.4505 0.1411

Search Engine

- Why Low rank approximation:
- data compression: billions to thousands
- filter out noise

Download Presentation

Connecting to Server..