review of linear algebra
Download
Skip this Video
Download Presentation
Review of Linear Algebra

Loading in 2 Seconds...

play fullscreen
1 / 50

Review of Linear Algebra - PowerPoint PPT Presentation


  • 116 Views
  • Uploaded on

Review of Linear Algebra. Fall 2014 The University of Iowa Tianbao Yang. Announcements. TA: Shiyao Wang Office hours: 3:30- 5:00 pm Tu / Th Office Location: 201C Homework-1 is available on ICON. Today’s Topics. Vector and Matrix Operation on Matrices/Vectors

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Review of Linear Algebra' - pearl-jensen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
review of linear algebra

Review of Linear Algebra

Fall 2014

The University of Iowa

Tianbao Yang

announcements
Announcements
  • TA: Shiyao Wang
  • Office hours: 3:30-5:00 pm Tu/Th
  • Office Location: 201C
  • Homework-1 is available on ICON
today s topics
Today’s Topics
  • Vectorand Matrix
  • Operation on Matrices/Vectors
  • Singular value decomposition
  • Norms
  • An Application in Text Analysis
vector
Vector
  • Scalar
    • a real number: 7
  • Vector
    • one dimensional array
    • representation: column vector
    • representation: row vector
vector1
Vector
  • Dimensionality or size:
    • number of scalars
  • Vector Space
    • all vectors of the same dimension
matrix
Matrix
  • Two dimensional array
  • Representation
  • (i,j)-th element:
  • A set of vectors
  • vector: a special matrix

rows

columns

matrix1
Matrix
  • Dimensionality or size
    • m*n (m rows and n columns)
  • Matrix Space:
today s topics1
Today’s Topics
  • Vectorand Matrix
  • Operation on Matrices/Vectors
  • Singular value decomposition
  • Norms
  • An Application in Text Analysis
operations
Operations
  • Matrix addition:
    • two matrices of the same size
    • (i,j)-th element:
  • Scalar multiplication:
    • results in the same size
  • Matrix subtraction:
operations1
Operations
  • Multiplication of a row vector and a column vector
  • Matrix Multiplication
    • ,
operations3
Operations
  • Transpose:
    • (i,j)-the element:
    • transpose of a column vector: row vector
  • Rules:
special matrices
Special Matrices
  • Square matrix:
  • Symmetric matrix:
  • Zero matrix
    • all elements are zeros
  • Identity Matrix:
    • each column (or row) standard basis
    • :
operations4
Operations
  • (Square) Matrix Inverse
    • similar to inverse of a scalar:
    • inverse of a square matrix:
    • if there exists:

Non-singular

operations5
Operations
  • Trace of a square matrix:
    • definition
    • rules
today s topics2
Today’s Topics
  • Vectorand Matrix
  • Operation on Matrices/Vectors
  • Singular value decomposition
  • Norms
  • An Application in Text Analysis
singular value decomposition

mm

mm

mn

mn

V is nn

V is nn

Singular Value Decomposition
  • A matrix:
  • Singular Value Decomposition (SVD)
  • The columns of are left singular vectors
  • The columns of are right singular vectors
  • is a diagonal matrix with singular values (positive values)
singular value decomposition1

mm

mn

V is nn

Singular Value Decomposition
  • Illustration of SVD dimensions and sparseness
singular value decomposition2
Singular Value Decomposition
  • Rank of a Matrix
    • organize singular values in descending order
    • the largest index that is non-zero
eigen value decomposition
Eigen-value Decomposition
  • Eigenvectors(for a square mm matrix S)
  • Example

(right) eigenvector

eigenvalue

eigen value decomposition4
Eigen-value Decomposition
  • This is generally true for symmetric square matrix
  • Columns of U are eigenvectors of S
  • Diagonal elements of  are eigenvalues of S

S = U *  * UT

eigen value decomposition5

mm

mn

V is nn

nn

nn

nn

Eigen-value Decomposition
  • A symmetric matrix:
  • Eigen-value Decomposition
  • The columns of are eigen-vectors
  • is a diagonal matrix with real eigen-values
positive semi definite matrix

mm

mn

V is nn

nn

nn

nn

Positive (Semi-)Definite Matrix
  • A symmetric matrix:
  • Eigen-value Decomposition
  • The columns of are eigen-vectors
  • is a diagonal matrix with Positive eigen-values
  • is a diagonal matrix with Non-negative eigen-values
today s topics3
Today’s Topics
  • Vectorand Matrix
  • Operation on Matrices/Vectors
  • Singular value decomposition
  • Norms
  • An Application in Text Analysis
inner product
Inner Product
  • inner product between two vectors
  • Norm of a Vector: (Euclidean Norm, norm)
inequalities
Inequalities
  • Cauchy-Schwarz Inequality
  • Triangle Inequality
p norm of a vector
p-Norm of a Vector
  • p-norm
    • p = 1 norm
    • p = 2 norm
    • p =  norm
norm of a matrix
Norm of a Matrix
  • Inner Product between two matrices
  • Norm of a Matrix (Frobenius norm)
other matrix norms
Other Matrix Norms
  • Induced Norm (operator norm):
    • p=2, spectral norm: maximum singular value
    • p=1, maximum absolute column sum
    • p= , maximum absolute row sum
other matrix norms1
Other Matrix Norms
  • Schatten Norm:
    • p=1, trace norm (or nuclear norm)
    • p=2, Frobenius norm
    • p= , Spectral norm
machine learning problems
Machine Learning Problems
  • Solve the following problems

Loss

norm

today s topics4
Today’s Topics
  • Vectorand Matrix
  • Operation on Matrices/Vectors
  • Singular value decomposition
  • Norms
  • An Application in Search Engine
search engine
Search Engine
  • A database of Webpages
  • A user-typed query
  • generate a list of relevant webpages
  • A ranking problem

https://www.facebook.com/

contain query words (LSI)

a lot of links to them (PageRank)

representation of documents
Representation of documents
  • webpage is a document
  • document contains many terms (words)
  • To represent a document
    • collect all meaningful terms
    • count the occurrence of each term in a document
representation of documents1
Representation of documents
  • Term-Document Matrix
search engine1
Search Engine
  • Represent the query in the same way
    • e.g. query: “computer system”

Query

0

0

1

0

1

0

0

0

0

0

0

0

search engine2
Search Engine
  • Retrieve Similar Documents
  • Query
  • Similarity
    • inner product
    • normalized inner product (cosine similarity)
    • Assume A is column normalized and q is normalized
search engine3

Concept

Concept

Rep. of Concepts in term space

Rep. of concepts in document space

Search Engine
  • Latent Semantic Indexing
    • SVD
search engine4
Search Engine
  • Low rank approximation:
    • approximate matrix with the largest singular values and singular vectors

Rank-k approximation

search engine5
Search Engine
  • Why Low rank approximation:
    • data compression: billions to thousands
    • filter out noise

Rank-k approximation

lsi example m 3
LSI: Example: m=3

Top three left singular vectors

-0.2214 -0.1132 0.2890

-0.1976 -0.0721 0.1350

-0.2405 0.0432 -0.1644

-0.4036 0.0571 -0.3378

-0.6445 -0.1673 0.3611

-0.2650 0.1072 -0.4260

-0.2650 0.1072 -0.4260

-0.3008 -0.1413 0.3303

-0.2059 0.2736 -0.1776

-0.0127 0.4902 0.2311

-0.0361 0.6228 0.2231

-0.0318 0.4505 0.1411

search engine6
Search Engine
  • Why Low rank approximation:
    • data compression: billions to thousands
    • filter out noise
ad