1 / 14

Introduction To Linear Discriminant Analysis. å³è‚²å¾· é™½æ˜Žå¤§å­¸æ”¾å°„é†«å­¸ç§‘å­¸ç ”ç©¶æ‰€ å°åŒ—æ¦®ç¸½æ•´åˆæ€§è…¦åŠŸèƒ½ç ”ç©¶å®¤. Linear Discriminant Analysis.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Introduction To Linear Discriminant Analysis

Linear Discriminant Analysis

For a given training sample set, determine a set of optimal projection axes such that the set of projective feature vectors of the training samples has the maximumbetween-class scatter and minimum within-class scatter simultaneously.

Linear Discriminant Analysis

Linear Discriminant Analysis seeks a projection that best separate the data .

Sb : between-class scatter matrix

Sw : within-class scatter matrix

LDA

Fisher discriminant analysis

Sol:

LDA

Fisher discriminant analysis

where

, = k1+k2

and let

LDA

Fisher discriminant analysis

LDA

Generalized eigenvalue problem.....Theorem 2

Let M be a real symmetric matrix with largest eigenvalue

then

and the maximum occurs when , i.e. the unit eigenvector associated with .

Proof :

LDA

Generalized eigenvalue problem.....proof of Theorem 2

LDA

Generalized eigenvalue problem.....proof of Theorem 2

Cor:

If M is a real symmetric matrix with largest eigenvalue .

And the maximum is achieved whenever ,where is the unit eigenvector associated with .

LDA

Generalized eigenvalue problem…….. Theorem 1

Let Sw and Sb be n*n real symmetric matrices . If Sw is positive definite, then there exists an n*n matrix V which achieves

The real numbers λ1….λn satisfy the generalized eiegenvalue equation :

: generalized eigenvector

: generalized eigenvalue

LDA

Generalized eigenvalue problem.....proof of Theorem 1

Let and be the unit eigenvectors and

eigenvalues of Sw, i.e

Now define then

where

Since ri ﹥0 (Sw is positive definite) , exist

LDA

Generalized eigenvalue problem.....proof of Theorem 1

LDA

Generalized eigenvalue problem.....proof of Theorem 1

We need to claim :

(applying a unitary matrix to a whitening process doesn’t affect it!)

(VT)-1 exists since det(VTSwV) = det (I )

→ det(VT) det(Sw) det(V) = det(I)

Because det(VT)= det(V)

→ [det(VT)]2 det(Sw) = 1 > 0

→ det(VT) 0

LDA

Generalized eigenvalue problem.....proof of Theorem 1

Procedure for diagonalizing Sw (real symmetric and positive definite) and Sb (real symmetric) simultaneously is as follows :

1. Find λi by solving

And then find normalized , i=1,2…..,n

2. normalized