1 / 10

# An Introduction of Independent Component Analysis (ICA) - PowerPoint PPT Presentation

An Introduction of Independent Component Analysis (ICA). Xiaoling Wang Jan. 28, 2003. What Is ICA?. Application: blind source separation (BSS) and deconvolution Motivation: “cocktail party problem” Assumption: two people speaking simultaneously, two microphones in different locations.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' An Introduction of Independent Component Analysis (ICA)' - alika

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### An Introduction of Independent Component Analysis (ICA)

Xiaoling Wang

Jan. 28, 2003

• Application: blind source separation (BSS) and deconvolution

• Motivation: “cocktail party problem”

• Assumption: two people speaking simultaneously, two microphones in different locations

• Assumption: sources are statistically independent

• Goal: it seeks a transformation to coordinates in which the data are maximally statistically independent

• Definition:

Mixing process

Demixing process

– mixing matrix, – separation matrix

Nonlinear mixing

Non-stationary

mixing

Linear mixing

Non-stationary

sources

Non-Gaussian sources

Gaussian sources

No noise

Independent

Factor analysis

Classical ICA

Factor Analysis

R diagonal

Approximations to

mutual information

Cumulant based

methods

Flexible

Source model

Switching

source model

Probabilistic

PCA

Fixed

source model

Kurtosis

minimization

No noise

FastICA

Infomax

PCA

orthogonal mixing

• Independence: the pdf of sources can be factorized

• Nongaussian is independent

• Seek the separation matrix W which maximize the nongaussianity of the estimated sources

• Kurtosis (4th order cumulant):

• Subgaussian: negative kurtosis

• Supergaussian: positive kurtosis

• Negentropy:

entropy

differential

entropy

negentropy

• Mutual information:

For ,

• Basic form:

• Choose an initial (e.g. Random) weight vector

• Let

• Let

• If not converged, go back to step 2

• For several units: decorrelation

• Let

• Let

• Model:

• Existence and uniqueness of solutions

• There always exists an infinity of solutions if the space of the nonlinear mixing functions is not limited

• Post-nonlinear problem

mixing

demixing

• Burel’s approach: neural solution, known nonlinearities on unknown parameters

• Krob & Benidir: high order moments, polynomial mixtures

• Pajunen et al.: SOMs, locally factorable pdf

• Pajunen et al.: GTM(generative topographic mapping), output distribution matches the known source distributions

• Post nonlinear mixtures:

• Taleb & Jutten: adaptive componentwise separation

• Yang et al.: two-layer neural network

• Puntonet et al.: nonlinearities are a power function, geometrical considerations