An Introduction of Independent Component Analysis (ICA)

Download Presentation

An Introduction of Independent Component Analysis (ICA)

Loading in 2 Seconds...

- 99 Views
- Uploaded on
- Presentation posted in: General

An Introduction of Independent Component Analysis (ICA)

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

An Introduction of Independent Component Analysis (ICA)

Xiaoling Wang

Jan. 28, 2003

- Application: blind source separation (BSS) and deconvolution
- Motivation: “cocktail party problem”
- Assumption: two people speaking simultaneously, two microphones in different locations

- Assumption: sources are statistically independent
- Goal: it seeks a transformation to coordinates in which the data are maximally statistically independent
- Definition:

Mixing process

Demixing process

– mixing matrix, – separation matrix

Nonlinear mixing

Non-stationary

mixing

Linear mixing

Non-stationary

sources

Non-Gaussian sources

Gaussian sources

No noise

Independent

Factor analysis

Classical ICA

Factor Analysis

R diagonal

Approximations to

mutual information

Cumulant based

methods

Flexible

Source model

Switching

source model

Probabilistic

PCA

Fixed

source model

Kurtosis

minimization

No noise

FastICA

Infomax

PCA

orthogonal mixing

- Independence: the pdf of sources can be factorized
- Nongaussian is independent
- Seek the separation matrix W which maximize the nongaussianity of the estimated sources

- Kurtosis (4th order cumulant):
- Subgaussian: negative kurtosis
- Supergaussian: positive kurtosis

- Negentropy:

entropy

differential

entropy

negentropy

- Mutual information:

For ,

- Basic form:
- Choose an initial (e.g. Random) weight vector
- Let
- Let
- If not converged, go back to step 2

- For several units: decorrelation
- Let
- Let

- Model:
- Existence and uniqueness of solutions
- There always exists an infinity of solutions if the space of the nonlinear mixing functions is not limited
- Post-nonlinear problem

mixing

demixing

- Burel’s approach: neural solution, known nonlinearities on unknown parameters
- Krob & Benidir: high order moments, polynomial mixtures
- Pajunen et al.: SOMs, locally factorable pdf
- Pajunen et al.: GTM(generative topographic mapping), output distribution matches the known source distributions
- Post nonlinear mixtures:
- Taleb & Jutten: adaptive componentwise separation
- Yang et al.: two-layer neural network
- Puntonet et al.: nonlinearities are a power function, geometrical considerations