EE645: Independent Component Analysis

438 Views

Download Presentation
## EE645: Independent Component Analysis

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**1. **EE645: Independent Component Analysis Elliot Taniguchi
Advisor: Dr. Kuh
May 16, 2003

**3. **Motivation for ICA Cocktail Party Problem
Suppose you are in a crowded room with many people. How do you understand what any one person is saying?
Separation of Independent Signals
Similar to Blind Source Separation
Little knowledge of the signals
Access to mixed signals only

**4. **Cocktail Party Problem

**5. **Cocktail Party Problem ICA Separation Algorithm
Separation of Speech Signals
Humans can separate multiple signals with only two ears/sensors
ICA needs as many ears/sensors as message signals
Here we assume he has four ears!

**6. **Recovered Messages

**7. **Mathematical Formulation Overview ICA Definition
ICA Assumptions
Independent Signals
Non-Gaussian
ICA Limitations
Scaling
Permutations
No. of Sensors

**8. **ICA Definition Mixed Signals in Matrix Notation

**9. **ICA Solution Signal Separation
Find using the ICA Algorithm

**10. **ICA Block Diagram (2 Signals)

**11. **ICA Assumption #1: Independence Probability Density Definition Expected Value Definition

**12. **ICA Assumption #2: Non-Gaussian Property of Gaussian signals
Addition of two independent Gaussian random variables is another single Gaussian random variable.
Information Lost!
Kurtosis Function
Special Case: kurt(N) = 0

**13. **Limitation #1: Scaling ICA maximizes independence between signals.

**14. **Limitation #2: Signal Permutations The mixing matrix and independent components are unknown.

**15. **Limitation #3: No. of Sensors Sensor Requirement
The number of separated signals cannot be larger than the number of inputs.
Current research is being done to reduce this constraint.

**16. **ICA Separation Technique Central Limit Theorem
If two random (non-Gaussian) signals added, the resulting signal will be more Gaussian than the original two random signals
ICA Separation Concept
Central Limit Theorem (in Reverse)
Maximizing Non-Gaussianity
Results in separating the two signals

**17. **Fast ICA Algorithm Overview Fixed-Point Algorithm
Implementation
Fast ICA algorithm
Extensions
Algorithm Speed & Performance
Currently the fastest
Most Commonly Used

**18. **Fast ICA Algorithm Choose a random initial weight vector.
Let,
Let,
Repeat until converges.

**19. **Fast ICA Extensions Preprocessing
Normalize mean to zero
Pre-Whitening
Activation Functions
g(u)=u^3
g(u)=u^2
g(u)=tanh(a1*u)
g(u)=u*exp(-a2*u^2/2)

**20. **Noise Separation Example Separation of Noise
Impulsive Noise
Additive White Gaussian Noise
Implementation
Two Sensor Setup
Fast ICA Algorithm

**21. **Noise Generation AWGN
Gaussian R.V.
Impulsive Noise
Poisson R.V.
Gaussian R.V.

**22. **Physical Setup

**23. **Noise Separation Example

**24. **Noise Separation Example

**25. **Noise Separation Example

**26. **Noise Separation Performance

**27. **Noise Separation Performance

**28. **Noise Separation & Feature Extraction ICA performs well in Blind Source Separation
ICA for Feature Extraction
Reduce Complexity of the Neural Network
Train only on the appropriate signal
Detection and Estimation of Hidden Signals

**29. **Noise Detection and Estimation: Method #1

**30. **Noise Detection and Estimation: Method #2

**31. **Noise Detection and Estimation:Conclusion Additional Preprocessing
Segmentation of Impulsive Noise (Time-Limited)
Possible Inputs to the Neural Network
Statistical Moments
Signal Samples
Possible Neural Networks
Back Propagation
SVM
Radial Basis Functions
Problem
Need to train Neural Network in Matlab

**32. **Digital Watermarking of Music Motivation
Popularity of Digital Storage Devices
Reliable, Fast, Ease of duplication, etc.
How to protect copyrighted information?
Leaving digital signatures of its artist
Essential Properties for Watermarking
Undetectable
Irremovable
Resilient

**33. **Detection & Estimation of Watermarks Detection of Watermark
Authenticate copyrighted music
Estimation of Watermark
Authenticate copyrighted music
Information on artist, producer, etc.

**34. **Watermarking Model Process
Mix the original musical data with watermark
Keep watermark Power relatively low
Ensure high quality of the watermarked music
Watermark is better hidden

**35. **Popular Digital Formats

**36. **Detection of Watermark Watermarking Detection Algorithm
Use the ICA model to randomly mix the watermark and music file.
Save the watermarked music in the popular *.wav format
Read the saved *.wave file. Separate the watermark and the music file.
Identify the watermark using statistical methods (mean, std, etc.)
Performance Statistic
Correlation Coefficient (Absolute Value)

**37. **Detection Performance

**38. **Estimation of Watermark Watermarking Estimation Algorithm
Use the ICA model to randomly mix the watermark and music file.
Save the watermarked music in the popular *.wav format
Read the saved *.wave file. Separate the watermark and the music file.
Identify the watermark using statistical methods (mean, std, etc.)
Digitize the watermark signal.
Performance Statistic
Bit Error Rate

**39. **Estimation Performance

**40. **Is the Music Content Preserved?

**41. **Resilience of Proposed Watermark Resiliency
Previous Simulations show that wav format is resilient to 8-bit and 16-bit quantization.
Can the watermark be detected after Mp3 compression and decompression?

**42. **Mp3 Compression/Decompression Actual Mp3 compression program used
CDex Version 1.40 Release
Mp3 (lossy) Compression
Down Sampling
Filter banks
Mp3 Decompression
Up Sampling
Reconstruction Filter

**43. **Detection and Estimation Performance Bad News
Correlation Coefficients 0
MSE 0.5
Possible Problems with Mp3 Compression
Down Sampling
Watermark information is lost
Quantization Noise
Watermark information absorbs into the quantization noise
Lossy Compression
11:1 Compression Rate

**44. **How to Improve its Resilience? Alternative Approaches
Synchronization of the music data
Time shift in Mp3 compression?
Storing watermark in certain frequencies (where less quantization occurs)
Error Coding
Hamming
Reed-Solomon

**45. **Conclusion Wave to Wave Format
Very good performance (even for 8-bit wave files)
SNR is very low.
Music Integrity is excellent
Mp3 Compression
Very bad performance
Alternative methods need to be found!
Need a greater understanding of current Mp3 Compression Algorithms

**46. **References [1] Araki and others. Suband Based Blind Source Separation with Appropriate Processing for Each Frequency Band. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003.
[2] Hoyer and Hyvarinen. Independent Component Analysis Applied to Feature Extraction from Colour and Stereo Images. August 2000.
[3] Hyvarinen, Aapo. The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis. http://www.cis.hut.fi/~aapo/.
[4] Hyvarinen, Aapo. Fast and Robust Fixed-Point Algorithms for Independent Component Analysis. http://www.cis.hut.fi/~aapo/. April 1999.
[5] Hyvarinen and Oja. Independent Component Analysis: A Tutorial. http://www.cis.hut.fi/projects/ica/. April 1999.
[6] Introduction to Blind Source Separation. http://www.cnl.salk.edu/~tewon/Blind/.
[7] Liu and others. A Digital Watermarking Scheme based on ICA Detection. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003.
[8] Mitra, Sanjit K. Digital Signal Processing: Second Addition. McGraw Hill, 1998.
[9] Shen and others. A Method for Digital Image Watermarking Using ICA. 4th International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). April 2003.