Structured Sparse Principal Component Analysis

1 / 28

# Structured Sparse Principal Component Analysis - PowerPoint PPT Presentation

Structured Sparse Principal Component Analysis. Authors: Rodolphe Jenatton , Guillaume Obozinski , Francis Bach. Reading Group Presenter: Peng Zhang Cognitive Radio Institute Friday, October 01, 2010. Outline. Introduction (in Imaging Sense) Principal Component Analysis (PCA)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Structured Sparse Principal Component Analysis

Authors: RodolpheJenatton, Guillaume Obozinski, Francis Bach

Peng Zhang

Friday, October 01, 2010

Outline
• Introduction (in Imaging Sense)
• Principal Component Analysis (PCA)
• Sparse PCA (SPCA)
• Structured Sparse PCA (SSPCA)
• Problem Statement
• The SSPCA Algorithm
• Experiments
• Conclusion and Other Thoughts
Introduction (Imaging Sense)
• The face recognition problem
• A database includes a huge amount of faces
• How to let computer to recognize different faces with database
• The challenge
• Huge amount of data
• Computation complexity
• The trick
• Represent the face using a weighted “face dictionary”
• Similar to code book in data compression
• Example: An 200 X 200 pixel face can be represented by 100 coefficients using the “face dictionary”
• The solution
• Principal component analysis (PCA)
PCA
• PCA
• A compression method
• Given a large amount of sample vectors {x}
• 2nd moment statistics of the sample vectors
• Eigen-decomposition finds the “dictionary” and “energy” of the dictionary codes
• Eigen-vectors {v} form the “dictionary”
• Eigen-values {d} give the “energy” of “dictionary” elements
PCA
• Original signal can be represented using only part of the dictionary
• Data is compressed with fewer elements
• Meaning of “dictionary” v:
• It is the weights of each elements in x
• The problem for PCA for face recognition: No physical meaning for “dictionary”
PCA

The Face Samples

The “dictionary”, eigen-faces

PCA

These eigen-faces can reconstruct original faces perfectly, but make no sense in real life

Face recognition

Structured SPCA

Non-sparse Eigen-faces from PCA

Sparse Eigen-faces from SPCA

But the eigen-faces are still meaningless most of time

• The SPCA goal:
• Make dictionary more interpretable
• The “sparse” solution: Limit the number of nonzeros
Structured SPCA

Eigen-faces from SSPCA

• The new idea, SSPCA
• Eigen-faces will be meaningful when some structured constraints are set
• Meaningful areas in faces are constrained in “grids”
Structured SPCA
• This paper’s contribution
• Add the “structure” constraint to make the dictionary more meaningful
• How the constraint works
• Meaningful dictionary is more close to “true” dictionary
• Meaningful dictionary is more robust against noise
• Meaningful dictionary is more accurate in face recognition
Outline
• Introduction
• Principal Component Analysis (PCA)
• Sparse PCA (SPCA)
• Structured Sparse PCA (SSPCA)
• Problem Statement
• The SSPCA Algorithm
• Experiments
• Conclusion and Other Thoughts
Problem Statement
• From SPCA to SSPCA
• The optimization problem
• X is sample matrix, U is coefficient matrix, V is dictionary
• ||.|| and are different types of norms
• The trick in SPCA
• L1 norm force the dictionary to be a sparse solution
Problem Statement

Structured SPCA, however, deal with a mixed l1/l2 minimization:

Right now it’s hard for me to understand the G and d

Problem Statement
• In short, the norm constraints have the following effects
• Dictionary has some structures
• All non-zeros in the dictionary will be confined inside a grid
Outline
• Introduction
• Principal Component Analysis (PCA)
• Sparse PCA (SPCA)
• Structured Sparse PCA (SSPCA)
• Problem Statement
• The SSPCA Algorithm
• Experiments
• Conclusion and Other Thoughts
The SSPCA Algorithm
• Making the dictionary sparser
• The norm,
• The new SSPCA problem:
The SSPCA Algorithm

Methods to solve a sequence of convex problems

Excerpt from Author’s slide

Excerpt from author’s slide:

Outline
• Introduction
• Principal Component Analysis (PCA)
• Sparse PCA (SPCA)
• Structured Sparse PCA (SSPCA)
• Problem Statement
• The SSPCA Algorithm
• Experiments
• Conclusion and Other Thoughts
Conclusion and Other Thoughts
• Conclusion
• This paper shows how to use SSPCA
• SSPCA gets better performance in denoising, face recognition and classification
• Other thoughts
• Usually, the meaningful dictionary in communication signals is Fourier dictionary
• But Fourier dictionary may not fit some transient signals or time-variant signals
• How to manipulate the G, d and norms to set constraints for our needs?