structured sparse principal component analysis l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Structured Sparse Principal Component Analysis PowerPoint Presentation
Download Presentation
Structured Sparse Principal Component Analysis

Loading in 2 Seconds...

play fullscreen
1 / 28

Structured Sparse Principal Component Analysis - PowerPoint PPT Presentation


  • 174 Views
  • Uploaded on

Structured Sparse Principal Component Analysis. Authors: Rodolphe Jenatton , Guillaume Obozinski , Francis Bach. Reading Group Presenter: Peng Zhang Cognitive Radio Institute Friday, October 01, 2010. Outline. Introduction (in Imaging Sense) Principal Component Analysis (PCA)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Structured Sparse Principal Component Analysis' - gamada


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
structured sparse principal component analysis

Structured Sparse Principal Component Analysis

Authors: RodolpheJenatton, Guillaume Obozinski, Francis Bach

Reading Group Presenter:

Peng Zhang

Cognitive Radio Institute

Friday, October 01, 2010

outline
Outline
  • Introduction (in Imaging Sense)
    • Principal Component Analysis (PCA)
    • Sparse PCA (SPCA)
    • Structured Sparse PCA (SSPCA)
  • Problem Statement
  • The SSPCA Algorithm
  • Experiments
  • Conclusion and Other Thoughts
introduction imaging sense
Introduction (Imaging Sense)
  • The face recognition problem
    • A database includes a huge amount of faces
    • How to let computer to recognize different faces with database
  • The challenge
    • Huge amount of data
    • Computation complexity
  • The trick
    • Represent the face using a weighted “face dictionary”
      • Similar to code book in data compression
      • Example: An 200 X 200 pixel face can be represented by 100 coefficients using the “face dictionary”
  • The solution
    • Principal component analysis (PCA)
slide4
PCA
  • PCA
    • A compression method
    • Given a large amount of sample vectors {x}
    • 2nd moment statistics of the sample vectors
    • Eigen-decomposition finds the “dictionary” and “energy” of the dictionary codes
      • Eigen-vectors {v} form the “dictionary”
      • Eigen-values {d} give the “energy” of “dictionary” elements
slide5
PCA
  • Original signal can be represented using only part of the dictionary
      • Data is compressed with fewer elements
  • Meaning of “dictionary” v:
    • It is the weights of each elements in x
  • The problem for PCA for face recognition: No physical meaning for “dictionary”
slide6
PCA

The Face Samples

The “dictionary”, eigen-faces

PCA

These eigen-faces can reconstruct original faces perfectly, but make no sense in real life

Face recognition

structured spca
Structured SPCA

Non-sparse Eigen-faces from PCA

Sparse Eigen-faces from SPCA

But the eigen-faces are still meaningless most of time

  • The SPCA goal:
    • Make dictionary more interpretable
    • The “sparse” solution: Limit the number of nonzeros
structured spca8
Structured SPCA

Eigen-faces from SSPCA

  • The new idea, SSPCA
    • Eigen-faces will be meaningful when some structured constraints are set
    • Meaningful areas in faces are constrained in “grids”
structured spca9
Structured SPCA
  • This paper’s contribution
    • Add the “structure” constraint to make the dictionary more meaningful
    • How the constraint works
    • Meaningful dictionary is more close to “true” dictionary
    • Meaningful dictionary is more robust against noise
    • Meaningful dictionary is more accurate in face recognition
outline10
Outline
  • Introduction
    • Principal Component Analysis (PCA)
    • Sparse PCA (SPCA)
    • Structured Sparse PCA (SSPCA)
  • Problem Statement
  • The SSPCA Algorithm
  • Experiments
  • Conclusion and Other Thoughts
problem statement
Problem Statement
  • From SPCA to SSPCA
    • The optimization problem
    • X is sample matrix, U is coefficient matrix, V is dictionary
    • ||.|| and are different types of norms
    • The trick in SPCA
      • L1 norm force the dictionary to be a sparse solution
problem statement12
Problem Statement

Structured SPCA, however, deal with a mixed l1/l2 minimization:

Right now it’s hard for me to understand the G and d

problem statement13
Problem Statement
  • In short, the norm constraints have the following effects
    • Dictionary has some structures
    • All non-zeros in the dictionary will be confined inside a grid
outline14
Outline
  • Introduction
    • Principal Component Analysis (PCA)
    • Sparse PCA (SPCA)
    • Structured Sparse PCA (SSPCA)
  • Problem Statement
  • The SSPCA Algorithm
  • Experiments
  • Conclusion and Other Thoughts
the sspca algorithm
The SSPCA Algorithm
  • Making the dictionary sparser
    • The norm,
    • The new SSPCA problem:
the sspca algorithm16
The SSPCA Algorithm

Methods to solve a sequence of convex problems

excerpt from author s slide
Excerpt from Author’s slide

Excerpt from author’s slide:

outline26
Outline
  • Introduction
    • Principal Component Analysis (PCA)
    • Sparse PCA (SPCA)
    • Structured Sparse PCA (SSPCA)
  • Problem Statement
  • The SSPCA Algorithm
  • Experiments
  • Conclusion and Other Thoughts
conclusion and other thoughts
Conclusion and Other Thoughts
  • Conclusion
    • This paper shows how to use SSPCA
    • SSPCA gets better performance in denoising, face recognition and classification
  • Other thoughts
    • Usually, the meaningful dictionary in communication signals is Fourier dictionary
    • But Fourier dictionary may not fit some transient signals or time-variant signals
    • How to manipulate the G, d and norms to set constraints for our needs?