online learning for matrix factorization and sparse coding n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Online Learning for Matrix Factorization and Sparse Coding PowerPoint Presentation
Download Presentation
Online Learning for Matrix Factorization and Sparse Coding

Loading in 2 Seconds...

play fullscreen
1 / 15

Online Learning for Matrix Factorization and Sparse Coding - PowerPoint PPT Presentation


  • 263 Views
  • Uploaded on

Online Learning for Matrix Factorization and Sparse Coding. Julien Mairal, Francis Bach, Jean Ponce and Guillermo Sapiro Journal of Machine Learning Research 2010. Introduction. This paper focuses on the large scale matrix factorization problem, including Dictionary learning for sparse coding

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Online Learning for Matrix Factorization and Sparse Coding' - jennessa-cherry


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
online learning for matrix factorization and sparse coding

Online Learning for Matrix Factorization and Sparse Coding

Julien Mairal, Francis Bach, Jean Ponce and Guillermo Sapiro

Journal of Machine Learning Research 2010

introduction
Introduction
  • This paper focuses on the large scale matrix factorization problem, including
    • Dictionary learning for sparse coding
    • Non-negative matrix factorization (NMF)
    • Sparse principal component analysis (SPCA)
  • Contributions of this paper:
    • An iterative online algorithm is proposed for large scale matrix factorization
    • This algorithm is proved to converge almost surely to a stationary point of the objective function
    • This algorithm is shown to be much faster than previous methods in the experiment.
problem statement
Problem Statement
  • Classical dictionary learning problem

Given a finite training set , the objective is to optimize the following function

where

  • Online Learning

This algorithm process one sample (or a mini-batch) at a time and sequentially minimize the following function:

optimizing the algorithm
Optimizing the Algorithm
  • Handling fixed-sized data sets
  • Scaling the “past” data
  • Mini-batch extension
proof of convergence
Proof of Convergence
  • Assumptions:
  • Main results
extensions to matrix factorization
Extensions to Matrix Factorization
  • Non-negative matrix factorization (NMF)
  • Non-negative sparse coding (NNSC)
  • Sparse principal component analysis (SPCA)
data for experiment
Data for Experiment
  • 1.25 million patches from Pascal VOC’06 image database
online vs batch
Online VS. Batch
  • Training data size: 1 million
  • OL1:
  • OL2:
  • OL3:
inpainting results
Inpainting Results
  • Image size: 12-Megapixel
  • Dictionary with 256 elements
  • Training data: 7 million 12 by 12 color patches
conclusion
Conclusion
  • A new online algorithm for learning dictionaries adapted to sparse coding tasks, and proven its convergence.
  • Experiments demonstrate that this algorithm is significantly faster than existing batch methods.
  • This algorithm can be extended to other matrix factorization problems such as non-negative matrix factorization and sparse principal component analysis.