Non-negative Matrix Factorization - PowerPoint PPT Presentation

sheng
non negative matrix factorization n.
Skip this Video
Loading SlideShow in 5 Seconds..
Non-negative Matrix Factorization PowerPoint Presentation
Download Presentation
Non-negative Matrix Factorization

play fullscreen
1 / 15
Download Presentation
230 Views
Download Presentation

Non-negative Matrix Factorization

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Non-negative Matrix Factorization Recent algorithms, extensions and available software Atina Dunlap Brooks (adbrook2@stat.ncsu.edu) North Carolina State University

  2. Recent Algorithms • Lee & Seung’s multiplicative updates are easy to understand and to implement • Can be very slow to converge • ALS can speed things up • Convergence theory is not particularly strong • Most NMF methods do not have robust convergence, but work well in practice

  3. Projected GradientDescent Method • Chih-Jen Lin (2007) • Bound-constrained optimization • Projected Gradient

  4. Projected GradientDescent Method • Can be applied to both the multiplicative updates and the ALS solution • Generally, greatest speed was achieved with the projected gradient combined with ALS

  5. Fast Non-NegativeMatrix Approximation • Kim, Sra & Dhillon (2007) • Employs Newton-type methods to solve NMF • Uses curvature information vs. gradient descent approach • Provide an exact method (good accuracy, but still slow) and a very fast inexact method


  6. References forAlgorithm Comparisons • Algorithms and Applications for Approximate Nonnegative Matrix Factorization by Berry, Browne, Langville, Pauca & Plemmons (2006) • Optimality, Computation, and Interpretations of Nonnegative Matrix Factorizations by Chu, Diele, Plemmons & Ragni (2004)

  7. Extensions • Tri-Factorization • Semi-NMF • Convex-NMF • Non-negative Tensor Factorization • Inferential Robust Matrix Factorization

  8. OrthogonalTri-factorization • Ding, Li, Peng & Park (2006) • Requiring orthogonality introduces uniqueness and improves clustering interpretations A = WSH, where WTW=I and HTH=I • W gives row clusters while H gives column clusters

  9. Semi-NMF • Ding, Li & Jordan (2006) • Allows A and W to contain negative values, but H is restricted to non-negative • Provides more flexibility (negative entries) and a clustering which is usually better than k-means

  10. Convex-NMF • Ding, Li & Jordan (2006) • Restricts W to be convex combinations of the columns of A • Ensures meaningful cluster centroids • W and H tend to be sparse

  11. Non-NegativeTensor Factorization • Uses n-way arrays instead of the 2-dimensional arrays used by NMF • Presentations during the workshop by Michael Berry and Bob Plemmons

  12. Inferential RobustMatrix Factorization • Fogel, Young, Hawkins & Ledirac (2007) • Uses the same method for robustness as Liu et al. (2003) for robust SVD • Paul Fogel will be presenting on an application

  13. Software - Matlab • Matlab Code • Patrik Hoyer • http://www.cs.helsinki.fi/u/phoyer/ • Includes Lee & Seung’s multiplicative updates and Hoyer’s sparseness • Chih-Jen Lin • http://www.csie.ntu.edu.tw/~cjlin/nmf/ • Includes projected gradient descent applied to multiplicative updates and ALS

  14. Software • C code – nnmf() • Simon Sheperd • http://www.simonshepherd.supanet.com/nnmf.htm • Very fast algorithm (as of 2004) • JMP script - irMF • Paul Fogel • http://www.niss.org/irMF/ • Inferential Robust Matrix Factorization

  15. Thank You