230 Views

Download Presentation
## Non-negative Matrix Factorization

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Non-negative Matrix Factorization**Recent algorithms, extensions and available software Atina Dunlap Brooks (adbrook2@stat.ncsu.edu) North Carolina State University**Recent Algorithms**• Lee & Seung’s multiplicative updates are easy to understand and to implement • Can be very slow to converge • ALS can speed things up • Convergence theory is not particularly strong • Most NMF methods do not have robust convergence, but work well in practice**Projected GradientDescent Method**• Chih-Jen Lin (2007) • Bound-constrained optimization • Projected Gradient**Projected GradientDescent Method**• Can be applied to both the multiplicative updates and the ALS solution • Generally, greatest speed was achieved with the projected gradient combined with ALS**Fast Non-NegativeMatrix Approximation**• Kim, Sra & Dhillon (2007) • Employs Newton-type methods to solve NMF • Uses curvature information vs. gradient descent approach • Provide an exact method (good accuracy, but still slow) and a very fast inexact method**References forAlgorithm Comparisons**• Algorithms and Applications for Approximate Nonnegative Matrix Factorization by Berry, Browne, Langville, Pauca & Plemmons (2006) • Optimality, Computation, and Interpretations of Nonnegative Matrix Factorizations by Chu, Diele, Plemmons & Ragni (2004)**Extensions**• Tri-Factorization • Semi-NMF • Convex-NMF • Non-negative Tensor Factorization • Inferential Robust Matrix Factorization**OrthogonalTri-factorization**• Ding, Li, Peng & Park (2006) • Requiring orthogonality introduces uniqueness and improves clustering interpretations A = WSH, where WTW=I and HTH=I • W gives row clusters while H gives column clusters**Semi-NMF**• Ding, Li & Jordan (2006) • Allows A and W to contain negative values, but H is restricted to non-negative • Provides more flexibility (negative entries) and a clustering which is usually better than k-means**Convex-NMF**• Ding, Li & Jordan (2006) • Restricts W to be convex combinations of the columns of A • Ensures meaningful cluster centroids • W and H tend to be sparse**Non-NegativeTensor Factorization**• Uses n-way arrays instead of the 2-dimensional arrays used by NMF • Presentations during the workshop by Michael Berry and Bob Plemmons**Inferential RobustMatrix Factorization**• Fogel, Young, Hawkins & Ledirac (2007) • Uses the same method for robustness as Liu et al. (2003) for robust SVD • Paul Fogel will be presenting on an application**Software - Matlab**• Matlab Code • Patrik Hoyer • http://www.cs.helsinki.fi/u/phoyer/ • Includes Lee & Seung’s multiplicative updates and Hoyer’s sparseness • Chih-Jen Lin • http://www.csie.ntu.edu.tw/~cjlin/nmf/ • Includes projected gradient descent applied to multiplicative updates and ALS**Software**• C code – nnmf() • Simon Sheperd • http://www.simonshepherd.supanet.com/nnmf.htm • Very fast algorithm (as of 2004) • JMP script - irMF • Paul Fogel • http://www.niss.org/irMF/ • Inferential Robust Matrix Factorization