Weighted Low-Rank Approximation Nathan Srebro and Tommi Jaakkola ICML 2003

1. Weighted Low-Rank ApproximationNathan Srebro and Tommi JaakkolaICML 2003 Presented by: Mingyuan Zhou Duke University, ECE February 18, 2011

2. Outline • Introduction • Low rank matrix factorization • Missing values and an EM procedure • Low rank logistic regression • Experimental results • Conclusions

3. Introduction • Factor model • Weighted norms • Efficient optimization methods

4. Low rank matrix factorization • Objective function • Solutions ( = 1)

5. Low rank matrix factorization • Solutions

6. Low rank matrix factorization • Since are unlikely to be diagonalizable for all rows, The critical points of the weighted low-rank approximation problem lack the eigenvector structure of the unweighted case. • Another implication of this is that the incremental structure of unweighted low-rank approximations is lost: an optimal rank-k factorization cannot necessarily be extended to an optimal rank-(k + 1) factorization.

7. Low rank matrix factorization

8. Missing values and an EM procedure • Initializing X with A or 0 • Initializing X with 0 and let

9. Missing values and an EM procedure

10. Low rank logistic regression

12. Experimental results

13. Experimental results

14. Conclusions