Coupled matrix factorizations using optimization
Download
1 / 17

Coupled Matrix Factorizations using Optimization - PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on

Coupled Matrix Factorizations using Optimization. Daniel M. Dunlavy , Tamara G. Kolda, Evrim Acar Sandia National Laboratories SIAM Conference on Computational Science and Engineering March 4, 2009.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Coupled Matrix Factorizations using Optimization' - xia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Coupled matrix factorizations using optimization

Coupled Matrix Factorizations using Optimization

Daniel M. Dunlavy, Tamara G. Kolda, Evrim Acar

Sandia National Laboratories

SIAM Conference on Computational Science and Engineering

March 4, 2009

Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

SAND2009-2389C

1/17


Motivating problems
Motivating Problems

  • Data with multiple types of two-way relationships

    • Bibliometric analysis

      • author-document, term-document, author-venue, etc.

      • Can we predict potential co-authors?

    • Movie ratings

      • movie-actor, user-movie, actor-award

      • Can we predict useful movie ratings for other users?

  • Consistent dimensionality reduction

  • Improved interpretation through non-negativity constraints

2/17


Some related work
Some Related Work

matrices of same size

  • Simultaneous factor analysis

    • Gramian matrices [Levin, 1966]

    • Test score covariance matrices over time [Millsap, et al., 1988]

  • Simultaneous diagonalization

    • Population differentiation in biology [Thorpe, 1988]

    • Blind source separation [Ziehe et al., 2004]

  • Generalized SVD

  • Damped or constrained least squares [Van Loan, 1976]

    • Microarray data analysis [Alter, et al., 2003]

    • Multimicrophone speech filtering [Doclo and Moonen, 2002]

  • Simultaneous Non-negative Matrix Factorization

    • Gene clustering in microarray data [Badea, 2007; 2008]

  • Tensor decompositions

    • Data mining, chemometrics, neuroscience[Kolda, Acar, Bro, Park, Zhang, Berry, Chen, Martin, CSE09]

matrices of same size

only 2 matrices

slow

at least one common dimension

3/17


Coupled non negative matrix factorization cnmf
Coupled Non-negative Matrix Factorization (CNMF)

  • Given

  • Solve

document-term

document-author

4/17


Method cnmf als
Method: CNMF-ALS

  • CNMF-ALS: Alternating Least Squares [Extends Berry, et al., 2006]

linear least squares

+ simple projection to constraint boundary

5/17


Method cnmf mult
Method: CNMF-MULT

  • CNMF-MULT: Multiplicative Updates [Badea, 2007; Badea, 2008; extends Lee and Seung, 2001]

6/17


Method cnmf opt
Method: CNMF-OPT

  • CNMF-OPT: Projective Nonlinear CG, More-Thuente LS[Extends Acar, Kolda, and Dunlavy, 2009 and Lin, 2007]

7/17


Matlab experiments
Matlab Experiments

Noise:

8/17








Future work
Future Work

  • Extending other promising methods to CNMF

    • Block principal pivoting based NMF [Park, et al. 2008]

    • Projected gradient NMF [Lin, 2007]

    • Projected Newton NMF [Kim, et al., 2008]

  • CNMF-OPT extensions

    • Sparse data, regularization [Acar, Kolda, and Dunlavy, 2009]

    • Sparsity constraints [Park, et al. 2008]

  • Numerical experiments

    • Scale to larger data sets

    • Comparisons on real data sets [Park, et al. 2008]

  • Alternate models / problem formulations

    • Coupling matrix and tensor decompositions (CNMF/CNTF)

15/17


Conclusions
Conclusions

  • Coupled matrix factorizations

    • Method for computing factorizations consistent along common dimensions in data

  • Results

    • CNMF-OPT

      • Fast and accurate

        • Overfactors well and handles noise well

    • CNMF-ALS

      • Fast, but not accurate

        • Overfactoring is a big challenge

    • CNMF-MULT

      • Accurate, but may be too slow (similar to NMF results)

  • Future Work

    • Identified several promising paths forward

16/17


Thank you
Thank You

Coupled Matrix Factorizations using Optimization

Danny Dunlavy

[email protected]

http://www.cs.sandia.gov/~dmdunla

17/17


ad