distributed nuclear norm minimization for matrix completion n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Distributed Nuclear Norm Minimization for Matrix Completion PowerPoint Presentation
Download Presentation
Distributed Nuclear Norm Minimization for Matrix Completion

Loading in 2 Seconds...

play fullscreen
1 / 12

Distributed Nuclear Norm Minimization for Matrix Completion - PowerPoint PPT Presentation


  • 170 Views
  • Uploaded on

Distributed Nuclear Norm Minimization for Matrix Completion. Morteza Mardani, Gonzalo Mateos and Georgios Giannakis ECE Department, University of Minnesota Acknowledgments : MURI (AFOSR FA9550-10-1-0567) grant. Cesme, Turkey June 19, 2012. 1. Learning from “Big Data”.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Distributed Nuclear Norm Minimization for Matrix Completion


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
distributed nuclear norm minimization for matrix completion

Distributed Nuclear Norm Minimization for Matrix Completion

Morteza Mardani, Gonzalo Mateos and Georgios Giannakis

ECE Department, University of Minnesota

Acknowledgments: MURI (AFOSR FA9550-10-1-0567) grant

Cesme, Turkey

June 19, 2012

1

learning from big data
Learning from “Big Data”

`Data are widely available, what is scarce is the ability to extract wisdom from them’

Hal Varian, Google’s chief economist

Fast

BIG

Productive

Ubiquitous

Revealing

Messy

Smart

2

K. Cukier, ``Harnessing the data deluge,'' Nov. 2011.

context
Context

Goal: Given few incomplete rows per agent, impute missing entries in a distributed fashion by leveraging low-rank of the data matrix.

Preference modeling

  • Imputation of network data

Smart metering

Network cartography

3

3

low rank matrix completion
Low-rank matrix completion

Noise-free

s.t.

  • Consider matrix , set
  • Sampling operator

?

?

?

?

  • Given incomplete (noisy) data

?

?

?

?

?

?

(as) has low rank

  • Goal: denoise observed entries, impute missing ones

?

?

  • Nuclear-norm minimization [Fazel’02],[Candes-Recht’09]

Noisy

4

problem statement
Problem statement
  • Network: undirected, connected graph

?

?

?

?

n

?

?

?

?

?

?

Goal: Given per node and single-hop exchanges, find

(P1)

  • Challenges
  • Nuclear norm is not separable
  • Global optimization variable

5

separable regularization
Separable regularization
  • Key result [Recht et al’11]

Lxρ

≥rank[X]

  • New formulation equivalent to (P1)

(P2)

  • Nonconvex; reduces complexity:

Proposition 1.If stationary pt. of (P2) and ,

then is a global optimum of (P1).

6

distributed estimator
Distributed estimator

(P3)

Consensus with

neighboring nodes

  • Network connectivity (P2)(P3)
  • Alternating-directions method of multipliers (ADMM) solver
    • Method [Glowinski-Marrocco’75], [Gabay-Mercier’76]
    • Learning over networks [Schizas et al’07]
  • Primal variables per agent :

n

  • Message passing:

7

slide9

Attractive features

  • Highly parallelizable with simple recursions
    • Unconstrained QPs per agent
    • No SVD per iteration
  • Low overhead for message exchanges
    • is and is small
    • Comm. cost independent of network size

Recap:

(P1)(P2)(P3)

Consensus

Nonconvex

Sep. regul.

Nonconvex

Centralized

Convex

Stationary (P3) Stationary (P2) Global (P1)

9

optimality
Optimality
  • Proposition 2.If converges to
  • and , then:
  • i)
  • ii) is the global optimum of (P1).
  • ADMM can converge even for non-convex problems [Boyd et al’11]
  • Simple distributed algorithm for optimal matrix imputation
    • Centralized performance guarantees e.g., [Candes-Recht’09] carry over

10

synthetic data
Synthetic data
  • Random network topology
    • N=20, L=66, T=66
  • Data
    • ,
    • ,

11

real data
Real data
  • Network distance prediction [Liau et al’12]
  • Abilene network data (Aug 18-22,2011)
    • End-to-end latency matrix
    • N=9, L=T=N
    • 80%missing data

Relative error: 10%

12

Data: http://internet2.edu/observatory/archive/data-collections.html