Graph sparsifiers a survey
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

Graph Sparsifiers : A Survey PowerPoint PPT Presentation


  • 42 Views
  • Uploaded on
  • Presentation posted in: General

Graph Sparsifiers : A Survey. Based on work by: Batson, Benczur , de Carli Silva, Fung, Hariharan , Harvey, Karger , Panigrahi , Sato, Spielman , Srivastava and Teng. Nick Harvey. Approximating Dense Objects by Sparse Ones. Floor joists Image compression.

Download Presentation

Graph Sparsifiers : A Survey

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Graph sparsifiers a survey

Graph Sparsifiers: A Survey

Based on work by: Batson, Benczur, de Carli Silva, Fung, Hariharan, Harvey, Karger, Panigrahi, Sato, Spielman, Srivastava and Teng

Nick Harvey


Approximating dense objects by sparse ones

Approximating Dense Objects by Sparse Ones

  • Floor joists

  • Image compression


Approximating dense graphs by sparse ones

Approximating Dense Graphsby Sparse Ones

  • Spanners: Approximate distances to within ®using only = O(n1+2/®) edges

  • Low-stretch trees: Approximate most distancesto within O(log n) using only n-1 edges

(n = # vertices)


Overview

Overview

  • Definitions

    • Cut & Spectral Sparsifiers

  • Cut Sparsifiers

    • A combinatorial construction

  • Spectral Sparsifiers

    • A random sampling construction

    • Derandomization


Cut sparsifiers

(Karger ‘94)

Cut Sparsifiers

  • Input: An undirected graph G=(V,E) with weights u : E !R+

  • Output: A subgraph H=(V,F) of G with weightsw : F!R+ such that |F| is small and

    w(±H(U)) = (1§²) u(±G(U)) 8Uµ V

weight of edges between U and V\U in H

weight of edges between U and V\U in G

U

U


Cut sparsifiers1

Cut Sparsifiers

  • Input: An undirected graph G=(V,E) with weights u : E !R+

  • Output: A subgraph H=(V,F) of G with weightsw : F!R such that |F| is small and

    w(±H(U)) = (1§²) u(±G(U)) 8Uµ V

weight of edges between U and V\U in H

weight of edges between U and V\U in G


Generic application of cut sparsifiers

Generic Applicationof Cut Sparsifiers

(Slow) Algorithm A for some problem P

(Dense) Input graph G

Exact/Approx Output

Min s-t cut, Sparsest cut,Max cut, …

(Efficient) Sparsification Algorithm S

Algorithm A(now faster)

Sparse graph H approx preserving solution of P

Approximate Output


Relation to expander graphs

Relation to Expander Graphs

  • Graph H on V is an expander if, for some constant c,|±H(U)| ¸c¢ |U| 8Uµ V, |U|·n/2

  • Let G be the complete graph on V. Note that|±G(U)| = |U|¢|VnU| · n¢|U|

  • If we give all edges of H weight w=n, thenw(±H(U)) ¸ c¢|±G(U)| 8Uµ V, |U|·n/2

  • Expanders are similar to sparsifiers of complete graph

G

H


Relation to expander graphs1

Relation to Expander Graphs

  • Fact: Pick a random graph where each edge appears independently with probability p=(log(n)/n). Gives an expander with O(n log n) edges with high probability.

G

H


Spectral sparsifiers

(Spielman-Teng ‘04)

Spectral Sparsifiers

  • Input: An undirected graph G=(V,E) withweights u : E !R+

  • Def: The Laplacianis the matrix LG such thatxTLGx = st2Eust (xs-xt)28x2RV.

  • LG is positive semidefinite since this is ¸ 0.

  • Example: Electrical Networks

    • View edge st as resistor of resistance 1/ust.

    • Impose voltage xv at every vertex v.

    • Ohm’s Power Law: P = V2/R.

    • Power consumed on edge st is ust (xs-xt)2.

    • Total power consumed is xTLGx.


Spectral sparsifiers1

(Spielman-Teng ‘04)

Spectral Sparsifiers

  • Input: An undirected graph G=(V,E) withweights u : E !R+

  • Def: The Laplacianis the matrix LG such thatxTLGx = st2Eust (xs-xt)28x2RV.

  • Output: A subgraphH=(V,F) of G with weightsw : F!R such that |F| is small and

    xTLHx = (1 §²) xTLGx8x2RV

    w(±H(U)) = (1 § ²) u(±G(U)) 8Uµ V

SpectralSparsifier

)

)

CutSparsifier


Cut vs spectral sparsifiers

Cut vs Spectral Sparsifiers

  • Number of Constraints:

    • Cut: w(±H(U)) = (1§²) u(±G(U)) 8UµV (2n constraints)

    • Spectral: xTLHx = (1§²) xTLGx8x2RV (1 constraints)

  • Spectral constraints are SDP feasibility constraints:

    (1-²) xTLGx· xTLHx· (1+²) xTLGx8x2RV

    , (1-²) LG¹LH¹ (1+²) LG

  • Spectral constraints are actually easier to handle

    • Checking “Is H is a spectral sparsifier of G?” is in P

    • Checking “Is H is a cut sparsifier of G?” isnon-uniform sparsest cut, so NP-hard

Here X ¹ Y means Y-X is positive semidefinite


Application of spectral sparsifiers

Application of Spectral Sparsifiers

  • Consider the linear system LGx = b.Actual solution is x := LG-1 b.

  • Instead, compute y := LH-1b,where H is a spectral sparsifier of G.

  • We know: (1-²) LG¹LH¹(1+²) LG

    )y has low multiplicative error: ky-xkLG·2²kxkLG

  • Computing y is fast since H is sparse:conjugate gradient method takes O(n|F|) time (where |F| = # nonzero entries of LH)


Application of spectral sparsifiers1

Application of Spectral Sparsifiers

  • Consider the linear system LGx = b.Actual solution is x := LG-1 b.

  • Instead, compute y := LH-1b,where H is a spectral sparsifier of G.

  • We know: (1-²) LG¹LH¹(1+²) LG

    )y has low multiplicative error: ky-xkLG·2²kxkLG

  • Theorem:[Spielman-Teng ‘04, Koutis-Miller-Peng ‘10]Can compute avector y with low multiplicative error in O(m log n (log log n)2) time. (m = # edges of G)


Results on sparsifiers

Results on Sparsifiers

Cut Sparsifiers

Spectral Sparsifiers

Karger ‘94

Benczur-Karger ‘96

Combinatorial

Fung-Hariharan-Harvey-Panigrahi ‘11

Spielman-Teng ‘04

Spielman-Srivastava ‘08

Linear Algebraic

Batson-Spielman-Srivastava ‘09

de Carli Silva-Harvey-Sato ‘11

These construct sparsifiers with n logO(1) n / ²2 edges

These construct sparsifiers with O(n / ²2) edges


Sparsifiers by random sampling

Sparsifiers by Random Sampling

  • The complete graph is easy!Random sampling gives an expander (ie. sparsifier) with O(n log n) edges.


Sparsifiers by random sampling1

Sparsifiers by Random Sampling

Eliminate most of these

  • Can’t sample edges with same probability!

  • Idea [BK’96]Sample low-connectivity edges with high probability, and high-connectivity edges with low probability

Keep this


Non uniform sampling algorithm bk 96

Non-uniform sampling algorithm [BK’96]

  • Input: Graph G=(V,E), weights u : E !R+

  • Output: A subgraph H=(V,F) with weights w : F !R+

  • Choose parameter ½

  • Compute probabilities { pe:e2E}

  • For i=1 to ½

  • For each edge e2E

  • With probability pe,Add e to F Increase we by ue/(½pe)

Can we dothis so that thecut values aretightly concentratedand E[|F|]=nlogO(1)n?

  • Note: E[|F|] ·½¢epe

  • Note: E[ we ] = ue8e2E

    • ) For every UµV, E[ w(±H(U)) ] = u(±G(U))


Benczur karger 96

Benczur-Karger ‘96

  • Input: Graph G=(V,E), weights u : E !R+

  • Output: A subgraph H=(V,F) with weights w : F !R+

  • Choose parameter ½

  • Compute probabilities { pe:e2E}

  • For i=1 to ½

  • For each edge e2E

  • With probability pe,Add e to F Increase we by ue/(½pe)

Can we dothis so that thecut values aretightly concentratedand E[|F|]=nlogO(1)n?

Can approximateall values in

m logO(1)n time.

  • Set ½ = O(logn/²2).

  • Let pe = 1/“strength” of edge e.

  • Cuts are preserved to within (1§²) and E[|F|] = O(n logn/²2)


Fung hariharan harvey panigrahi 11

Fung-Hariharan-Harvey-Panigrahi ‘11

  • Input: Graph G=(V,E), weights u : E !R+

  • Output: A subgraph H=(V,F) with weights w : F !R+

  • Choose parameter ½

  • Compute probabilities { pe:e2E}

  • For i=1 to ½

  • For each edge e2E

  • With probability pe,Add e to F Increase we by ue/(½pe)

Can we dothis so that thecut values aretightly concentratedand E[|F|]=nlogO(1)n?

Can approximateall values in

O(m + n log n) time

  • Set ½ = O(log2n/²2).

  • Let pst = 1/(min cut separating s and t)

  • Cuts are preserved to within (1§²) and E[|F|] = O(n log2n/²2)


Graph sparsifiers a survey

  • Letkuv = min size of a cut separating u and v.Recall sampling probability is pe = 1/ke

  • Partition edges into connectivity classesE = E1[E2[ ... Elog nwhere Ei = { e : 2i-1·ke<2i }

  • Prove weight of sampled edges that each cuttakes from each connectivity class has low error

  • Key point: Edges in ±(U)ÅEi have roughly same pe

  • This yields a sparsifier

U


Graph sparsifiers a survey

  • Notation:

    • C = ±(U) is a cut

    • Ci= ±(U) ÅEi is a cut-induced set

  • Need to prove: for every Ci

Prove weight of sampled edges that each cuttakes from each connectivity class has low error

C2

C3

C1

C4


Graph sparsifiers a survey

  • Notation:Ci= ±(U) ÅEi is a cut-induced set

Prove 8 cut-induced set Ci

  • Key Ingredients

    • Hoeffding bound: Provesmall

    • Bound on # small cut-induced sets:For most of these events, u(C) is large.In other words, #{ cut-induced sets Ciinduced by a small cut C }is small.

C2

C3

C1

C4


Counting small cut induced sets

Counting Small Cut-Induced Sets

  • Theorem:[Fung-Hariharan-Harvey-Panigrahi ‘11]

    Let G=(V,E) be a graph. Fix any BµE.

    Suppose ke¸K for all e in B. (kuv = min size of a cut separating u and v)

    Then, for every ®¸1,|{ ±(U) ÅB : |±(U)|·®K }| < n2®.

  • Corollary: [Karger ‘93]

    Let G=(V,E) be a graph.

    Let K be the edge-connectivity of G. (i.e., global min cut value)

    Then, for every ®¸1,|{ ±(U) : |±(U)|·®K }| < n2®.


Summary for cut sparsifiers

Summary for Cut Sparsifiers

  • Do non-uniform sampling of edges,with probabilities based on “connectivity”

  • Analysis involves:

    • Decomposing the graph

    • Hoeffding bounds to analyze each “cut”

    • Cut-counting theorem: “few small cuts”

  • BK’96 had weaker cut-counting theorem, but had more complicated “connectivity” notion.

  • Can get sparsifiers with O(n log n / ²2) edges

    • Optimal for any independent sampling algorithm


Spectral sparsification

Spectral Sparsification

  • Input: Graph G=(V,E), weights u : E !R+

  • Recall: xTLGx = st2Eust (xs-xt)2

  • Goal: Find weights w : E !R+ such that very fewwe are non-zero, and(1-²) xTLGx·e2EwexTLex·(1+²) xTLGx8x2RV

    , (1- ²) LG¹e2EweLe¹ (1+²) LG

  • General Problem: Given matrices Le satisfying eLe=LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG

Call this xTLstx


The general problem sparsifying sums of psd matrices

The General Problem:Sparsifying Sums of PSD Matrices

  • General Problem: Given PSD matrices Les.t. eLe=LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG

  • Theorem:[Ahlswede-Winter ’02]Randomized alg gives w with O( n log n/²2 ) non-zeros.

  • Theorem:[de Carli Silva-Harvey-Sato ‘11],building on [Batson-Spielman-Srivastava ‘09]Deterministic alg gives w with O( n/²2 ) non-zeros.

    • Cut & spectral sparsifiers with O(n/²2) edges [BSS’09]

    • Sparsifiers with more properties and O(n/²2) edges [dHS’11]


Vector case

Vector Case

Vector Case

  • Vector problem: Given vectors ve2[0,1]ns.t. eve = v,find coefficients we, mostly zero, such thatkeweve - v k1 · ²

  • Theorem [Althofer ‘94, Lipton-Young ‘94]:There is a w with O(log n/²2) non-zeros.

  • Proof: Random sampling & Hoeffding inequality.

  • Multiplicative version: There is a w with O(n log n/²2) non-zeros such that (1-²) v·eweve· (1+²) v

  • General Problem: Given PSD matrices Les.t. eLe = L, find coefficients we, mostly zero, such that (1-²) L¹eweLe¹ (1+²) L


Concentration inequalities

Concentration Inequalities

  • Theorem: [Chernoff ‘52, Hoeffding ‘63]Let Y1,…,Yk be i.i.d. randomnon-negative real numberss.t. E[ Yi ] = Z andYi·uZ. Then

  • Theorem: [Ahlswede-Winter ‘02]Let Y1,…,Yk be i.i.d. randomPSD nxn matricess.t. E[ Yi ] = Z andYi¹uZ. Then

The only difference


Balls bins example

“Balls & Bins” Example

  • Problem: Throw k balls into n bins. Wantmax load / min load · 1+². How big should k be?

  • AW Theorem: Let Y1,…,Yk be i.i.d. random PSD matricessuch that E[ Yi ] = Z andYi¹uZ. Then

  • Solution: Let Yi be all zeros, except for a single n in a random diagonal entry.

    Then E[ Yi ] = I =: Z, and ¸max(Yi Z-1) = n =: u.Set k = £(n logn/²2). Then, with high probability,every diagonal entry of i Yi/k is in [1-²,1+²].


Solving the general problem

Solving the General Problem

  • General Problem: Given PSD matrices Les.t. eLe = LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG

  • AW Theorem: Let Y1,…,Yk be i.i.d. random PSD matricessuch that E[ Yi ] = Z andYi¹uZ. Then

  • Solve General Problem with O(nlogn/²2) non-zeros

    • Repeat k:=£(nlogn/²2) times

      • Pick an edge e with probability pe := Tr(LeLG-1)/n

      • Increment we by 1/k¢pe


Derandomization

Derandomization

  • Vector problem: Given vectors ve2[0,1]ns.t. eve = v,find coefficients we, mostly zero, such thatkeweve - vk1 · ²

  • Theorem[Young ‘94]: The multiplicative weights method deterministically gives w with O(log n/²2) non-zeros

    • Or, use pessimistic estimators on the Hoeffding proof

  • General Problem: Given PSD matrices Les.t. eLe = LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG

  • Theorem [de Carli Silva-Harvey-Sato ‘11]:The matrix multiplicative weights method (Arora-Kale ‘07)deterministically gives w with O(n log n/²2) non-zeros

    • Or, use matrix pessimistic estimators (Wigderson-Xiao ‘06)


Mwum for balls bins

MWUM for “Balls & Bins”

  • Let ¸i = load in bin i. Initially ¸=0. Want: 1·¸i and ¸i·1.

  • Introduce penalty functions “exp(l-¸i)” and “exp(¸i-u)”

  • Find a bin ¸i to throw a ball into such that,increasing l by ±l and u by ±u, the penalties don’t grow.

    i exp(l+±l - ¸i’) · i exp(l-¸i)

    i exp(¸i’-(u+±u)) · i exp(¸i-u)

  • Careful analysis shows O(n log n/²2) balls is enough

¸ values:

u

l

0

1


Mmwum for general problem

MMWUM for General Problem

  • Let A=0 and ¸ its eigenvalues. Want: 1·¸i and ¸i·1.

  • Use penalty functions Tr exp(lI-A) and Tr exp(A-uI)

  • Find a matrix Le such that adding ®Le to A,increasing l by ±l and u by ±u, the penalties don’t grow.

    Trexp((l+±l)I- (A+®Le)) · Trexp(lI-A)

    Trexp((A+®Le)-(u+±u)I) · Tr exp(A-uI)

  • Careful analysis shows O(n log n/²2) matrices is enough

¸ values:

u

l

0

1


Beating sampling mmwum

Beating Sampling & MMWUM

  • To get a better bound, try changing the penalty functions to be steeper!

  • Use penalty functions Tr (A-lI)-1 and Tr (uI-A)-1

  • Find a matrix Le such that adding ®Le to A,increasing l by ±l and u by ±u, the penalties don’t grow.

    Tr((A+®Le)-(l+±l)I)-1· Tr(A-lI)-1

    Tr((u+±u)I-(A+®Le))-1· Tr (uI-A)-1

All eigenvaluesstay within [l, u]

¸ values:

u

l

0

1


Beating sampling mmwum1

Beating Sampling & MMWUM

  • To get a better bound, try changing the penalty functions to be steeper!

  • Use penalty functions Tr (A-lI)-1 and Tr (uI-A)-1

  • Find a matrix Le such that adding ®Le to A,increasing l by ±l and u by ±u, the penalties don’t grow.

    Tr((A+®Le)-(l+±l)I)-1· Tr(A-lI)-1

    Tr((u+±u)I-(A+®Le))-1· Tr (uI-A)-1

  • General Problem: Given PSD matrices Les.t. eLe = LG, find coefficients we, mostly zero, such that (1-²) LG¹eweLe¹ (1+²) LG

  • Theorem:[Batson-Spielman-Srivastava ‘09] in rank-1 case,[de Carli Silva-Harvey-Sato ‘11] for general caseThis gives a solution w with O( n/²2 ) non-zeros.


Applications

Applications

  • Theorem:[de Carli Silva-Harvey-Sato ‘11]Given PSD matrices Les.t. eLe = L, there is analgorithm to find w with O( n/²2 ) non-zeros such that (1-²) L¹eweLe¹ (1+²) L

  • Application 1: Spectral Sparsifiers with CostsGiven costs on edges of G, can find sparsifier H whose cost isat most (1+²) the cost of G.

  • Application 2: Simultaneous Spectral SparsifiersGiven two graphs G1 & G2 with a bijection on their edges,can choose edges that simultaneously sparsify G1 & G2.

  • Application 3: Sparse SDP Solutionsmin { cTy : iyiAiº B, y¸0 } where Ai’s and B are PSDhas nearly optimal solution with O(n/²2) non-zeros.


Open questions

Open Questions

  • Use of sparsifiers in other areas (infoviz, etc.)

  • Sparsifiers for directed graphs

  • Construction of expander graphs

  • More control of the weights we

  • A combinatorial proof of spectral sparsifiers

  • More applications of our general theorem


  • Login