A Combinatorial, Primal-Dual Approach to Semidefinite Programming

1. Satyen Kale (Yahoo! Research) Joint work with SanjeevArora(Princeton) A Combinatorial, Primal-Dual Approach to Semidefinite Programming

2. Semidefinite Programming Semidefinite Program (SDP): find Xs.t. A1² X · 0 A2² X · 0  Am² X · 0 X º 0 Tr(X) = 1 All eigenvalues are non-negative ´9 v1, v2, …, vns.t. Xij = vi¢vj A ² B = Tr(AB) = ijAijBij Density matrix

3. Max Cut [GW’95] (aÇ:b)Æ(:aÇc)Æ(aÇb)Æ(:cÇb) 1 0 0 0 1 1 0 1 Graph Coloring [KMS’98, ACC’06] Constraint Satisfaction [ACMM’05, R ’08, RS ‘09,..] Semidefinite Programming Balanced Partitioning [ARV’04] Control Theory SDP

4. Algorithms for SDP MISSING Combinatorial, Primal-Dual algorithms (analogous to flow algorithms via LP) A1² X · 0  An² X · 0 iwiAi² X · 0 • Interior point methods [NN’90, A’95]: • O(√n ¢ m3) time • Ellipsoid method[GLS’81]: • O(n8) iterations • Lagrangian Relaxation [KL’95], [AHK’05]: • Reduction to eigenvectors • poly(1/) dependence on, limits applicability

5. Challenges in designing efficient, combinatorial algorithms for SDP • Positive semidefiniteness hard to maintain • Rounding algorithms exploit geometric structure (e.g. negative-type metrics) • Matrix operations inefficient to implement

6. Our Work A general scheme that yields fast, combinatorial, primal-dual approximation algorithms for various combinatorial optimization problems using SDP relaxations

7. Our Results: Primal-Dual algorithms • Õ(m + n1.33) [S’09] Unknown how to achieve using LP! • Õ(m + n1.33) [S’09]

8. Near-linear time alg for MAXCUT • Also yields Õ(m) time algorithm for approximating MAXCUTSDP • Gives first near-linear implementation of Goemans-Williamson approximation algorithm • Previous best: Õ(nm) [KL’95]

9. LP via Multiplicative Weights [L’88] a1¢ x · a2¢ x ·  am¢ x · x ¸ 0 i xi = 1 a1¢ x · 0 a2¢ x · 0  am¢ x · 0 x ¸ 0 i xi = 1 Thm: Finds -feasible x in O(2log(n)/2)iterations. = maxij |aij| (“width”) Coordinate-wise Multiplicative Weights Initialize: x1= (1/n, …, 1/n) Update: xt+1 = xt£ exp(-ait)/t t = normalization factor xt Violation Checker Find its.t. ait¢xt >  ait Convex combination of constraints allowed: iyiai¢xt>, where yi¸ 0, iyi = 1. Hence Primal-Dual. Analysis uses tas potential function

10. SDP via Matrix Multiplicative Weights A1² X · 0 A2² X · 0  Am² X · 0 X º 0 Tr(X) = 1 Thm: Finds -feasible X inO(2log(n)/2) iterations. = maxikAik (“width”) Matrix MW Initialize: X1= (1/n)I Update: Xt+1= exp(-ts=1Ais)/t t = normalization factor Xt Violation Checker Find its.t. Ait²Xt> Ait Convex combination of constraints allowed: iyiAi²Xt>, whereyi¸ 0, iyi = 1. Hence Primal-Dual. Analysis uses t as potential function

11. The Matrix Exponential • Matrix exponential: exp(A) = I + A + A2/2! + A3/3! + … • Always PSD:exp(A) º 0 • Tricky beast: exp(A+B) = exp(A) exp(B) • Computation: • O(n3)time • Computing exp(A)v: Õ(m) time (arises as soln of ODE) Golden-Thompson inequality: Tr(exp(A+B)) ·Tr(exp(A)exp(B)) Matrix MW Initialize: X1= (1/n)I Update: Xt+1= exp(-ts=1Ais)/t t = normalization factor

12. v2 v1 SDP Opt: X Xij = vi¢vj vn Approximation via SDP Relaxations Quadratic Program Reduction Input Relaxation Rounding Algorithm Primal-Dual SDP SDP: Vector variables SDP Solver

13. v2 v1 vn Primal-Dual SDP Framework Quadratic Program Reduction Input Rounding Algorithm Relaxation Primal-Dual SDP y1,…, ym Matrix MW SDP: Vector variables X Xij = vi¢vj

14. Cut (S, S’) is c-balanced if |S|, |S’| ¸cn Min c-Balanced Separator: c-balanced cut of min capacity Numerous applications: Divide-and-conquer algs Markov chains Geometric embeddings Clustering Layout problems … S S’ Approximating Balanced Separator G = (V, E)

15. S S’ vi = -1 vi = 1 SDP for Balanced Separator G = (V, E) ¼kvi – vjk2 = 0 if i, j on same side, = 1 otherwise SDP min i,j2 E ¼kvi – vjk2 8i: kvik2 = 1 i, j ¼kvi – vjk2 ¸ c(1-c)n2 8ijk: kvi–vjk2 + kvj–vkk2¸kvi–vkk2

16. SDP (C1)i,j2 E ¼kvi – vjk2< (C2) 8i: kvik2 = 1 (C3) i, j ¼kvi – vjk2 ¸ c(1-c)n2 (C4) 8ijk: kvi–vjk2 + kvj–vkk2¸kvi–vkk2 To find violated constraints, given X (thus, vi): Check (C2) and (C3) To check (C1) and (C4): find multicommodity flow s.t. Total flow from any node is at most d =Õ(/n) Total flow on any edge is at most 1 i,jfijkvi – vjk2¸4 Either (C1) or (C4) violated Finding Violated Constraints Binary search parameter Bounds width  fij = total flow betweeni, j

17. d d v2 d d v1 d d d vn Xij = vi¢vj Finding flow: O(log n) approx kvik2 = 1 ij¼kvi – vjk2¸ c(1-c)n2 To find flow s.t.ijfijkvi – vjk2¸4 Max flow • Thm: This yields either: • a flow s.t. ijfijkvi – vjk2¸4 • a cut of value O(log(n))

18. d d d d v2 v1 d d d d vn Xij = vi¢vj Finding flow: O(plog n) approx kvik2 = 1 ij¼kvi – vjk2¸ c(1-c)n2 To find flow s.t.ijfijkvi – vjk2¸4 Multicommodity flow • Thm: This yields either: • a flows.t. ijfijkvi – vjk2¸4 • a cut of value O(√log(n))

19. Other Applications of Matrix MW

20. Future Work • Make SDP solver efficient in practice • Sampling, rescaling techniques might be useful • ``Combinatorialize’’ interior-point methods? • Near linear-time algs for approx max flow?

21. Thank You!