1 / 20

A Combinatorial, Primal-Dual Approach to Semidefinite Programs

A Combinatorial, Primal-Dual Approach to Semidefinite Programs. Satyen Kale Microsoft Research Joint work with Sanjeev Arora Princeton University. Max Cut [GW’95]. Balanced Partitioning [ARV’04]. (a Ç : b) Æ ( : a Ç c) Æ (a Ç b) Æ ( : c Ç b).

elita
Download Presentation

A Combinatorial, Primal-Dual Approach to Semidefinite Programs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A Combinatorial, Primal-Dual Approach to Semidefinite Programs Satyen Kale Microsoft Research Joint work with Sanjeev Arora Princeton University

  2. Max Cut [GW’95] Balanced Partitioning [ARV’04] (a Ç:b) Æ (:a Ç c) Æ (a Ç b) Æ (:c Ç b) 1 0 0 0 1 1 0 1 Graph Coloring [KMS’98, ACC’06] Constraint Satisfaction [ACMM’05] The Ubiquity of Semidefinite Programming dx(t)/dt = Ax(t) Control Theory SDP

  3. MISSING Combinatorial, Primal-Dual algorithms A1² X ¸ b1  Am² X ¸ bm i wi(Ai² X – bi) ¸ 0 Algorithms for SDP • Interior point methods [NN’90, A’95]: • O(√m(n3 + m)) time • Ellipsoid method [GLS’81]: • O(n8) iterations • Lagrangian Relaxation [KL’95], [AHK’05]: • Reduction to eigenvectors • poly(1/) dependence on, limits • applicability (e.g. Sparsest Cut)

  4. Primal-Dual algorithms for LP:Multicommodity Flows Unit capacities Objective:Maximize total flow, while respecting edge capacities

  5. 1 1 1 1 1 1 1 1 1 1 1 1 1 Primal-Dual algorithms for LP:Multicommodity Flows 1 +  1 • Edge weights we = 1 • Repeat until some e reaches capacity: • Find shortest path p • Route 2/log(m) flow on p • Update we for all e 2 p as weà we¢ (1 + ) • Output flow. Thm [GK’98]: Stops in Õ(m) rounds with (1 – 2)¢OPT flow. Total running time is Õ(m2).

  6. Primal-Dual algorithms for LP:Multicommodity Flows • Primal-Dual algorithm • Combinatorial • Multiplicative Weights Update Rule • Edge weights we = 1 • Repeat until some e reaches capacity: • Find shortest path p • Route 2/log(m) flow on p • Update we for all e 2 p as weà we¢ (1 + ) • Output flow. Thm [GK’98]: Stops in Õ(m) rounds with (1 – 2)¢OPT flow. Total running time is Õ(m2).

  7. Starting point for our work: Analogous primal-dual algorithms for SDP? • Difficulties: • Positive semidefiniteness hard to maintain • Rounding algorithms exploit geometric structure (e.g. negative-type metrics) • Matrix operations inefficient to implement • Our work: a generic scheme that yields fast, combinatorial, primal-dual algorithms for various optimization problems using SDP

  8. Our Results: Primal-Dual algorithms [AHK’04] [AHK’04]

  9. Our Results: Primal-Dual algorithms [AHK’04] Cannot be achieved by LPs even! [AHK’04]

  10. Our Results: Near-linear time algorithm for Max Cut • Õ(n + m) time algorithm to approximate Max Cut SDP to (1 + o(1)) factor • Previous best: Õ(n2) time algorithm by Klein, Lu ‘95

  11. Prototype MW for LPs: Winnow [L’88] - -  - a1¢ x ¸ a2¢ x ¸  am¢ x ¸ x ¸ 0 i xi = 1 0 0  0 Thm: Finds x in O(2log(n)/2) iterations. = maxij |aij| MW for LP: Init: x = (1/n, …, 1/n) Update: xj = xj¢ exp(- aij)/  = norm. factor x Oracle Find i s.t. ai¢ x < - ai Convex comb. of constraints allowed: i yi ai¢ x < -, where yi¸ 0, i yi = 1. Hence Primal-Dual. MW algorithm: boosting, hard-core sets, zero-sum games, flows, portfolio, … Analysis uses  as potential fn.

  12. Prototype Matrix MW for SDPs - -  - A1² X ¸ A2² X ¸  Am² X ¸ X º 0 Tr(X) = 1 0 0  0 Thm: Finds X in O(2log(n)/2) iterations. = maxikAik Matrix MW for SDP: Init: X = I/n Update: X= exp(-ts=1 Ais)/  = norm. factor X Oracle Find it s.t. Ait² X < - Ait Convex comb. of constraints allowed: i yi Ai²X < -, where yi¸ 0, i yi = 1. Hence Primal-Dual. Analysis uses  as potential fn.

  13. The Matrix Exponential • Matrix exponential: exp(A) = I + A + A2/2! + A3/3! + … • Always PSD:exp(A) º 0 • Tricky beast: exp(A+B) = exp(A) exp(B) • Computation: • O(n3) time • Usually, can use J-L lemma • Only need exp(A)v products: Õ(m) time Golden-Thompson inequality: Tr(exp(A+B)) · Tr(exp(A)exp(B)) Matrix MW for SDP: Init: X = I/n Update: X= exp(-ts=1 Ais)/  = norm. factor

  14. Input v2 v1 SDP Opt vn Approximation via SDP Relaxations 0-1 Quadratic Program Reduction Relaxation Rounding Algorithm Primal-Dual SDP SDP: Vector vars SDP Solver

  15. Input v2 v1 vn Primal-Dual SDP Framework 0-1 Quadratic Program Reduction Rounding Algorithm Relaxation Primal-Dual SDP y1,…, ym Matrix MW SDP: Vector vars X

  16. Approximating Balanced Separator Cut (S, S’) is c-balanced if |S|, |S’| ¸cn Min c-Balanced Separator:c-balanced cut of min capacity Numerous applications: Divide-and-conquer algorithms Markov chains Geometric embeddings Clustering Layout problems … S S’ G = (V, E)

  17. SDP for Balanced Separator ¼kvi – vjk2 = 0 if i, j on same side, = 1 otherwise SDP: min i,j 2 E ¼kvi – vjk2 8i: kvik2 = 1 8ijk: kvi–vjk2 + kvj–vkk2¸kvi–vkk2 i, j ¼kvi – vjk2 ¸ c(1-c)n2 G = (V, E) S S’ vi = -1 vi = 1

  18. Implementing Oracle Primal: mini,j 2 E ¼kvi – vjk2 8i: kvik2 = 1 8ijk: kvi–vjk2 + kvj–vkk2¸kvi–vkk2 i, j ¼kvi – vjk2 ¸ c(1-c)n2 Oracle: given X (thus, vi), : check first and third constraints  inequalities: dual vars , multicommodity flow s.t. total flow from any node is · d no capacity is exceeded i,j fijkvi – vjk2¸ Bounds eigenvalues viademand graph Laplacian • Thm: Givenvi, : • Max-flow)desired flow or cut of • value O(log(n)¢). • Multicommodity flow) desired flow • or cut of value O(√log(n)¢). fij = total flow between i, j

  19. Conclusions and Future Work • Matrix MW algorithm: more applications in • Solving SDPs: e.g. Min Linear Arrangement • Quantum algorithms: density matrix is a central concept • Learning: e.g. online variance minimization [WK COLT’06], online PCA [WK NIPS’06] • Other applications? • Linear time algorithm for Sparsest Cut? • Our algorithm runs in Õ(m + n1.5) time for an O(log n) approximation

  20. Thank you!

More Related