1 / 42

Recent Progress in Approximability

Recent Progress in Approximability. Administrivia. Please Fill Up Survey: http:// www.surveymonkey.com/s/9TSVQM7. Most agreeable times: Monday 2:30-4:00 Wednesday 4:00-5:30 Thursday 4:00-5:30 Friday 1:00-2:30.

mauli
Download Presentation

Recent Progress in Approximability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Recent Progress in Approximability

  2. Administrivia Please Fill Up Survey: http://www.surveymonkey.com/s/9TSVQM7 Most agreeable times: Monday 2:30-4:00 Wednesday 4:00-5:30 Thursday 4:00-5:30 Friday 1:00-2:30 Evaluation: 6-8 short homeworks and class participation.

  3. Max Cut Max CUT Input: A weighted graph G Find: A Cut with maximum number/weight of crossing edges 10 15 7 1 1 3 MaxCut is NP-complete (Karp’s original list of 21 NP-complete problems (1971) Fraction of crossing edges

  4. Approximation Algorithms An algorithm Ais an α-approximation for a problem if for every instance I, A(I) ≥ α ∙ OPT(I) --Vast Literature--

  5. Max Cut Trivial ½ Approximation Assign each vertex randomly to left or right side of the cut 10 15 7 Analysis For every edge e, Probability[edge is cut] = ½ Fraction of edges cut = ½ Optimum MaxCut < 1 So, Solution returned = ½ > ½ *Optimum MaxCut 1 1 3 Till 1994, this was the state of the art. Many linear programming techniques were known to NOT get any better approximation. Max CUT Input: A weighted graph G Find: A Cut with maximum number/weight of crossing edges

  6. The Tools Till 1994, A majority of approximation algorithms directly or indirectly relied on Linear Programming. In 1994, Semidefinite Programming based algorithm for Max Cut [Goemans-Williamson] Semidefinite Programming - A generalization of Linear Programming. Semidefinite Programming is the one of the most powerful tools in approximation algorithms.

  7. Max Cut SDP Quadratic Program Variables : x1 , x2 … xn xi = 1 or -1 Maximize Semidefinite Program Variables : v1 , v2 … vn • | vi |2= 1 Maximize -1 1 -1 10 1 -1 15 1 7 1 1 1 -1 -1 -1 3 -1 Relax all the xi to be unit vectors instead of {1,-1}. All products are replaced by inner products of vectors

  8. MaxCut -1 1 Semidefinite Program [Goemans-Williamson 94] Variables : v1 , v2 … vn • |vi|2= 1 Maximize v2 Max Cut Problem Given a graph G, Find a cut that maximizes the number of crossing edges -1 10 1 -1 15 1 7 v1 v3 1 1 1 -1 -1 -1 3 Semidefinite Program: [Goemans-Williamson 94] Embedd the graph on the N - dimensional unit ball, Maximizing ¼ (Average Squared Length of the edges) -1 v5 v4

  9. MaxCut Rounding v2 • Cut the sphere by a random hyperplane, and output the induced graph cut. • A 0.878 approximation for the problem. • [Goemans-Williamson] v1 v3 v5 v4

  10. Analysis Rounding Ratio > 0.878 SDP Optiumum v2 v2 10 Integrality Gap 15 7 v1 v1 v3 v3 1 1 Algorithm’s Output 3 Optimal MaxCut 0 1 Algorithm Output > 0.878 X SDP Optimum > 0.878 X Optimum MaxCut v5 v5 v4 v4

  11. Rounding Ratio > 0.878 Integrality Gap SDP Optiumum 0 1 For any rounding algorithm A, and a SDP relaxation ¦ v2 v2 value of optimal solution value of SDP solution minimum over all instances 10 = 15 7 v1 v1 v3 v3 1 1 Algorithm’s Output rounding – ratioA (approximation ratio) 3 ≤ Optimal MaxCut integrality gap = “algorithm achieves the gap’’ minimum over all instances v5 v5 value of rounded solution value of SDP solution = v4 v4

  12. Inapproximability Is 0.878 the best possible approximation ratio for MaxCut? 10 Polynomial time reduction 1 -1 3-SAT Instance 15 1 7 1 1 1 -1 -1 -1 3 Satisfiable MaxCut value = K Unsatisfiable MaxCut value < K

  13. What we need.. Polynomial time reduction 3-SAT Instance 10 1 -1 15 1 7 1 (Completeness) Satisfiable MaxCut value = K 1 1 -1 -1 -1 3 MaxCut value < 0.9K (Soundness) Unsatisfiable If we had a polytime 0.95 approximation algorithm for MaxCut  A polytime algorithm for 3-SAT

  14. A probabilistically checkable proof (PCP) Goal: Alice wants to prove to Bob that 3-SAT instance A is satisfiable 3-SAT Instance A Bob (polytime machine) Satisfying assignment Alex

  15. A probabilistically checkable proof (PCP) Goal: Alice wants to prove to Bob that 3-SAT instance A is satisfiable Probabilistically Checkable Proof A cut of value > 0.9 10 10 1 1 Alex -1 -1 Bob (polytime machine) 15 15 1 1 7 7 3-SAT Instance A 1 1 3-SAT Instance A 1 1 1 1 -1 -1 -1 -1 -1 -1 3 3 Verifier (Bob): Sample a random edge in graph, Accept if edge is cut. Polynomial time reduction Polynomial time reduction Prob[Bob Accepts] = Value of the Cut

  16. Suppose, Polynomial time reduction 3-SAT Instance 10 1 (Completeness) Satisfiable MaxCut value = 0.99 -1 15 1 7 1 1 1 -1 MaxCut value < 0.9 -1 (Soundness) Unsatisfiable -1 3 Completeness: There exists a ``proof” that Bob accepts with probability 0.99 Soundness: No matter what Alex does, Bob accepts with probability < 0.9 Bob reads only 2 bits of the proof!!

  17. Analogy to Math Proofs Could you check the proof of a theorem with any reasonable confidence by reading only 3 bits of the proof??? Guess: Probably Not.. Max-SNP complexity class was defined, because it was believable that we will never be able to get a Gap Reduction aka Probabilistically Checkable Proof for NP.

  18. PCP Theorem: [Arora-Lund-Motwani-Sudan-Szegedy 1991] Max-3-SAT is NP-hard to approximate better than 1- 10^{-100}. Long and very difficult proof, simplified over the years.. (*Check out History of PCP Theorem: http://www.cs.washington.edu/education/courses/cse533/05au/pcp-history.pdf) Completely new proof by IritDinur in 2005. Corollary: Max-Cut is NP-hard to approximate better than 1- 10^{-200}.

  19. Hastad’s 3-Query PCP[HåstadSTOC97] For any ε> 0, NP has a 3-query probabilistically checkable proof system such that: • Completeness = (1 – ε) • Soundness = 1/2 + ε Verifier reads only 3-bits, and checks a linear equation on them! Xi + Xj = Xk + c (mod p) Alternately,

  20. Hastad’s 3-Query PCP[1997] For any ε> 0, given a set of linear equations modulo 2 , it is NP-hard to distinguish between: • (1 – ε) – fraction of the equations can be satisfied. • 1/2 + ε– fraction of the equations can be satisfied. All equations are of the form Xi + Xj = Xk + c (mod p) By Very Clever Gadget reductions,[Sudan-Sorkin-Trevisan-Williamson]MaxCut is NP-hard to approximate beyond 0.94.

  21. Gap for MaxCUT Algorithm = 0.878 Hardness = 0.941 Approximability of CSPs ALGORITHMS [Charikar-Makarychev-Makarychev 06] [Goemans-Williamson] [Charikar-Wirth] [Lewin-Livnat-Zwick] [Charikar-Makarychev-Makarychev 07] [Hast] [Charikar-Makarychev-Makarychev 07] [Frieze-Jerrum] [Karloff-Zwick] [Zwick SODA 98] [Zwick STOC 98] [Zwick 99] [Halperin-Zwick 01] [Goemans-Williamson 01] [Goemans 01] [Feige-Goemans] [Matuura-Matsui] [Trevisan-Sudan-Sorkin-Williamson] MAX k-CSP Unique Games MAX 3-CSP NP HARD MAX 3-AND MAX 3-MAJ MAX E2 LIN3 MAX 3 DI-CUT MAX 4-SAT MAX DI CUT MAX 3-SAT MAX CUT MAX 2-SAT 0 1 MAX Horn SAT MAX k-CUT

  22. x-y = 11 (mod 17) x-z = 13 (mod 17) … …. z-w = 15(mod 17) Given linear equations of the form: Xi – Xk = cik mod p Satisfy maximum number of equations. Towards bridging this gap, In 2002, SubhashKhot introduced the Unique Games Conjecture Unique Games Conjecture [Khot 02] [KKMO] For every ε> 0, for large enough p, Given : 1-ε(99%) satisfiable system, NP-hard to satisfy ε(1%)fraction of equations.

  23. Unique Games Conjecture A notorious open problem. Hardness Results: No constant factor approximation for unique games. [Feige-Reichman]

  24. UGC HARD NP HARD Assuming UGC MAX k-CSP UGC Hardness Results [Khot-Kindler-Mossel-O’donnell] [Austrin 06] [Austrin 07] [Khot-Odonnell] [Odonnell-Wu] [Samorodnitsky-Trevisan] Unique Games MAX 3-CSP MAX 3-AND MAX 3-MAJ For MaxCut, Max-2-SAT, Unique Games based hardness = approximation obtained by Semidefinite programming! MAX E2 LIN3 MAX 3 DI-CUT MAX 4-SAT MAX DI CUT MAX 3-SAT MAX CUT MAX 2-SAT MAX Horn SAT MAX k-CUT 0 1

  25. The Connection GENERIC ALGORITHM MAX k-CSP UGC Hard Unique Games Theorem: Assuming Unique Games Conjecture, For every CSP, “the simplest semidefinite programs give the best approximation computable efficiently.” MAX 3-CSP MAX 3-AND MAX 3-MAJ MAX E2 LIN3 MAX 3 DI-CUT MAX 4-SAT MAX DI CUT MAX 3-SAT MAX CUT 0 1 MAX 2-SAT MAX Horn SAT MAX k-CUT

  26. Assuming the Unique Games Conjecture, A simple semidefinite program (Basic-SDP) yields the optimal approximation ratio for • Constraint Satisfaction Problems [Raghavendra`08][Austrin-Mossel] • Max Cut [Khot-Kindler-Mossel-ODonnell][Odonnell-Wu] • Max 2Sat [Austrin07][Austrin08] Metric Labeling Problems [Manokaran-Naor-Raghavendra-Schwartz`08] • Multiway Cut, 0-extension • Ordering CSPs [Charikar-Guruswami-Manokaran-Raghavendra-Hastad`08] • Max Acyclic Subgraph, Betweeness Is the conjecture true? • Strict Monotone CSPs [Kumar-Manokaran-Tulsiani-Vishnoi`10] • Vertex Cover [Khot-Regev], Hypergraph Vertex Cover Many many ways to disprove the conjecture! Find a better algorithm for any one of these problems. Kernel Clustering Problems [Khot-Naor`08,10] Grothendieck Problems[Khot-Naor, Raghavendra-Steurer]

  27. The UG Barrier Constraint Satisfaction Problems If UGC is true, Then Simplest SDPs give the best approximation possible. Graph Labelling Problems Ordering CSPs Kernel Clustering Problems UGC HARD Monotone Min-One CSPs If UGC is false, Hopefully, a new algorithmic technique will arise.

  28. What if UGC is false? Could existing techniques ( LPs/SDPs) disprove the UGC?

  29. What if UGC is false?

  30. UGC is false New algorithms? • Constraint Satisfaction Problems [Raghavendra`08] • Max Cut, Max 2Sat Metric Labeling Problems [MNRS`08] • Multiway Cut, 0-extension • Ordering CSPs [GMR`08] • Max Acyclic Subgraph, Betweeness Unique Games Problem X • Strict Monotone CSPs [KMTV`10] • Vertex Cover, Hypergraph Vertex Cover Kernel Clustering Problems [KN`08,10] Grothendieck Problems[KNS`08, RS`09] … UGC is false  New algorithm for Problem X [Feige-Kindler-Odonnell,Raz’08, BHHRRS’08] Despite considerable efforts, No such reverse reduction known for any of the above problems

  31. Graph Expansion A random neighbor of a random vertex in S is outside of S with probability expansion(S) d-regular graph G d Extremely well-studied, many different contexts pseudo-randomness, group theory, online routing, Markov chains, metric embeddings, … Uniform Sparsest Cut Problem Given a graph G compute ФGand find the set S achieving it. # edges leaving S expansion(S) = d |S| • Approximation Algorithms: • Cheeger’s Inequality [Alon][Alon-Milman] • Given a graph G, if the normalized adjacency matrix has second eigen value λ2 then, • A log n approximation algorithm [Leighton-Rao 98-99?]. • A sqrt(log n) approximation algorithm using semidefinite programming [Arora-Rao-Vazirani 2004]. Conductance of Graph G vertex set S minimum |S| ≤ n/2 ФG = expansion(S)

  32. A Reverse Reduction Theorem [R-Steurer10] UGC is false  New algorithms to approximate expansion of small sets in graphs Finding Small Non Expanding Sets Suppose there exists is a small community say (0.1% of the population) 99% of whose friends are within the community.. Find one such close-knit community. Graph (Social Network) Close-knit community

  33. STILL OPEN: Reverse reduction from Max Cut or Vertex Cover to Unique Games.

  34. What if UGC is false? Could existing algorithmic techniques (LPs/SDPs) disprove the UGC?

  35. Could LPs/SDPs disprove the UGC?

  36. Question I: Could some small Linear Program give a better approximation for MaxCut or Vertex Cover thereby disproving the UGC? Probably Not! Question II: Could some small SemiDefinite Program give a better approximation for MaxCut or Vertex Cover thereby disproving the UGC? [Charikar-Makarychev-Makarychev][Schoenebeck-Tulsiani] For MaxCut, for several classes of linear programs, exponential sized linear programs are necessary to even beat the trivial ½ approximation! We don’t know.

  37. The Simplest Relaxation for MaxCut Max Cut SDP: Embedd the graph on the N - dimensional unit ball, Maximizing ¼ (Average squared length of the edges) v2 Does adding triangle inequalities improve approximation ratio? (and thereby disprove UGC!) v1 v3 In the integral solution, all the vectors vi are 1,-1. Thus they satisfy additional constraints For example : (vi – vj)2 + (vj – vk)2 ≥ (vi – vk)2 (the triangle inequality) v5 v4

  38. [Arora-Rao-Vazirani 2002] For Sparsest Cut, SDP with triangle inequalities gives approximation. An -approximation would disprove the UGC! [Goemans-Linial Conjecture 1997] SDP with triangle inequalities would yield -approximation for Sparsest Cut. [Khot-Vishnoi 2005] SDP with triangle inequalities DOES NOT give approximation for Sparsest Cut SDP with triangle inequalities DOES NOT beat the Goemans-Williamson 0.878 approximation for Max Cut

  39. Until 2009: Adding a simple constraint on every 5 vectors could yield a better approximation for MaxCut, and disproves UGC! Building on the work of [Khot-Vishnoi], [Khot-Saket 2009][Raghavendra-Steurer 2009] Adding all validlocal constraints on at most vectors to the simple SDP DOES NOT improve the approximation ratio for MaxCut [Barak-Gopalan-Hastad-Meka-Raghavendra-Steurer 2009] Change to in the above result. As of Now: A natural SDP of size (the round of Lasserre hierarchy) could disprove the UGC. [Barak-Brandao-Harrow-Kelner-Steurer-Zhou 2012] round of Laserre hierarchy solves all known instances of Unique Games.

  40. Constraint Satisfaction Problems Max 3 SAT Find an assignment that satisfies the maximum number of clauses. {x1 ,x2 , x3 , x4 , x5} {0,1} Clauses Variables Finite Domain Constraints Kind of constraints permitted Different CSPs

  41. Why play this game? Connections between SDP hierarchies, Spectral Graph Theory and Graph Expansion. New algorithms based on SDP hierarchies. [Raghavendra-Tan] Improved approximation for MaxBisection using SDP hierarchies [Barak-Raghavendra-Steurer] Algorithms for 2-CSPs on low-rank graphs. New Gadgets for Hardness Reductions: [Barak-Gopalan-Hastad-Meka-Raghavendra-Steurer] A more efficient long code gadget. Deeper understanding of the UGC – why it should be true if it is.

More Related