Loading in 2 Seconds...

Optimal Algorithms and Inapproximability Results for Every CSP?

Loading in 2 Seconds...

- By
**loan** - Follow User

- 128 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Optimal Algorithms and Inapproximability Results for Every CSP?' - loan

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Optimal Algorithms and Inapproximability Results for Every CSP?

### Rounding Scheme(For Boolean CSPs)

### Rounding Scheme(For Boolean CSPs)

Prasad Raghavendra

University of Washington

Seattle

Constraint Satisfaction ProblemA Classic Example : Max-3-SAT

Given a 3-SAT formula,

Find an assignment to the variables that satisfies the maximum number of clauses.

Equivalently the largest fraction of clauses

Constraint Satisfaction Problem

Instance :

- Set of variables.
- Predicates Pi applied on variables

Find an assignment that satisfies the largest fraction of constraints.

Problem :

Domain : {0,1,.. q-1}

Predicates : {P1, P2 , P3 … Pr}

Pi : [q]k -> {0,1}

Max-3-SAT

Domain : {0,1}

Predicates :

P1(x,y,z) = x ѵ y ѵz

Variables :

{x1 , x2 , x3 ,x4 , x5}

Constraints : 4 clauses

Generalized CSP (GCSP)

Replace Predicates by Payoff Functions

(bounded real valued)

Problem :

Domain : {0,1,.. q-1}

Pay Offs:{P1, P2 , P3 … Pr}

Pi : [q]k -> [-1, 1]

Objective :

Find an assignment that maximizes the Average Payoff

Pay Off Functions can be Negative

Can model Minimization Problems like Multiway Cut, Min-Uncut.

Examples of GCSPs

Max-3-SAT

Max Cut

Max Di Cut

Multiway Cut

Metric Labelling

0-Extension

Unique Games

d- to - 1 Games

Label Cover

Horn Sat

x-z = 13 (mod 17)

…

….

z-w = 15(mod 17)

Unique GamesA Special CaseE2LIN mod p

Given a set of linear equations of the form:

Xi – Xj = cij mod p

Find a solution that satisfies the maximum number of equations.

Unique Games Conjecture[Khot 02]An Equivalent Version[Khot-Kindler-Mossel-O’Donnell]

For every ε> 0, the following problem is

NP-hard for large enough prime p

Given a E2LIN mod p system, distinguish between:

- There is an assignment satisfying 1-εfraction of the equations.
- No assignment satisfies more than εfraction of equations.

Unique Games Conjecture

A notorious open problem, no general consensus either way.

Hardness Results:

No constant factor approximation for unique games. [Feige-Reichman]

Why is UGC important?

UG hardness results are intimately connected to the limitations of Semidefinite Programming

Max Cut SDP

Quadratic Program

Variables : x1 , x2 … xn

xi = 1 or -1

Maximize

Semidefinite Program

Variables : v1 , v2 … vn

- | vi |2= 1

Maximize

-1

10

1

-1

15

1

7

1

1

1

-1

-1

-1

3

-1

Relax all the xi to be unit vectors instead of {1,-1}.

All products are replaced by inner products of vectors

MaxCut Rounding

v2

Cut the sphere by a random hyperplane, and output the induced graph cut.

- A 0.878 approximation for the problem.

v1

v3

v5

v4

2-CSP over {0,..q-1}

Total PayOff

Arbitrary k-ary GCSP

- SDP is similar to the one used by [Karloff-Zwick]Max-3-SAT algorithm.
- It is weaker than k-rounds of Lasserre / LS+ heirarchies

Two Curves

Integrality Gap Curve

S(c) = smallest value of the integral solution, given SDP value c.

If UGC is true:

U(c) ≥ S(c)

If UGC is false:

U(c) is meaningless!

UGC Hardness Curve

U(c) = The best polytime computable solution, assuming UGC given an instance with value c.

U(c)

S(c)

0

Optimum Solution

1

S(c) = SDP Integrality Gap

U(c) = UGC Hardness Curve

UG Hardness ResultTheorem 1:

For every constant η > 0, and every GCSP Problem,

U(c) < S(c+ η) + η

Roughly speaking,

Assuming UGC, the SDP(I), SDP(II),SDP(III) give best possible approximation for every CSP

U(c)

U(c)

S(c)

0

Optimum Solution

1

Consequences

If UGC is true, then adding more constraints does not help for any CSP

Lovasz-Schriver, Lasserre, Sherali-Adams heirarchies do not yield better approximation ratios for any CSP in the worst case.

S(c) = SDP Integrality Gap

U(c) = UGC Hardness Curve

Efficient Rounding SchemeRoughly speaking,

There is a generic polytime rounding scheme that is optimal for every CSP, assuming UGC.

U(c)

S(c)

0

Optimum Solution

1

Theorem:

For every constant η > 0, and every GCSP,

there is a polytime rounding scheme that outputs a solution of value U(c-η) – η

If UGC is true, then for every Generalized Constraint Satisfaction Problem :

- If UGC is false??
- Hardness result doesn’t make sense.
- How good is the rounding scheme?

NP-hard

U(c)

S(c)

algorithm

0

Optimum Solution

1

Unconditionally

Roughly Speaking,

For 2-CSPs, the Approximation ratio obtained is at least the red curve S(c)

The rounding scheme achieves the integrality gap of SDP for 2-CSPs (both binary and q-ary cases)

Theorem:

Let A(c) be rounding scheme’s performance on input with SDP value = c. For every constant η > 0

A(c) > S(c- η) - η

S(c)

0

Optimum Solution

1

As good as the best

SDP(II) and SDP(III) are the strongest SDPs used in approximation algorithms for 2-CSPs

The Generic Algorithm is at least as good as the best known algorithms for 2-CSPs

Examples:

Max Cut [Goemans-Williamson]

Max-2-SAT [Lewin-Livnat-Zwick]

Unique Games [Charikar-Makarychev-Makarychev]

Computing Integrality Gaps

Theorem:

For any η, and any 2-CSP, the curve S(c) can be computed within error η.

(Time taken depends on ηand domain size q)

S(c)

Explicit bounds on the size of an integrality gap instance for any 2-CSP.

0

Optimum Solution

1

Related Work

[Austrin 07]

Assuming UGC, and a certain additional conjecture:

``For every boolean 2-CSP, the best approximation is given by SDP(III)”

[O’Donnell-Wu 08]

Obtain matching approximation algorithm, UGC hardness and SDP gaps for MaxCut

- F : {-1,1}R {-1,1}
- Toss random coins
- Make a few queries to F
- Output either ACCEPT or REJECT

F is a dictator function

F(x1 ,… xR) = xi

F is far from every dictator function

(No influential coordinate)

Pr[ACCEPT ] = Completeness

Pr[ACCEPT ] =

Soundness

Connections

SDP Gap Instance

SDP = 0.9

OPT = 0.7

[Khot-Vishnoi]

For sparsest cut, max cut.

[This Paper]

UG Hardness

0.9 vs 0.7

Dictatorship Test

Completeness = 0.9

Soundness = 0.7

[Khot-Kindler-Mossel-O’Donnell]

All these conversions hold for every GCSP

A Dictatorship Test for Maxcut

A dictatorship test is a graph G on the hypercube.

A cut gives a function F on the hypercube

Completeness

Value of Dictator Cuts

F(x) = xi

Soundness

The maximum value attained by a cut far from a dictator

Hypercube = {-1,1}100

An Example : Maxcut

Completeness

Value of Dictator Cuts =

SDP Value (G)

Graph G

SDP Solution

v2

10

Soundness

Given a cut far from every dictator :

It gives a cut on graph G with the same value.

In other words,

Soundness ≤ OPT(G)

15

7

v1

v3

1

1

3

v5

100 dimensional hypercube

v4

From Graphs to Tests

Graph G (n vertices)

v2

H

10

15

Constant independent of size of G

7

v1

v3

1

1

3

100 dimensional hypercube : {-1,1}100

For each edge e, connect every pair of vertices in hypercube separated by the length of e

v5

v4

SDP Solution

Completeness

For each edge e, connect every pair of vertices in hypercube separated by the length of e

Echoice ofedge e=(u,v)inG

[EX,Y in 100 dimhypercube with dist |u-v|^2 [ (F(X)-F(Y))2 ] ]

v2

v1

-1

v3

Set F(X) = X1

-1

-1

(X1 – Y1)2

X1 is not equal to Y1 with probability |u-v|2, hence completeness = SDP Value (G)

1

v5

1

1

v4

100 dimensional hypercube

The Invariance Principle

Invariance Principle for Low Degree Polynomials

[Rotar] [Mossel-O’Donnell-Oleszkiewich], [Mossel 2008]

“If a low degree polynomial F has no influential coordinate, then F({-1,1}n) and F(Gaussian) have similar distribution.”

A generalization of the following fact :

``Sum of large number of {-1,1} random variables

has similar distribution as

Sum of large number of Gaussian random variables.”

From Hypercube to the Sphere

100

Dimensional hypercube

[-1,1]

F :

Express F as a multilinear polynomial using Fourier expansion, thus extending it to the sphere.

100 dimensional sphere

P :

Real numbers

Nearly always

[-1,1]

Since F is far from a dictator, by invariance principle, its behaviour on the sphere is similar to its behaviour on hypercube.

A Graph on the Sphere

Graph G (n vertices)

v2

S

10

15

7

v1

v3

1

1

3

100dimensional sphere

For each edge e, connect every pair of vertices in sphere separated by the length of e

v5

v4

SDP Solution

Hypercube vs Sphere

S

H

P : sphere -> Nearly {-1,1}

Is the multilinear extension of F

F:{-1,1}100 -> {-1,1}

is a cut far from every dictator.

By Invariance Principle,

MaxCut value of F on H≈Maxcut value of P on S.

Soundness

For each edgee in the graph Gconnect every pair of vertices in hypercube separated by the length of e

G

v2

S

v1

v3

Alternatively, generate S as follows:

Take the union of all possible rotations of the graph G

S consists of union of disjoint copies of G. Thus,

MaxCut Value of S < Max cut value of G.

v5

Hence MaxCut value of F on H is at most the max cut value of G.

Soundness ≤ MaxCut(G)

v4

Algorithmically,

Given a cut F of the hypercube graph H

- Extend F to a function P on the sphere using its Fourier expansion.
- Pick a random rotation of the SDP solution to the graph G
- This gives a random copy Gc of G inside the sphere graph S
- Output the solution assigned by P to GC

1) Tests of the verifier are same as the constraints in instance G

2) Completeness = SDP(G)

Key LemmaDICTG

Dictatorship Test

on functions

F : {-1,1}n ->{-1,1}

Any

CSP Instance G

RoundF

Rounding Scheme

on CSP Instances G

Any

Function

F: {-1,1}n→ {-1,1}

If F is far from a dictator,

RoundF (G) ≈ DICTG (F)

UG Hardness Result

Instance

SDP = c

OPT = s

Worst Case Gap Instance

Dictatorship Test

Completeness = c

Soundness <= s

UG Hardness

Completeness = c

Soundness <= s

Theorem 1:

For every constant η > 0, and every GCSP Problem,

U(c) < S(c+ η) + η

Generic Rounding Scheme

Solve SDP(III) to obtain vectors (v1 ,v2 ,… vn)

Add little noise to SDP solution (v1 ,v2 ,… vn)

For all multlinear polynomials P(y1 ,y2, .. y100) do

Round using P(y1 ,y2, .. y100)

Output the best solution obtained

P is Multilinear polynomial in 100 variables with coefficients in [-1,1]

Soundness of any Dictatorship Test

≥ U(c)

AlgorithmAny

Dictatorship Test

Completeness = c

Dictatorship Test (I)

Completeness = c

UG Hardness

Completeness = c

Instance I

SDP = c

OPT = ?

There is some function F : {0,1}R -> {0,1} that has

Pr[F is accepted] ≥ U(c)

By Key Lemma, Performance of F as rounding polynomial on instance I = Pr[F is accepted] > U(c)

Related Developments

- Multiway Cut and Metric Labelling problems.
- Maximum Acyclic Subgraph problem
- Bipartite Quadratic Optimization Problem (Computing the Grothendieck constant)

[Manokaran, Naor, Schwartz, Raghavendra]

[Guruswami,Manokaran, Raghavendra]

[Raghavendra,Steurer]

Conclusions

Unique Games and Invariance Principle connect :

Integrality Gaps, Hardness Results ,Dictatorship tests and Rounding Algorithms.

These connections lead to new algorithms, and hardness results unifying several known results.

Rounding Scheme was discovered by the reversing the soundness analysis.

This fact was independently observed by Yi Wu

MaxCut Rounding

- Cut the sphere by a random hyperplane, and output the induced graph cut.
- Equivalently,
- Pick a random direction g.
- For each vector vi , project vi along g
- yi = vi . g
- Assign
- xi = 1 if yi > 0
- xi = 0 otherwise.

v2

v1

v3

v5

v4

SDP Rounding Schemes

For any CSP, it is enough to do the following:

Instead of one random projection, pick sufficiently many (say 100) projections

Use a multi linear polynomial P to process the projections

SDP Vectors

(v1 , v2 .. vn )

Random Projection

Projections

(y1 , y2 .. yn )

Process the projections

Assignment

UG Hardness Results

Instance

SDP = c

OPT = s

Worst Case Gap Instance

Dictatorship Test

Completeness = c

Soundness <= s

UG Hardness

Completeness = c

Soundness <= s

Theorem 1:

For every constant η > 0, and every GCSP Problem,

U(c) < S(c+ η) + η

Multiway Cut and Labelling Problems

[Manokaran, Naor, Schwartz, Raghavendra]

Theorem:

Assuming Unique Games Conjecture,

The earthmover linear program gives the best approximation.

10

15

7

1

1

3

Theorem:

Unconditionally, the simple SDP does not give better approximations than the LP.

3-Way Cut:

Separate the 3-terminals while separating the minimum number of edges

Maximum Acyclic Subgraph

[Guruswami,Manokaran, Raghavendra]

Given a directed graph, order the vertices to maximize the number of forward edges.

Theorem:

Assuming Unique Games Conjecture,

The best algorithm’s output is as good as a random ordering.

Theorem:

Unconditionally, the simple SDP does not give better approximations than random.

The Grothendieck Constant

[Raghavendra,Steurer]

The Grothendieck constant is the smallest constant k(H) for which the following inequality holds for all matrices :

The constant is just the integrality gap of the SDP for bipartite quadratic optimization.

Value of the constant is between 1.6 and 1.7 but is unknown yet.

Grothendieck Constant

[Raghavendra,Steurer]

Theorem:

There is an algorithm to compute arbitrarily good approximations to the Grothendieck constant.

Theorem:

There is an efficient algorithm that solves the bipartite quadratic optimization problem to an approximation equal to Grothendieck constant.

If all this looks deceptively simple,

then it is because there was deception

Working with several probability distributions at once.

UG Hardness Results

Best UG Hardness

=

Integrality Gap

U(c) < S(c+η) + η

Instance

SDP = c

OPT = s

Worst Case Gap Instance

Dictatorship Test

Completeness = c

Soundness <= s

UG Hardness

Completeness = c

Soundness <= s

Soundness of any Dictatorship Test

≥ U(c)

AlgorithmAny

Dictatorship Test

Completeness = c

Dictatorship Test (I)

Completeness = c

UG Hardness

Completeness = c

Instance I

SDP = c

OPT = ?

There is some function F : {0,1}R -> {0,1} that has

Pr[F is accepted] ≥ U(c)

By Key Lemma, Performance of F as rounding polynomial on instance I = Pr[F is accepted] > U(c)

Unconditional Results For 2-CSPs

On some instance I with SDP value = c , algorithm outputs a solution with value s.

For every function F far from dictator ,

Performance of F in rounding I ≤ s

By Key Lemma, For every such F

Pr[ F is accepted by Dict(I) ] ≤ s

Thus the Dict(I) is a test with soundness s.

Unconditional Results For 2-CSPs

Integrality Gap instance

SDP = c

OPT ≤ s

Algorithm’s performance matches the integrality gap of the SDP

UG Hardness

Completeness = c

Soundness = s

Dictatorship Test(I)

Completeness = c

Soundness = s

[Khot-Vishnoi]

UG Integrality Gap instance

Computing Integrality Gaps

Integrality gap of a SDP relaxation

= Worst case ratio of Integral Optimum

SDP Optimum

Worst Case over all instances - an infinite set

Due to tight relation of integrality gaps/ dictatorship tests for 2-CSPs

Integrality gap of a SDP relaxation

= Worst case ratio of Soundness

Completeness

This time the worst case is along all dictatorship tests on {-1,1}R

- a finite set that can be discretized.

v1 , v2 , v3 = SDP Vectors

Local Random VariablesFix an edge e = (1,2).

There exists random variables a1 a2taking values {-1,1} such that:

1

A13

A12

A23

3

2

For every edge, there is a local distribution over integral solutions such that:

All the moments of order at most 2 match the inner products.

E[a1 a2] = v1∙ v2

E[a12] = |v1|2 E[a22] = |v2|2

v1 , v2 , v3 = SDP Vectors

A12,A23,A31 = Local Distributions

Dictatorship TestPick an edge (i,j)

Generate ai,aj in {-1,1}Ras follows:

The kth coordinates aik ,ajkcome from distribution Aij

Add noise toai,aj

Accept if

F(ai) ≠ F(aj)

Input Function:

F : {-1,1}R -> {-1,1}

1

Max Cut Instance

A12

2

3

A12,A23,A31 = Local Distributions

AnalysisMax Cut Instance

Pick an edge (i,j)

Generate ai,aj in {-1,1}Ras follows:

The kth coordinates aik,ajkcome from distribution Aij

Add noise toai,aj

Accept if

F(ai) ≠ F(aj)

1

2

3

Input Function:

F : {-1,1}R -> {-1,1}

A12,A23,A31 = Local Distributions

CompletenessInput Function is a Dictator : F(x) = x1

Suppose (a1 ,a2) is sampled from A12 then :

E[a11 a21] = v1∙ v2 E[a112] = |v1|2 E[a212] = |v2|2

Summing up, Pr[Accept] = SDP Value(v1 , v2 ,v3)

v1 , v2 , v3 = SDP Vectors

Global Random Variablesg = random Gaussian vector.

(each coordinate generated by i.i.d normal variable)

b1 = v1 ∙ g

b2 = v2 ∙ g

b3 = v3 ∙ g

1

B

3

2

There is a global distribution B=(b1 ,b2 ,b3) over real numbers such that:

All the moments of order at most 2 match the inner products.

E[b1 b2] = v1∙ v2 E[b2 b3] = v2∙ v3 E[b3 b1] = v3∙ v1

E[b12] = |v1|2 E[b22] = |v2|2 E[b32] = |v3|2

Rounding with Polynomials

1

Input Polynomial : F(x1 ,x2 ,.. xR)

Generate

b1 = (b11 ,b12 ,… b1R)

b2 = (b21 ,b22 ,… b2R)

b3 = (b31 ,b32 ,… b3R)

with each coordinate (b1t ,b2t ,b3t) according to global distribution B

Compute F(b1),F(b2) ,F(b3)

Round F(b1),F(b2),F(b3) to {-1,1}

Output the rounded solution.

B

3

2

Invariance

- Suppose F is far from every dictator then since A12 and B have same first two moments,
- F(a1),F(a2) has nearly same distribution as F(b1),F(b2)
- F(b1), F(b2) are close to {-1,1}

From Gap instances to Gap instances

Instance

SDP = c

OPT = s

A Gap Instance for the Strong SDP for

CSP

Dictatorship Test

Completeness = c

Soundness = s

UG Hardness

Completeness = c

Soundness = s

UG Gap instance for a

Strong SDP

For each variable u in CSP,

Introduce q variables : {u0 , u1 ,.. uq-1 }

uc = 1,

ui = 0 for i≠c

Payoff foru,v :

P(u,v) = ∑a ∑b P(a,b)uavb

u = c

2-CSP over {0,..q-1}

Total PayOff

Arbitrary k-ary GCSP

SDP is similar to the one obtained by k-rounds of Lasserre

Rounding Scheme was discovered by the reversing the soundness analysis.

This fact was independently observed by Yi Wu

SDP Rounding Schemes

For any CSP, it is enough to do the following:

Instead of one random projection, pick sufficiently many projections

Use a multilinear polynomial P to process the projections

SDP Vectors

(v1 , v2 .. vn )

Random Projection

Projections

(y1 , y2 .. yn )

Process the projections

Assignment

Algorithm

Solve SDP(III) to obtain vectors (v1 ,v2 ,… vn)

Smoothen the SDP solution (v1 ,v2 ,… vn)

For all multlinear polynomials P(y1 ,y2, .. yR) do

Round using P(y1 ,y2, .. yR)

Output the best solution obtained

“For all multilinear polynomials P(y1 ,y2, .. yR) do”

- All multilinear polynomials with coefficients bounded within [-1,1]

- Discretize the set of all such multi-linear polynomials

There are at most a constant number of such polynomials.

Smoothening SDP Vectors

Let u1 ,u2 .. un denote the SDP vectors corresponding to the following distribution over integral solutions:

``Assign each variable uniformly and independently at random”

Substitute

vi* ∙ vj* = (1-ε) (vi ∙ vj) + ε (ui∙ uj)

Non-Boolean CSPs

There will be q rounding polynomials instead of one polynomial.

Projection is in the same fashion:

Yi(j) = v0∙vi + (1-ε)(vi – (v0∙vi)v0) ∙ w(j)

To Round the Output of the polynomial, do the following:

From Gap instances to Gap instances

Instance

SDP = c

OPT = s

A Gap Instance for the Strong SDP for

CSP

Worst Case Instance

Dictatorship Test

Completeness = c

Soundness = s

UG Hardness

Completeness = c

Soundness = s

UG Gap instance for a

Strong SDP

Remarks

For every CSP and every ε > 0, there is a large enough constant R such that

- Approximation achieved is within ε ofoptimal for all CSPs if Unique Games Conjecture is true.
- For 2-CSPs, the approximation ratio is within ε of the integrality gap of the SDP(I).

Rounding Schemes

Very different rounding schemes for every CSP.

with often complex analysis.

Max Cut - Random hyperplane cutting

Multiway cut - Complicated Cutting the simplex.

- Our algorithm is a generic rounding procedure.
- Analysis does not compute the approximation factor, but indirectly shows that it is equal to the integrality gap.

“Sample R independent vectors : w1, w2 ,.. wReach with i.i.d Gaussian components.

For all multlinear polynomials P(y1 ,y2, .. yR) do

Compute xi = P(vi∙ w1 , vi∙ w2 ,.. vi∙ wR)”

Goemans-Williamson rounding uses one single random projection, this algorithm uses a constant number of random projections.

Semidefinite Programming

- Linear program over the inner products
- Strongest algorithmic tool in approximation algorithms
- Used in a large number of algorithms.

Integrality gap of a SDP relaxation =

Worst case ratio of Integral Optimum

SDP Optimum

More Constraints?

Most SDP algorithms use simple relaxations with few constraints.

[Arora-Rao-Vazirani] used the triangle inequalities to get sqrt(log n) approximation for sparsest cut.

Can the stronger SDPs yield better approximation ratios for problems of interest?

Max Cut SDP

Quadratic Program

Variables : x1 , x2 … xn

xi = 1 or -1

Maximize

Semidefinite Program

Variables : v1 , v2 … vn

- | vi |2= 1

Maximize

-1

10

1

-1

15

1

7

1

1

1

-1

-1

-1

3

-1

Relax all the xi to be unit vectors instead of {1,-1}.

All products are replaced by inner products of vectors

Download Presentation

Connecting to Server..