Approximation Algorithms

Approximation Algorithms

Outlines • Why approximation algorithm? • Approximation ratio • Approximation vertex cover problem • Approximation traveling salesman problem (TSP) • Other interesting problems Approximation Algorithms

3 approaches for NP-complete Problems • For small inputs, an exponential algorithm is OK. • Try to solve important special cases in polynomial time. • Find near-optimal solutions in polynomial time (worst case or on average). • An algorithm that returns near-optimal solutions is called an approximation algorithm. Approximation Algorithms

Approximation Ratio If, for any input of size n for a problem, the cost C of the solution produced by an algorithm is within a factor of ρ(n) of the cost C∗ of an optimal solution (wheremax(C/C∗ , C∗/C) ≤ ρ(n)), we say that, • the algorithm has an approximation ratio of ρ(n)or • the algorithm is a ρ(n)-approximation algorithm. Approximation Algorithms

On Approximation Ratio • maximization problem • 0 < C ≤ C∗ • C∗ /C gives the factor by which the cost of an optimal solution is larger than the cost of the approximate solution. • minimization problem • 0 < C∗ ≤ C • C/C∗ gives the factor by which the cost of the approximate solution is larger than the cost of an optimal solution. • A 1-approximation algorithm produces an optimal solution. • large approximation ratio => a solution can be much worse than optimal. Approximation Algorithms

Polynomial-time approximation algorithms may: • have small constant approximation ratios. • have approximation ratios that grow as functions of the input size n. • achieve increasingly smaller approximation ratios by using more and more computation time. Approximation Algorithms

Approximation scheme • An approximation algorithm that takes as input both • an instance of the problem, and • a value > 0 such that for any fixed , the scheme is a (1 + )-approximation algorithm. Approximation Algorithms

Polynomial-time Approximation Scheme An approximation scheme is • a polynomial-time approximation scheme • if for any fixed > 0, the scheme runs in time polynomial in the size n of its input instance. • a fully polynomial-time approximation scheme • if it is an approximation scheme and its running time is polynomial both in 1/and in the size n of the input instance. • Ex: running time of O((1/)2n3). Approximation Algorithms

Approximate Vertex Cover

Vertex Cover • A vertex cover of an undirected graph G = (V, E) is a subset V’ ⊆V such that if (u, v)is an edge of G, then either u∈V or v∈V. • The size of a vertex cover is the number of vertices in it. • An optimal vertex cover is a vertex cover of minimum size in anundirected graph. • The vertex-cover problem is to find an optimal vertex cover in a given undirected graph. Approximation Algorithms

Example of Vertex Covers A B A B C D E C D E F F Approximation Algorithms

Approximate Vertex Cover AVC(G) C ←  E ← E[G] while E  do Choose an arbitrary edge (u,v) fromE C ← C ∪ {u, v} E=E-{(x, y)| x=u or x=v or y=u or y=v} return C Approximation Algorithms

C ←  E ← E[G] while E  do Choose an arbitrary edge (u,v) fromE C ← C ∪ {u, v} E = E - {(x, y)| x=u or x=v or y=u or y=v} return C Find Approximted Vertex Covers (1) A B C D E F Approximation Algorithms

C ←  E ← set of edges in G while E  do Choose an arbitrary edge (u,v) fromE C ← C ∪ {u, v} E = E - {(x, y)| x=u or x=v or y=u or y=v} return C b c d a e f g Find Approximted Vertex Covers (2) b c d a e f g Approximation Algorithms

Approximation ratio of AVC(1) AVC is a polynomial-time 2-approximation algorithm. Proof From the algorithm, AVC runs in O(V+E), which is polynomial time. Let C* be a set of optimal vertex cover. Let C be the set of vertices returned by AVC. C is a vertex cover because AVC loops until all edges in the graph is removed. Next, we will show that |C|/|C*| ≤ 2. Approximation Algorithms

Approximation ratio of AVC(2) Let A be the set of all edges chosen inthe loop. Since all edges incident on the chosen nodes in C are removed, no two edges in A share an endpoint. An edge is chosen in the loop when neither of its endpoints are already in C. Thus, |C|=2|A|. Approximation Algorithms

Approximation ratio of AVC(3) For any vertex v in C*, there is at least one edge in A with v as an endpoint (because C* is a cover). And, no two edges in A share an endpoint. Thus, |A| ≤ |C*|. Thus, |C|=2|A|≤ 2|C*|. That is, |C|/|C*| ≤ 2. Thus, AVC has approximation ratio of 2. Approximation Algorithms

Traveling Salesman Problem

Traveling salesmanproblem (TSP) • Given a complete undirected graph G=(V, E)that has a nonnegative integer cost c(u, v)associated with each edge (u, v)∈ E, find a hamiltonian cycle (a tour) of G with minimum cost. C(A) =  c(u, v) (u,v)∈A • TSP is an NP-complete problem. Approximation Algorithms

TSP with triangle inequality • The cost function c satisfies the triangle inequality if for all vertices u, v,w ∈ V, • c(u, w)≤ c(u, v)+ c(v, w) . • Example of cost functions that satisfythe triangle inequality • Euclidean distance in the plane. • Even with the triangle inequality, TSP is NP-complete. Approximation Algorithms

Approximation TSP w triangle inequality ATSP(G, c) Select a vertex r∈V[G]to be a “root” vertex Compute a minimum spanning tree T for G from root r using MST-PRIM(G, c, r). Let L be the list of vertices visited in a preorder tree walk of T. Return the hamiltonian cycle H that visits the vertices in the order L. Approximation Algorithms

Example: Approximate TSP (1) From: Cormen, Rivest, Leiserson and Stein, Introduction to Algorithms, MIT Press, 2001. Graph whose cost is the euclidean distance Minimum spanning tree with root a. Approximation Algorithms

Example: Approximate TSP (2) From: Cormen, Rivest, Leiserson and Stein, Introduction to Algorithms, MIT Press, 2001. Preorder traversal and preorder walk Tour obtained from preorder walk Approximation Algorithms

Example: Approximate TSP (3) From: Cormen, Rivest, Leiserson and Stein, Introduction to Algorithms, MIT Press, 2001. Approximate tour Shortest tour Approximation Algorithms

ATSP is 2-approximation algorithm(1) APPROX-TSP-TOUR is a 2-approximation algorithm for the traveling-salesman problem with the triangle inequality. Proof Let H* denote an optimal tour for the given set of vertices. If an edge is removed from H*, the result is a spanning tree, say Ts. Thus, c(Ts)≤ c(H*). Let T be a minimum spanning tree. Then, c(T)≤ c(Ts). Thus, c(T)≤ c(H*). Approximation Algorithms

ATSP is 2-approximation algorithm(2) Let W be a full walk of T. Then, W lists the vertices when they are first visited and also whenever they are returned to after a visit to a subtree. Since the full walk traverses every edge of T exactly twice, c(W)= 2c(T ). From c(T)≤ c(H*) and c(W)= 2c(T ), we get c(W)≤ 2c(H*). Approximation Algorithms

ATSP is 2-approximation algorithm(3) However, W is not a tour, since it visits some vertices more than once. By the triangle inequality, we can delete a visit to any vertex from W and the cost does not increase. (If a vertex v is deleted from W between visits to u and w, the resulting ordering specifies going directly from u to w.) By repeatedly applying this operation, we can remove from W all but the first visit to each vertex. Approximation Algorithms

ATSP is 2-approximation algorithm(4) Let H be the cycle corresponding to this preorder walk. H is a hamiltonian cycle, because every vertex is visited exactly once. Since H is obtained by deleting vertices from the full walk W, c(H)≤ c(W). c(H)≤ 2c(H*) since c(H)≤ c(W) & c(W)≤ 2c(H*). That is, APPROX-TSP-TOUR is a 2-approximation algorithm. Approximation Algorithms

MAX-3-CNF satisfiability A randomized approximation algorithm

MAX-3CNF-Satisfiability • Given a 3-CNF expression , satisfiability, find an assignment of the variables that maximizes the number of clauses evaluating to 1. • According to the definition of 3-CNF satisfiability, we require each clause to consist of exactly three distinct literals. • We further assume that no clause contains both a variable and its negation. • We now show that randomly setting each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm. Approximation Algorithms

Randomized MAX-3-CNF-SAT is 8/7-approximate (1) Given an instance of MAX-3-CNF satisfiability with m clauses and variables x1, x2, . . . , xn, the randomized algorithm that independently sets each variable to 0/1 with prob. 1/2 is a randomized 8/7-approximation algorithm. Proof Suppose that we have independently set each variable to 1 with probability 1/2 and to 0 with probability 1/2. Approximation Algorithms

Randomized MAX-3-CNF-SAT is 8/7-approximate (2) For i = 1, 2, . . . , n, we define the indicator random variable Yi = I {clause i is satisfied} , so that Yi = 1 as long as at least one of the literals in the ith clause has been set to 1. Since no literal appears more than once in the same clause, and since we have assumed that no variable and its negation appear in the same clause, the settings of the three literals in each clause are independent. Approximation Algorithms

Randomized MAX-3-CNF-SAT is 8/7-approximate (3) A clause is not satisfied only if all three of its literals are set to 0, and so Pr {clause i is not satisfied} = (1/ 2)3 = 1/ 8. Thus, Pr {clause i is satisfied} = 1 − 1/ 8 = 7/ 8. Therefore, E [Yi] = 7/ 8. Let Y be the number of satisfied clauses overall, so that Y = Y1 + Y2 +· · · +Ym. Then, we have m E [Y ] = E [ ∑ Yi] i=1 Approximation Algorithms

Randomized MAX-3-CNF-SAT is 8/7-approximate (4) m E [Y ] = E [ ∑ Yi] mi =1 = ∑ E [Yi] (by linearity of expectation) i=1m = ∑ 7/8 i=1 = 7m/8 . Since m is an upper bound on the number of satisfied clauses, the approximation ratio is at most m/(7m/ 8)= 8/ 7. Approximation Algorithms

Approximating weighted vertex cover Using linear programming

Minimum-weight vertex-cover problem • For any vertex cover V′ ⊆ V, we define the weight of the vertex cover w(V′)= ∑ w(v). vV ′ • Given an undirected graph G = (V, E)in which each vertex vV has an associated positive weight w(v), find a vertex cover of minimum weight. • We cannot apply • the algorithm used for unweighted vertex cover, nor • random solution. • Solution: • Use linear programming to compute a lower bound on the weight of the minimum-weight vertex cover. • Round this solution and use it to obtain a vertex cover. Approximation Algorithms

Objective/Constraint for Linear Programming Minimize w(V′)= ∑ w(v) x(v). vV ′ subject to x(u) +x(v)≥ 1 for each v V (every edge must be covered) x(v) ≤ 1 for each v V x(v) ≥ 0 for each vV. (a vertex is either in or not in the cover) Approximation Algorithms

AMINVC AMINVC(G,w) C = Ø computex, an optimal solution to the linear program for each v V do ifx(v)≥ 1/2 then C = C  {v} return C Approximation Algorithms

The approx.-ratio of AMINVC is 2 (1) AMINVC is 2-approximation algorithm for the minimum-weight vertex-cover problem. Proof Let C* be an optimal solution to the minimum-weight vertex-cover problem. Let z* be the value of an optimal solution to the linear program. Since an optimal vertex cover is a feasible solution to the linear program, z* ≤ w(C*). Approximation Algorithms

The approx.-ratio of AMINVC is 2 (2) Next, we claim that by rounding the fractional values of the variablesx(v), we produce a set C that is a vertex cover and satisfies w(C)≤ 2z*. To see that C is a vertex cover, consider any edge (u, v)E. Because x(u)+ x(v)≥ 1, at least one ofx(u)andx(v)is at least 1/2. Then, at least one of u and v will be included in the vertex cover, and so every edge will be covered. Approximation Algorithms

The approx.-ratio of AMINVC is 2 (3) Now we consider the weight of the cover. We have z* = ∑ w(v) ·x(v) vV ≥ ∑ w(v)·x(v) ≥ ∑ w(v)·(1/2) vV:x(v)≥1/2vV:x(v)≥1/2 = ∑ w(v)·(1/2) = (1/2)·∑ w(v) vCvC = w(C)/2 Thus, w(C)≤ 2z* ≤ 2w(C*) AMINVC is a 2-approximation algorithm. Approximation Algorithms

Subset Sum Fully polynomial-time approximate scheme

Exponential-time Exact Subset Sum EXACT-SUBSET-SUM(S, t) n ← |S| L0 ← 0 for i ← 1 to n do Li← MERGE-LISTS(Li−1, Li−1 + xi) remove from Lievery element a, a>t return the largest element in Ln O(2n) Max. size of Liis 2n. Approximation Algorithms

Let S = {1, 4, 5} and t =8. L0 = 0 L1 = 0, 1, L2 = 0, 1, 4, 5, L3 = 0, 1, 4, 5, 6, 9, 10 . n ← |S| L0 ← 0 for i ← 1 to n do Li← MERGE(Li−1,Li−1+xi) Li+1= Li– {a|aLi, a>t} return the largest element in Ln Example Approximation Algorithms

fully polynomial-time approximation scheme • Approximation by trimming each list Liafter it is created. • If two values in L are close to each other, then one of them can be removed. • Let δ, 0 < δ < 1, be a trimming parameter. • x approximatesy with trimming parameter δ if y/(1 + δ)≤ x ≤ y . • To trim a list L by δ • remove an element z from L if there is an element x in L′ which approximates y. Approximation Algorithms

Trimming a list: Example Let δ = 0.1 and L = 10, 11, 12, 15, 20, 21, 22, 23, 24, 29. To trim L, we obtain L′ = 10, 12, 15, 20, 23, 29. 11 is represented by 10 21 and 22 are represented by 20 24 is represented by 23. Approximation Algorithms

APPROX-SUBSET-SUM APPROX-SUBSET-SUM(S, t, ) n = |S| L0 = 0 for i = 1 to n do Li= MERGE-LISTS(Li−1, Li−1 + xi) Li= TRIM(Li, /2n) Li+1= Li– {a| aLi and a>t } Find the largest value, z*, in Ln return z* Approximation Algorithms

Approximation Algorithms