Lectures on NP-hard problems and Approximation algorithms

1 / 41

# Lectures on NP-hard problems and Approximation algorithms - PowerPoint PPT Presentation

Lectures on NP-hard problems and Approximation algorithms. COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski. Overview. Previous lectures: Greedy algorithms Dynamic programming Network flows These lectures: NP-hard problems Approximation algorithms. P versus NP.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Lectures on NP-hard problems and Approximation algorithms' - omer

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Lectures on NP-hard problems andApproximation algorithms

Lecturer: Dariusz Kowalski

NP-hard problems and Approximation algorithms

Overview

Previous lectures:

• Greedy algorithms
• Dynamic programming
• Network flows

These lectures:

• NP-hard problems
• Approximation algorithms

NP-hard problems and Approximation algorithms

P versus NP

Decision problem: problem for which the answer must be either YES or NO

Polynomial time algorithm: there is a constant c such that the algorithm solves the problem in time O(nc) for every input of size n

P (polynomial time) - class of decision problems for which there is a polynomial time deterministic algorithm solving the problem

NP (nondeterministic polynomial time) - class of decision problems for which there is a certifier which can check a witness in polynomial time

NP-hard problems and Approximation algorithms

Certifying in polynomial time

Representation of decision problem: set of inputs which are correct (and should be answered YES, while others should be answered NO)

An efficient certifier B for problem X:

• B is in P
• s is in X iff B(s,t) = YES for some t of size polynomial in s

Example:

Problem: is there a clique of size k in a given graph with n nodes?

Certifier: if a given graph has a clique of size k then given

this clique as the second parameter we can answer YES

NP-hard problems and Approximation algorithms

Computing an efficient certifier

How to compute witnesses for an efficient

certifier for a given problem?

Fact:

If the witnesses for the efficient certifier can be

found in polynomial time then the problem is in P.

Conclusion:

P is included in NP

Open question:

P = NP ?

NP-hard problems and Approximation algorithms

Polynomial reductions

Example: decision problem

if there exists a perfect matching in a bipartite graph

can be reduced to network flow problem in polynomial time (by adding source, target and directing the edges)

Other problems for undirected graphs (in NP and not known to be in P):

• Independent Set of nodes
• Vertex cover
• Set Cover

NP-hard problems and Approximation algorithms

Polynomial reductions

Definition:

Problem X is polynomial-time reducible to problem Y, or

problem Y is at least as hard as problem X, iff

problem X can be solved by an algorithm which works in

polynomial time and uses polynomial number of calls to the

black box solving problem Y.

Notation: XPY

Transitivity Property: if XPY and YPZ then XPZ

NP-hard problems and Approximation algorithms

Independent Set to Vertex Cover

Independent Set: given a graph G ofnnodes and parameter k,

is there a set of k nodes such that none two of them are connected by

an edge?

Vertex Cover: given a graph G ofnnodes and parameter k,

is there a set of k nodes such that every edge has at least one

end selected?

Polynomial Reduction:

• Solve Vertex-Cover(n-k) for the same graph

Proof:

Set S of size n - k is a vertex cover set in G iff

There is no edge between remaining k nodes iff

Set of k remaining nodes is independent in G

NP-hard problems and Approximation algorithms

Vertex Cover to Set Cover

Vertex Cover: given a graph G ofnnodes and parameter k, is there a set of k

nodes such that every edge has at least one end among selected nodes?

Set Cover: given nnodes, m sets which cover the set of nodes, and parameter k,

is there a family of k sets that covers all n nodes?

Polynomial Reduction:

• Let each edge from the graph correspond to the node for SC system
• For each vertex in the graph create a set of incident edges to SC system
• Solve Set-Cover(m,n,k) for the created SC system with m nodes and n sets

Proof:

Each node-edge is covered by at least one set-vertex since each node is covered.

This covering is minimal.

NP-hard problems and Approximation algorithms

NP-completeness and NP-hardness

NP-complete: class of problems X such that every

problem from NP is polynomial-time reducible to X.

Optimization problems: problems where the

answer is a number (maximum/minimum possible)

Each optimization problem has its decision version, e.g.,

• Find a maximum Independent Set
• Is there an Independent Set of size k?

NP-hard: class of optimization problems X such that its

decision version is NP-complete.

Example: having solution for decision version of

Independent Set problem, we can probe a parameter k,

starting from k = 1 , to find the size of the maximum independent set

NP-hard problems and Approximation algorithms

Approximation algorithms

Having an NP-hard problem, we do not know at this

moment any polynomial-time algorithm solving the

problem (exact solution)

How to find an almost optimal solution?

Approximation algorithm with ratio a > 1 gives a solution

A such that

OPT A a OPT for a min-optimization problems

(1/a)  OPT A OPT for a max-optimization problems

where OPT is the optimal solution.

NP-hard problems and Approximation algorithms

2-approximation for VC

Minimum Vertex Cover - NP-hard problem

(maximum is trivially n)

Algorithm:

Initialize set C to an empty set

While there are remaining edges:

• Choose an edge {v,w} with the largest degree, where degree of an edge is a sum of degrees of its ends v,w in a current graph G
• Put v,w to C
• Remove all the edges adjacent to nodes v,w from graph G

Output: witness set C and its size

NP-hard problems and Approximation algorithms

Analysis of 2-approximation for VC

Correctness:

Each edge is removed only after one of its ends is chosen

to set C, so each edge is covered

Termination:

In each iteration we remove at least one edge from the

graph, and there are less than n2 edges

Approximation ratio 2:

For each edge {v,w} selected at the beginning of an iteration at least

one end must be in min-VC, and we selected two, so set C is at most

twice bigger than the min-VC

Time complexity:O(m + n)

Exercise

NP-hard problems and Approximation algorithms

Approximation for SC

Minimum Set Cover - NP-hard problem

(maximum is trivially m)

Greedy Algorithm:

Initialize set C to empty set

While there are uncovered nodes:

• Choose a set F which covers the largest number of uncovered nodes
• Put F to C
• Remove all nodes covered by F

Output: witness set C and its size

NP-hard problems and Approximation algorithms

Analysis of approximation for SC

Correctness:

Each node is marked as covered when we put the set covering it to set C.

The algorithm stops when all nodes are covered.

Termination:

In each iteration we cover at least one new node, and there are n nodes.

Approximation ratio log n:

• Let Si be the set selected to C in ith iteration, and denote by si the number of uncovered nodes covered by Si; OPT be the minimum covering set
• Let cv= 1/ si for each node v which was covered by Si for the first time
• The following holds: |C| = vcv
• For every set S = Si : vScv  H(|S|) (H(i)=ji1/jdenotes harmonic number)
• |C| = vcv  SOPT vScv  H(n) SOPT1 = H(n) |OPT|  |OPT| log n

Time and memory complexities:

O(M + n), where M is the sum of cardinalities of sets

Exercise

NP-hard problems and Approximation algorithms

Conclusions
• Decision problems P and NP-complete
• Polynomial-time reduction
• Optimization problems in NP-hard
• Maximum Independent Set
• Minimum Vertex Cover
• Minimum Set Cover
• Approximation algorithms - polynomial time
• Min-VC with ratio 2
• Min-SC with ratio log n

NP-hard problems and Approximation algorithms

Textbook and Questions

• Chapters 8 and 11, Sections 8.1, 8.2, 8.3, 11.3, 11.4

EXERCISES:

• What is the time and memory complexities of min-VC approximation algorithm with ratio 2 and min-SC algorithm?
• Consider a modification of min-VC algorithm: choose a node which covers the largest number of uncovered edges. Is it a 2-approximation algorithm?
• Having a 2-approximation algorithm for min-VC, is it easy to modify it to be a 2-approximation algorithm for max-IS (since there is a simple polynomial-time reduction between these two problems)?

NP-hard problems and Approximation algorithms

Overview

Previous lectures:

• NP-hard problems and approximation algorithms
• Graph problems (IS, VC)
• Set problem (SC)

This lecture:

• NP-hard numerical problems and their approximation
• Numerical Knapsack problem
• Weighted Independent Set

NP-hard problems and Approximation algorithms

Knapsack problem

Input: set of n items, each represented by its weight wi and value vi ; thresholds W and V

Decision problem: is there a set of items of total weight at most W and total value V ?

Optimization problem: find a set of items with

• total weight at most W , and
• maximum possible value

Assumptions:

• weights and values are positive integers
• each weight is at most W

NP-hard problems and Approximation algorithms

NP-hardness of knapsack

Knapsack is NP-hard problem, but

there exists pseudo-polynomial algorithm (complexity is polynomial in terms of values)

Typical numerical polynomial algorithm: polynomial in logarithm from the maximum values (longest representation)

Existence of pseudo-polynomial solution often yields very good approximation schemes

NP-hard problems and Approximation algorithms

Dynamic pseudo-polynomial optimization algorithm

Let v* be the maximum (integer) value of an item.

Consider any order of objects.

Let OPT(i,v) denote the minimum possible total weight of a subset of items 1,2,…,i which has total value v

Dynamic formula for i = 0,1,…,n-1 and v = 0,1,…,nv* :

OPT(i+1,v) =

= min{ OPT(i,v) , wi+1 + OPT(i,max{0,v-vi+1})}

Formula OPT does not provide direct solution for our problem, but can be easily adapted: maximum value of knapsack is the maximum value v such that OPT(n,v)  W

NP-hard problems and Approximation algorithms

Dynamic algorithm

Initialize array M[0…n,0…nv*] for storing OPT(i,v)

Fill positions M[,0] and M[0,] with zeros

For i = 0,1,…,n-1

For v = 0,1,…,nv*

M[i+1,v] :=

= min{ M[i,v] , wi+1 + M[i,max{0,v-vi+1}] }

Go through the whole array M and find the maximum value v such that M[n,v]  W

NP-hard problems and Approximation algorithms

Complexities

Time: O(n2v*)

• Initializing array M : O(n2v*)
• Iterating loop: O(n2v*)
• Searching for maximum v : O(n2v*)

Memory: O(n2v*)

NP-hard problems and Approximation algorithms

Polynomial approximation algorithm

Algorithm:

• Fix b = (/(2n)) v*
• Set (by rounding up) xi = [vi/b]
• Solve knapsack problem for values xi and weights wi using dynamic program
• Return set of computed items and its total value in terms of the sum of vi’s

NP-hard problems and Approximation algorithms

Analysis

PTAS: polynomial time approximation scheme - for any fixed positive  it produces (1+)-approximation in polynomial time (but  is hidden in big Oh)

Time: O(n2x*) = O(n3/)

Approximation: (1+)

NP-hard problems and Approximation algorithms

Analysis of approximation ratio

Recall notation:

• b = (/(2n)) v*
• xi = [vi/b]

Approximation: (1+)

Let S denote the set of items returned by the algorithm

• vi  bxi  vi + b iSbxi - b|S| iSvi

iSbxi  v* = 2nb/  (2/ -1)nb  iSvi

iOPTvi  iOPTbxi  iSbxi  b|S|+iS (bxi - b)

b(2/ -1)n + iSvi   iSvi + iSvi = (1+) iSvi

NP-hard problems and Approximation algorithms

Weighted Independent Set

Optimization problem:

Weighted Independent Set: given graph G ofnvaluednodes, find an independent set of maximum value (set of nodes such that none two of them are connected by an edge)

Even for values 1 problem remains NP-hard, which is not the case for knapsack problem! WIS problem is an example of strong NP-hard problem, and no PTAS is known for it

NP-hard problems and Approximation algorithms

Conclusions
• Optimization numerical problem in NP-hard
• Maximum Knapsack
• Weighted Independent Set
• PTAS in time O(n3) for Knapsack, based on dynamic programming

NP-hard problems and Approximation algorithms

Textbook and Questions
• Chapters 6 and 11, Sections 6.4, 11.8
• Is it possible to design an efficient Knapsack algorithm based on dynamic programming for the case where weights are small (values can be arbitrarily large)
• How to implement arithmetical operations: + - * / and rounding, each in time proportional to at most square of the length of the longest number? What are the complexity formulas?

NP-hard problems and Approximation algorithms

Overview

Previous lectures:

• NP-hard problems
• Approximation algorithms
• Greedy (VC and SC)
• Dynamic Programming (Knapsack)

This lecture:

• Approximation through integer programming

NP-hard problems and Approximation algorithms

Vertex Cover

Weighted Vertex Cover: (weights are in nodes)

• Decision problem:
• given weighted graph G ofnnodes and parameter k,
• is there a set of nodes with total weight k such that every edge has at least one end in this set?
• Optimization problem:
• given weighted graph G ofnnodes, what is the minimum total weight of a set such that every edge has at least one end in this set?

NP-hard problems and Approximation algorithms

Approximation algorithms

Having an NP-hard problem, we do not know in this

moment any polynomial-time algorithm solving the

problem (exact solution)

How to find almost optimal solution?

Approximation algorithm with ratio a > 1 gives a solution

A such that

OPT  A a OPT for a min-optimization problems

OPT/a A OPT for a max-optimization problems

where OPT is an optimal solution.

NP-hard problems and Approximation algorithms

2-approximation for VC

Minimum Vertex Cover - NP-hard problem even for all weights =1

(maximum is trivially n)

Algorithm: (for all weights equal to 1)

Initialize set C to empty set

While there are remaining edges:

• Choose an edge {v,w} (with the largest degree, where degree of an edge is a sum of degrees of its ends v,w in a current graph G )
• Put v,w to C
• Remove all the edges adjacent to nodes v,w from graph G

Output: witness set C and its size

NP-hard problems and Approximation algorithms

Integer Programming
• Represent the problem as Integer Programming
• Relax the problem to Linear Programming
• Solve Linear Programming
• Round the solution to get integers

NP-hard problems and Approximation algorithms

Integer and linear programs

Set of constraints (linear equations):

x1 , x2 0

x1 + 2x2  6

2x1 + x2  6

Function to minimize (linear):

4x1 + 3x2

Linear programming:

• variables are real numbers
• there are polynomial time algorithms solving it (e.g., interior point method - by N. Karmarkar in 1984); simplex method is not polynomial

Integer programming:

• variables are integers
• problem is NP-hard

NP-hard problems and Approximation algorithms

VC as Integer Program

Set of constraints :

xi {0,1} for every node i

xi + xj  1 for every pair {i,j}  E

Function to minimize:

i xiwi

Example: x1 , x2 , x3 , x4 {0,1}

x1 + x3  1 , x1 + x4  1 , x2 + x4  1 , x2 + x3  1

Minimize:x1 + x2 + x3 + x4

x1

x4

x2

x3

NP-hard problems and Approximation algorithms

Relaxation to Linear Program

Set of constraints :

yi [0,1] for every node i

yi + yj  1 for every pair {i,j}  E

Function to minimize:

i yiwi

Example: y1 , y2 , y3 , y4 [0,1]

y1 + y3  1 , y1 + y4  1 , y2 + y4  1 , y2 + y3  1

Minimize:y1 + y2 + y3 + y4

y1

y4

y2

y3

NP-hard problems and Approximation algorithms

Rounding the linear program solution

Obtained exact Linear Program solution yi [0,1] for every node i

satisfying

yi + yj  1 for every pair {i,j}  E

How to obtain a (approximate?) solution for Integer Program?

Rounding: for every node i

xi =1 iffyi  1/2 (otherwisexi =0)

Example: y1 , y2 , y3 , y4 = 1/2

x1 , x2 , x3 , x4 = 1

Optimum solution (minimum) e.g.:

x1 , x2 = 1, x3 , x4 = 0

x1

x4

x2

x3

NP-hard problems and Approximation algorithms

Analysis

Correctness: since each xi {0,1} and each edge is guarded by constraint xi + xj  1 which is satisfied also after rounding

Time: time for solving linear program plus O(m+n)

Approximation:

Each xi is at most twice as large asyi hence the weighted sum of xi is also at most twice bigger than the weighted sum of yi

Example: y1 , y2 , y3 , y4 = 1/2

x1 , x2 , x3 , x4 = 1

Optimum solution (minimum) e.g.:

x1 , x2 = 1, x3 , x4 = 0

x1

x4

x2

x3

NP-hard problems and Approximation algorithms

Conclusions
• Decision problems P and NP-complete
• Polynomial-time reduction
• Optimization problems in NP-hard
• Maximum Independent Set
• Minimum Vertex Cover
• Minimum Set Cover
• Maximum Knapsack
• Approximation algorithms - polynomial time
• Greedy (VC, SC)
• Dynamic program (Knapsack)
• Integer and Linear programs (weighted VC)

NP-hard problems and Approximation algorithms

Textbook and Questions

Chapter 11, Section 11.6

EXERCISES:

• Could we solve Weighted VC by modification of greedy algorithm solving (pure) VC?
• What approximation we get if we apply randomized rounding, i.e.,

xi =1 withprobability yj(otherwisexi =0)

• Traveling Salesman Problem : Section 8.5
• TSP can not be approximated with a constant unless P=NP
• Constant approximation of TSP problem under the assumption that the weights satisfy metric conditions (symmetric weights satisfying triangle inequality)

NP-hard problems and Approximation algorithms