1 / 58

# Design and Analysis of Computer Algorithm Lecture 10 - PowerPoint PPT Presentation

Design and Analysis of Computer Algorithm Lecture 10. Pradondet Nilagupta Department of Computer Engineering. Acknowledgement.

Related searches for Design and Analysis of Computer Algorithm Lecture 10

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Design and Analysis of Computer Algorithm Lecture 10' - shawn

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Design and Analysis of Computer AlgorithmLecture 10

Department of Computer Engineering

• This lecture note has been summarized from lecture note on Data Structure and Algorithm, Design and Analysis of Computer Algorithm all over the world. I can’t remember where those slide come from. However, I’d like to thank all professors who create such a good work on those lecture notes. Without those lectures, this slide can’t be finished.

• Tractable and intractable problems

• NP-complete problems

• Here: Classify problems as tractable or intractable.

• Problem is tractable if there exists a polynomial bound algorithm that solves it.

• An algorithm is polynomial bound if its worst case growth rate can be bound by a polynomial p(n) in the size n of the problem

• Standard working definition: polynomial time

• On an input of size n the worst-case running time is O(nk) for some constant k

• Polynomial time: O(n2), O(n3), O(1), O(n lg n)

• Not in polynomial time: O(2n), O(nn), O(n!)

• Are some problems solvable in polynomial time?

• Of course: every algorithm we’ve studied provides polynomial-time solution to some problem

• We define P to be the class of problems solvable in polynomial time

• Are all problems solvable in polynomial time?

• No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given

• Such problems are clearly intractable, not in P

• Problem is intractable if it is not tractable.

• Any algorithm that solves it is not polynomial bound.

• It has a worst case growth rate f(n) which cannot be bound by a polynomial p(n) in the size n of the problem.

• For intractable problems the bounds are:

• If problem is intractable, no point in trying to find an efficient algorithm

• Any algorithm too slow for large inputs.

• To solve use approximations, heuristics, etc.

• Sometimes we need to solve only a restricted version of the problem.

• If restricted problem tractable design an algorithm for restricted problem

• Turing showed some problems so hard that no algorithm can solve them (undecidable)

• Other researchers showed some decidable problems from automata, mathematical logic, etc. are intractable

• These problems are so hard that they cannot be solved in polynomial time by a “nondeterministic” computer

• Many practical problems for which no one has yet found a polynomial bound algorithm.

• Examples: traveling salesperson, knapsack, graph coloring, etc.

• Most design automation problems such as testing and routing.

• Many networks, database and graph problems.

• A variety of algorithms based on backtracking, branch and bound, etc.

• None can be shown to be polynomial bound

• Problems can be solved by a polynomial bound verification algorithm

• The theory of NP-completeness enables showing that these problems are at least as hard as NP-complete problems

• Practical implication of knowing problem is NP-complete is that it is probably intractable ( whether it is or not has not been proved yet)

• So any algorithm that solves it will probably be very slow for large inputs

• Decision problems

• The class P

• Nondeterministic algorithms

• The class NP

• The concept of polynomial transformations

• The class of NP-complete problems

• Decision problems

• Converting optimization problems into decision problems

• The relationship between an optimization problem and its decision version

• The class P

• Verification algorithms

• The class NP

• A decision problem answers yes or no for a given input

• Examples:

• Is there a path from s to t of length at most k?

• Does graph G contain a Hamiltonian cycle?

• A hamiltonian cycle of a graph G is a cycle that includes each vertex of the graph exactly once.

• Problem: Given a graph G, does G have a hamiltonian cycle?

• Optimization problems can be converted to decision problems (typically) by adding a bound B on the value to optimize, and asking the question:

• Is there a solution whose value is at most B? (for a minimization problem)

• Is there a solution whose value is at least B? (for a maximization problem)

• Given:

• A finite set C={c1,...,cm} of cities,

• A distance function d(ci, cj) of nonnegative numbers.

• Find the length of the minimum distance tour which includes every city exactly once

• Given a finite set C={c1,...,cm} of cities, a distance function d(ci, cj) of nonnegative numbers and a bound B

• Is there a tour of all the cities (in which each city is visited exactly once) with total length at most B?

• If we have a solution to the optimization problem we can compare the solution to the bound and answer “yes” or “no”.

• Therefore if the optimization problem is tractable so is the decision problem

• If the decision problem is “hard” the optimization problems are also “hard”

• Definition: The class P

• P is the class of decision problems that are polynomially bounded.

• there exist a deterministic algorithm

• Definition: The class NP

• NP is the class of decision problems for which there is a polynomially bounded non-deterministic algorithm.

• The name NP comes from “Non-deterministic Polynomially bounded.”

• there exist a non-deterministic algorithm

• Theorem: P NP

• The goal of a verification algorithm is to verify a “yes” answer to a decision problem’s input.

• The inputs to the verification algorithm are the original input and a certificate (possible solution)

• A verification algorithm for TS, verifies that a given TS tour has length at most B

• Given the problem PATH (does there exist a path of length k or less in a graph G between vertices u and v?), and a certificate p.

• It is simple to verify that the length of p is at most k (we have to also check that p is indeed a path from u to v).

• Other problems like HAMILTONIAN CYCLE are not known to have polynomial bound algorithms but given a hamiltonian cycle, it is easy to verify that the cycle is indeed hamiltonian in polynomial time.

• A verification algorithm, takes a problem instance x and verifies it, if there exists a certificate y such that the answer for x with certificate y is “yes”

• Given a decision problem d.

• A verification algorithm for d is polynomial bound if given an input x to d,

there exists a certificate y, such that |y|=O(|x|c) where c is a constant,

and a polynomial bound algorithm A(x, y) that verifies an answer “yes” for d with input x

• NP is the class of decision problems for which there is a polynomial bounded verification algorithm

• It can be shown that:

• all decision problems in P, and

• decision problems such as traveling salesman and knapsack are also in NP

• The non-deterministic “guessing” phase.

• Some completely arbitrary string s, “proposed solution”

• each time the algorithm is run the string may differ

• The deterministic “verifying” phase.

• a deterministic algorithm takes the input of the problem and the proposed solution s, and

• return value true or false

• The output step.

• If the verifying phase returned true, the algorithm outputs yes. Otherwise, there is no output.

• Summary so far:

• P = problems that can be solved in polynomial time

• NP = problems for which a solution can be verified in polynomial time

• Unknown whether P = NP (most suspect not)

• Hamiltonian-cycle problem is in NP:

• Cannot solve in polynomial time

• Easy to verify solution in polynomial time (How?)

• Can solve variant of TSP which is in form of a decision problem

• TSP*: Given a complete directed graph G with cost for each edge, and an integer k. Return YES, if there is a tour with total distance  k; NO otherwise

• Can be solved in polynomial time with nondeterministic computer

• How?

• Cannot be converted to polynomial time algorithm for regular computer

• Why?

• We will see that NP-Complete problems are the “hardest” problems in NP:

• If any one NP-Complete problem can be solved in polynomial time…

• …then every NP-Complete problem can be solved in polynomial time…

• …and in fact every problem in NP can be solved in polynomial time (which would show P = NP)

• Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

• A problem Q is NP-complete

• if it is in NP and

• it is NP-hard.

• A problem Q is NP-hard

• if every problem in NP

• is reducible to Q.

Theorem:

Let A be in NP-Complete, and B is in NP.

If A PB, then B is also NP-complete.

• A problem P is reducible to a problem Q if

• there exists a polynomial reduction function T such that

• For every string x,

• if x is a yes input for P, then T(x) is a yes input for Q

• if x is a no input for P, then T(x) is a no input for Q.

• T can be computed in polynomially bounded time.

• Problems that are solvable in polynomial time on a regular computer are said to be in class P

• All problems in P are solvable in p-time on nondeterministic computer

• Some problems in NP are NP-complete

• e.g., Clique problem for undirected graphs

• All problems solvable in exponential time is an even bigger class

• Note that all problems solvable in p-time are certainly solvable in exponential time

Exponential time problems

NP problems

TOH

NP-Complete problems

TSP*

P problems

SORTING

T(x)

Algorithm

for Q

T

Yes or no

x

(an input

For P)

An input

for Q

• Problem P is reducible to Q

• P p Q

• Transforming inputs of P to inputs of Q

• Reducibility relation is transitive.

• The crux of NP-Completeness is reducibility

• Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P

• What do you suppose “easily” means?

• This rephrasing is called transformation

• Intuitively: If P reduces to Q, P is “no harder to solve” than Q

• An example:

• P: Given a set of Booleans, is at least one TRUE?

• Q: Given a set of integers, is their sum positive?

• Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

• Another example:

• Solving linear equations is reducible to solving quadratic equations

• How can we easily use a quadratic-equation solver to solve linear equations?

• If P is polynomial-time reducible to Q, we denote this P p Q

• Definition of NP-Hard and NP-Complete:

• If all problems R  NP are reducible to P, then P is NP-Hard

• We say P is NP-Complete if P is NP-Hard and P  NP

• If P p Q and P is NP-Complete, Q is alsoNP - Complete

• This is the key idea you should take away today

• Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable

• Therefore it can be important to prove that a problem is NP-Complete

• Don’t need to come up with an efficient algorithm

• Can instead work on approximation algorithms

• What steps do we have to take to prove a problem Pis NP-Complete?

• Pick a known NP-Complete problem Q

• Reduce Q to P

• Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q

• Prove the transformation works

• Prove it runs in polynomial time

• Oh yeah, prove P NP (What if you can’t?)

it is very difficult for me to check whether you are right.

give me a solution, it is very easy for me to verify whether

you are right.

Loosely speaking, problems that are difficult to compute, but

easy to verify are known as Non-deterministic Polynomial.

Any NP problem can be converted to SATin polynomial time.

• One of the first problems to be proved NP-Complete was satisfiability (SAT):

• Given a Boolean expression on n variables, can we assign values such that the expression is TRUE?

• Ex: ((x1x2)  ((x1 x3)  x4)) x2

• Cook’s Theorem: The satisfiability problem is NP-Complete

• Note: Argue from first principles, not reduction

• Proof: not here

• Even if the form of the Boolean expression is simplified, the problem may be NP-Complete

• Literal: an occurrence of a Boolean or its negation

• A Boolean formula is in conjunctive normal form, or CNF, if it is an AND of clauses, each of which is an OR of literals

• Ex: (x1  x2)  (x1  x3  x4)  (x5)

• 3-CNF: each clause has exactly 3 distinct literals

• Ex: (x1  x2  x3)  (x1  x3  x4)  (x5  x3  x4)

• Notice: true if at least one literal in each clause is true

• Satisfiability of Boolean formulas in 3-CNF form (the 3-CNF Problem) is NP-Complete

• Proof: Nope

• The reason we care about the 3-CNF problem is that it is relatively easy to reduce to others

• Thus by proving 3-CNF NP-Complete we can prove many seemingly unrelated problems NP-Complete

3-CNF  Clique

• What is acliqueof a graph G?

• A: a subset of vertices fully connected to each other, i.e. a complete subgraph of G

• The clique problem: how large is the maximum-size clique in a graph?

• Can we turn this into a decision problem?

• A: Yes, we call this the k-clique problem

• Is the k-clique problem within NP?

this graph contains a 4-clique

3-CNF  Clique

• What should the reduction do?

• A: Transform a 3-CNF formula to a graph, for which a k-clique will exist (for some k) iff the 3-CNF formula is satisfiable

3-CNF  Clique

• The reduction:

• Let B = C1  C2  …  Ck be a 3-CNF formula with k clauses, each of which has 3 distinct literals

• For each clause put a triple of vertices in the graph, one for each literal

• Put an edge between two vertices if they are in different triples and their literals are consistent, meaning not each other’s negation

• Run an example: B = (x  y  z)  (x  y  z )  (x  y  z )

3-CNF  Clique

• Prove the reduction works:

• If B has a satisfying assignment, then each clause has at least one literal (vertex) that evaluates to 1

• Picking one such “true” literal from each clause gives a set V’ of k vertices. V’ is a clique (Why?)

• If G has a clique V’ of size k, it must contain one vertex in each triple (clause) (Why?)

• We can assign 1 to each literal corresponding with a vertex in V’, without fear of contradiction

Clique  Vertex Cover

• A vertex cover for a graph G is a set of vertices incident to every edge in G

• The vertex cover problem: what is the minimum size vertex cover in G?

• Restated as a decision problem: does a vertex cover of size k exist in G?

• Thm : vertex cover is NP-Complete

Clique  Vertex Cover

• First, show vertex cover in NP(How?)

• Next, reduce k-clique to vertex cover

• The complement GC of a graph G contains exactly those edges not in G

• Compute GC in polynomial time

• G has a clique of size k iff GC has a vertex cover of size |V| - k

Clique  Vertex Cover

• Claim: If G has a clique of size k,GC has a vertex cover of size |V| - k

• Let V’ be the k-clique

• Then V - V’ is a vertex cover in GC

• Let (u,v) be any edge in GC

• Then u and v cannot both be in V’ (Why?)

• Thus at least one of u or v is in V-V’ (why?), so edge (u, v) is covered by V-V’

• Since true for any edge in GC, V-V’ is a vertex cover

Clique  Vertex Cover

• Claim: If GC has a vertex cover V’  V, with |V’| = |V| - k, then G has a clique of size k

• For all u,v V, if (u,v)  GC then u  V’ or v  V’ or both (Why?)

• Contrapositive: if u  V’ and v  V’, then (u,v)  E

• In other words, all vertices in V-V’ are connected by an edge, thus V-V’ is a clique

• Since |V| - |V’| = k, the size of the clique is k

• Literally hundreds of problems have been shown to be NP-Complete

• Some reductions are profound, some are comparatively easy, many are easy once the key insight is given

• You can expect a simple NP-Completeness proof on the final

Vertex Cover (VC)

Instance: Graph G=(V,E) and integer k

Question: Does there exist a vertex cover of size at most k?

(V’ V is a vertex cover if for each (u,v)E, either uV’or vV’).

Independent Set (IND)

Instance: Graph G=(V,E) and integer k

Question: Does G have an independent set of size at least k?

(V’ V is an independent set if for any u,vV’, (u,v)E.)

Clique

Instance: Graph G=(V,E) and integer k

Question: Does G have a clique of size at least k?

(V’ V is a clique if for any u,vV’, (u,v)E.)

Hamiltonian Path (HP)

Instance: Graph G=(V,E) and integer k

Question: Does there exist a Hamiltonian Path of G?

That is, does  a simple path of length |V| - 1?

3 Color Problem (3COL)

Instance: Graph G=(V,E)

Question: Can we color the nodes of G with three three colors such that no two adjacent nodes of G have the same color?

Subset-sum

Instance: Given a set of integers

Question: does there exist a subset that adds up to some target T?

Partition

Bin Packing

Integer Linear Programming (ILP)